CN114758139B - Method for detecting accumulated water in foundation pit - Google Patents
Method for detecting accumulated water in foundation pit Download PDFInfo
- Publication number
- CN114758139B CN114758139B CN202210678384.7A CN202210678384A CN114758139B CN 114758139 B CN114758139 B CN 114758139B CN 202210678384 A CN202210678384 A CN 202210678384A CN 114758139 B CN114758139 B CN 114758139B
- Authority
- CN
- China
- Prior art keywords
- image
- foundation pit
- image data
- feature extraction
- texture
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
- G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
- G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Abstract
The invention provides a foundation pit accumulated water detection method, which comprises the following steps of S1: acquiring a foundation pit water accumulation image for marking, and taking the foundation pit water accumulation image as basic training image data; s2: extracting the characteristics of the obtained image data; s3: inputting the extracted characteristic data into a convolutional neural network model for training; s4: acquiring image data to be detected, inputting the image data into the trained network model, and outputting a detection result; s5: and setting a time period, acquiring video image data of the set time period, processing to obtain the time-dependent change process of the pixel value of the highlight area, performing frequency spectrum analysis, and further performing foundation pit water accumulation judgment through fluctuation characteristics. The method and the device realize accurate identification and detection aiming at the accumulated water image of the foundation pit in the construction site.
Description
Technical Field
The invention relates to the technical field of foundation pit accumulated water image processing, in particular to a foundation pit accumulated water detection method.
Background
Convolutional Neural Networks (CNNs) are a class of feed forward Neural Networks (fed Neural Networks) that include convolution computations and have a deep structure, and are one of the representative algorithms of deep learning (deep learning). Convolutional Neural Networks have a feature learning (rendering) capability, and can perform Shift-Invariant classification (Shift-Invariant classification) on input information according to a hierarchical structure thereof, and thus, are also called "Shift-Invariant Artificial Neural Networks (SIANN)".
Image recognition, which refers to a technique for processing, analyzing and understanding images by a computer to recognize various different patterns of objects and objects, is a practical application of applying a deep learning algorithm. Image recognition technology at present is generally divided into face recognition and commodity recognition, and the face recognition is mainly applied to security inspection, identity verification and mobile payment; the commodity identification is mainly applied to the commodity circulation process, in particular to the field of unmanned retail such as unmanned goods shelves and intelligent retail cabinets. The traditional image identification process is divided into four steps: image acquisition → image preprocessing → feature extraction → image recognition. And performing image identification, namely performing feature extraction and classification on the image obtained by image processing. The basic and common methods in the recognition method are statistical (or decision theory), syntactic (or structural), neural network, template matching, and geometric transformation.
Foundation engineering is the foundation of projects, and foundation pit safety is of utmost importance, especially in rainy seasons. Foundation ditch ponding can restrict the engineering and progress, and more serious condition ponding can permeate the earthwork root, arouses the large face collapse, causes the incident to influence whole pit safety, and the quick foundation ditch ponding condition of discerning can effectively help the building site to draw up the solution according to the ponding condition, reduces the engineering loss, and nevertheless foundation ditch water is because of optical characteristic in the reality for foundation ditch ponding discernment has the difficulty, among the prior art, has not yet had the scheme to the effective detection discernment of foundation ditch ponding.
How to quickly, accurately and effectively identify the foundation pit water accumulation is a problem to be solved urgently in the field of foundation engineering application.
Disclosure of Invention
In order to solve the problems, the invention provides a foundation pit water accumulation detection method, which is based on image processing and deep learning, analyzes and processes a picture shot by a construction site camera, identifies whether water accumulation exists in a foundation pit or not and the water accumulation condition according to the characteristics of water and the outline of a water area, extracts color characteristics and texture characteristics and removes a sky part area based on foundation pit water accumulation image data to obtain characteristic data capable of accurately describing a foundation pit water accumulation image, solves the problems of low resolution, high noise and the like of a water body label by utilizing the deep learning technology, improves the water accumulation detection and identification precision, and realizes accurate and effective detection and identification of the foundation pit water accumulation.
The invention provides a method for detecting accumulated water in a foundation pit, which has the following specific technical scheme:
s1: acquiring a foundation pit water accumulation image for marking, and taking the foundation pit water accumulation image as basic training image data;
s2: extracting the characteristics of the obtained image data;
s3: inputting the extracted characteristic data into a convolutional neural network model for training;
s4: and acquiring image data to be detected, inputting the image data into the trained network model, and outputting a detection result.
Further, in step S2, color feature extraction and texture feature extraction are performed on the obtained static water surface features in the foundation pit water image.
Further, the color feature extraction specifically comprises the following steps:
carrying out gray processing on the foundation pit accumulated water image;
determining a different brightness area in the water area of the foundation pit ponding image compared with surrounding construction site scenery;
and carrying out self-adaptive gray threshold segmentation on the acquired gray level image to obtain a highlight area in the image.
Further, after the color feature extraction is performed with the adaptive gray threshold segmentation, whether a sky part exists in a water body region of the obtained image is judged, and if yes, the sky part in the image is removed.
Further, the process of removing the sky part in the image is as follows:
scanning a line of pixels at the top of the image after the self-adaptive gray threshold segmentation in sequence, if high-brightness pixel points exist, carrying out four-connected region growing on the pixel points, detecting whether high-brightness pixel points exist around the pixel points, if so, carrying out four-connected region growing on the pixel points again, if not, stopping growing, and after all the pixel points stop growing, judging that the growing region is a sky part in the image, and removing the sky part.
Further, the extraction of the texture features is based on a K-means clustering method to extract weak texture regions in the image.
Further, the extraction of the texture features extracts the average value of the features of the four texture feature values in the four directions of 0 °, 45 °, 90 ° and 135 ° as the feature value of the corresponding feature.
Further, in step S2, after the extracting the texture feature, the method further includes:
converting the picture into an HSV color space, dividing the picture into subblocks with the size of 8x8, and extracting the saturation brightness ratio color characteristic of each subblock;
and (3) compressing the gray level and extracting 4 texture characteristic values obtained by calculating the gray level co-occurrence matrix from each sub-block sample to obtain characteristic data.
Further, after step S4, the method further includes:
s5: and setting a time period, acquiring video image data in the set time period, processing to obtain the time-dependent change process of the pixel value of the highlight area, performing frequency spectrum analysis, and further performing foundation pit water accumulation judgment through fluctuation characteristics.
The invention has the following beneficial effects:
acquiring characteristic data capable of accurately describing a foundation pit ponding image by acquiring a foundation pit static image and extracting and processing color characteristics and texture characteristics, inputting the acquired characteristic data into a convolutional neural network model, and performing image identification detection to realize accurate detection on whether ponding exists in a foundation pit on a construction site;
based on the actual environment of building site to and the actual light condition when acquireing foundation ditch image behind the rain, the foundation ditch ponding image of gathering mostly has the sky part, influences subsequent ponding and detects discernment, through the high bright characteristic of sky part, carries out getting rid of sky part to the image, has improved the accuracy that subsequent ponding detected the discernment.
According to the obtained characteristic data, after the characteristic data are input into the convolutional neural network for identification and detection, the image data in a period of time are obtained through the characteristic that the water surface has fluctuation, the water surface fluctuation characteristic is extracted, whether accumulated water exists in the foundation pit or not is further confirmed, and identification errors are reduced.
Drawings
FIG. 1 is a schematic flow chart of the method of example 1 of the present invention;
FIG. 2 is a schematic flow chart of the method of embodiment 2 of the present invention.
Detailed Description
In the following description, technical solutions in the embodiments of the present invention are clearly and completely described, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Example 1
The embodiment 1 of the invention discloses a foundation pit accumulated water detection method, which comprises the following specific steps as shown in figure 1:
s1: acquiring a foundation pit water accumulation image for marking, and taking the foundation pit water accumulation image as basic training image data;
the camera device of accessible job site itself carries out position and angular adjustment according to actual conditions for obtain foundation ditch image, need not to increase field device, practices thrift the cost.
S2: extracting the characteristics of the obtained image data;
and the characteristic extraction comprises color characteristic extraction and texture characteristic extraction, and the color characteristic extraction and the texture characteristic extraction are carried out on the static water surface characteristics in the obtained foundation pit ponding image.
The color feature extraction comprises the following specific processes:
carrying out gray level processing on the foundation pit accumulated water image;
determining a different brightness area in the water area of the foundation pit ponding image compared with surrounding construction site scenery;
and carrying out self-adaptive gray threshold segmentation on the acquired gray image to obtain a highlight area in the image.
In this embodiment, the color feature extraction further includes removing a sky part, in general, the brightness of the sky is the highest, and the sky part is likely to exist in a picture obtained from a construction site, so that it is necessary to remove the sky part;
after the color feature extraction is carried out with self-adaptive gray threshold segmentation, whether a sky part exists in a water body region of the obtained image is judged, and if yes, the sky part in the image is removed.
The process of removing the sky part in the image is as follows:
scanning the uppermost line of pixels of the image after the self-adaptive gray threshold segmentation in sequence, if high-brightness pixel points exist, carrying out four-connected region growing on the pixels, detecting whether high-brightness pixel points also exist around the pixels, if so, carrying out four-connected region growing on the pixels again, if not, stopping growing, and after all the pixel points stop growing, judging the growing region as the sky part in the image, determining the high-brightness region which is communicated based on the uppermost line of pixels as the sky, and removing the high-brightness region from the high-brightness region.
In the texture feature extraction, since the texture of the static water body is generally weak and is often strong compared with the texture of the surrounding worksite scene, in this embodiment, the weak texture region in the image is extracted based on a K-means clustering method.
In this embodiment, the texture feature extraction is to extract an average value of features of four texture feature values in four directions of 0 °, 45 °, 90 °, and 135 ° as a feature value of a corresponding feature.
In this embodiment, after the texture feature extraction, the method further includes:
converting the picture into an HSV color space, dividing the picture into subblocks with the size of 8x8, and extracting the saturation brightness ratio color characteristic of each subblock;
and (3) compressing the gray level and extracting 4 texture characteristic values obtained by calculating the gray level co-occurrence matrix from each sub-block sample to obtain characteristic data.
S3: inputting the extracted characteristic data into a convolutional neural network model for training;
the convolutional neural network can adopt the existing model frame structure, and the algorithm has a simple structure and is convenient to implement, and is not specifically limited herein.
S4: and acquiring image data to be detected, inputting the image data into the trained network model, and outputting a detection result.
Example 2
Embodiment 1 of the present invention discloses a method for detecting accumulated water in a foundation pit, as shown in fig. 2, steps S1 to S4 are based on embodiment 1 described above, and will not be repeated here,
in this embodiment, after the step S1-S4 of performing the foundation pit water accumulation detection to obtain the model output result, the method further includes:
s5: and setting a time period, acquiring video image data in the set time period, processing to obtain the time-dependent change process of the pixel value of the highlight area, performing frequency spectrum analysis, and further performing foundation pit water accumulation judgment through fluctuation characteristics.
The water surface fluctuates for a long time, so that whether the water body exists can be further accurately determined based on the water surface fluctuation characteristic for a period of time.
Specifically, video image data within a preset time are obtained, the video image data are processed, a change process of a pixel value of a highlight area along with time is obtained, spectrum analysis is carried out on the data, when point spectrums exceeding a preset proportion are similar and amplitude-frequency characteristics meet preset conditions, the highlight is judged to have volatility, and the highlight area is further determined to be water accumulation.
The invention is not limited to the foregoing embodiments. The invention extends to any novel feature or any novel combination of features disclosed in this specification and any novel method or process steps or any novel combination of features disclosed.
Claims (2)
1. A foundation pit accumulated water detection method is characterized by comprising the following steps:
s1: acquiring a foundation pit water accumulation image for marking, and taking the foundation pit water accumulation image as basic training image data;
s2: performing feature extraction on the obtained image data, including performing color feature extraction and texture feature extraction on the static water surface features in the obtained foundation pit ponding image;
the color feature extraction comprises the following specific steps:
carrying out gray level processing on the foundation pit accumulated water image;
determining a brightness area in the water area of the foundation pit ponding image, which is different from the surrounding site scenery;
carrying out self-adaptive gray threshold segmentation on the acquired gray image to obtain a highlight area in the image; after the color feature extraction is carried out with self-adaptive gray threshold segmentation, judging whether a sky part exists in a water body area of the obtained image, and if so, removing the sky part in the image;
the process of removing the sky part in the image is as follows:
scanning a line of pixels at the top of an image after self-adaptive gray threshold segmentation in sequence, if high-brightness pixel points exist, carrying out four-connected region growing on the pixels, detecting whether high-brightness pixel points exist around the pixels, if so, carrying out four-connected region growing on the pixels again, if not, stopping growing, and after all the pixel points stop growing, judging that the growing region is a sky part in the image, and removing the sky part;
extracting the texture features, namely extracting the average values of the features of the four texture feature values in four directions of 0 degree, 45 degrees, 90 degrees and 135 degrees as the feature values of the corresponding features;
after texture feature extraction, the method further comprises the following steps:
converting the picture into an HSV color space, dividing the picture into subblocks with the size of 8x8, and extracting the saturation brightness ratio color characteristic of each subblock;
compressing the gray level and extracting 4 texture characteristic values obtained by calculating the gray level co-occurrence matrix from each sub-block sample to obtain characteristic data;
s3: inputting the extracted characteristic data into a convolutional neural network model for training;
s4: acquiring image data to be detected, inputting the image data into the trained network model, and outputting a detection result;
s5: and setting a time period, acquiring video image data in the set time period, processing to obtain the time-dependent change process of the pixel value of the highlight area, performing frequency spectrum analysis, and further performing foundation pit water accumulation judgment through the volatility characteristic.
2. The method for detecting accumulated water in foundation pit according to claim 1, wherein the texture feature extraction is based on a K-means clustering method to extract weak texture regions in the image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210678384.7A CN114758139B (en) | 2022-06-16 | 2022-06-16 | Method for detecting accumulated water in foundation pit |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210678384.7A CN114758139B (en) | 2022-06-16 | 2022-06-16 | Method for detecting accumulated water in foundation pit |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114758139A CN114758139A (en) | 2022-07-15 |
| CN114758139B true CN114758139B (en) | 2022-10-21 |
Family
ID=82336525
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210678384.7A Active CN114758139B (en) | 2022-06-16 | 2022-06-16 | Method for detecting accumulated water in foundation pit |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114758139B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116740577B (en) * | 2023-08-10 | 2024-01-12 | 瞳见科技有限公司 | Road ponding identification method, system, terminal and storage medium |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102721451A (en) * | 2012-05-30 | 2012-10-10 | 吴江市精工铝字制造厂 | Foundation pit accumulated water monitoring system on basis of RFID (Radio Frequency Identification Device) technology |
| CN105809131A (en) * | 2016-03-08 | 2016-07-27 | 宁波裕兰信息科技有限公司 | Method and system for carrying out parking space waterlogging detection based on image processing technology |
| CN109670404A (en) * | 2018-11-23 | 2019-04-23 | 江苏理工学院 | A kind of road ponding image detection method for early warning based on mixed model |
| CN110569755A (en) * | 2019-08-26 | 2019-12-13 | 浙江工商大学 | Intelligent accumulated water detection method based on video |
| CN111598908A (en) * | 2020-04-24 | 2020-08-28 | 山东易华录信息技术有限公司 | Image analysis screening system and device |
| CN112255709A (en) * | 2020-10-21 | 2021-01-22 | 中铁第四勘察设计院集团有限公司 | Automatic lifting type flood monitoring device and method based on image recognition |
| CN112446304A (en) * | 2020-11-10 | 2021-03-05 | 成都鹏业软件股份有限公司 | Flame detection method and system thereof |
| CN112580497A (en) * | 2020-12-17 | 2021-03-30 | 上海依图智安人工智能科技有限公司 | Image recognition method and device |
| CN113469504A (en) * | 2021-06-09 | 2021-10-01 | 重庆交通大学 | Dynamic testing method, medium and system for water traffic risk |
| CN113832980A (en) * | 2021-11-08 | 2021-12-24 | 中建安装集团南京建设有限公司 | Integrated pump station type intelligent foundation pit drainage system |
| CN114399460A (en) * | 2021-11-30 | 2022-04-26 | 上汽大众汽车有限公司 | Method and system for detecting depth of accumulated water on road surface and vehicle |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108549880B (en) * | 2018-04-28 | 2021-06-25 | 深圳市商汤科技有限公司 | Collision control method and device, electronic equipment and storage medium |
| CN109900712B (en) * | 2019-04-17 | 2022-03-29 | 中国人民解放军国防科技大学 | Wind power blade mechanical fault on-line detection system and method based on camera measurement |
| CN111583229B (en) * | 2020-05-09 | 2024-01-05 | 江苏野马软件科技有限公司 | Road surface fault detection method based on convolutional neural network |
| CN213394400U (en) * | 2020-10-21 | 2021-06-08 | 中铁第四勘察设计院集团有限公司 | Automatic lifting type flood monitoring device based on image recognition |
| US20220132723A1 (en) * | 2020-11-02 | 2022-05-05 | Deere & Company | Agricultural characteristic confidence and control |
| AU2021101264A4 (en) * | 2021-03-11 | 2021-05-06 | East China University Of Technology | A foundation pit drainage system for flood control |
| CN113706523B (en) * | 2021-09-10 | 2023-08-15 | 华能广东海门港务有限责任公司 | Belt deviation and abnormal operation state monitoring method based on artificial intelligence technology |
| CN113936132B (en) * | 2021-12-16 | 2022-03-11 | 山东沃能安全技术服务有限公司 | Method and system for detecting water pollution of chemical plant based on computer vision |
| CN114229641B (en) * | 2021-12-20 | 2024-02-23 | 苏州汇川控制技术有限公司 | Method, apparatus, device and readable storage medium for determining water level in elevator hoistway |
-
2022
- 2022-06-16 CN CN202210678384.7A patent/CN114758139B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102721451A (en) * | 2012-05-30 | 2012-10-10 | 吴江市精工铝字制造厂 | Foundation pit accumulated water monitoring system on basis of RFID (Radio Frequency Identification Device) technology |
| CN105809131A (en) * | 2016-03-08 | 2016-07-27 | 宁波裕兰信息科技有限公司 | Method and system for carrying out parking space waterlogging detection based on image processing technology |
| CN109670404A (en) * | 2018-11-23 | 2019-04-23 | 江苏理工学院 | A kind of road ponding image detection method for early warning based on mixed model |
| CN110569755A (en) * | 2019-08-26 | 2019-12-13 | 浙江工商大学 | Intelligent accumulated water detection method based on video |
| CN111598908A (en) * | 2020-04-24 | 2020-08-28 | 山东易华录信息技术有限公司 | Image analysis screening system and device |
| CN112255709A (en) * | 2020-10-21 | 2021-01-22 | 中铁第四勘察设计院集团有限公司 | Automatic lifting type flood monitoring device and method based on image recognition |
| CN112446304A (en) * | 2020-11-10 | 2021-03-05 | 成都鹏业软件股份有限公司 | Flame detection method and system thereof |
| CN112580497A (en) * | 2020-12-17 | 2021-03-30 | 上海依图智安人工智能科技有限公司 | Image recognition method and device |
| CN113469504A (en) * | 2021-06-09 | 2021-10-01 | 重庆交通大学 | Dynamic testing method, medium and system for water traffic risk |
| CN113832980A (en) * | 2021-11-08 | 2021-12-24 | 中建安装集团南京建设有限公司 | Integrated pump station type intelligent foundation pit drainage system |
| CN114399460A (en) * | 2021-11-30 | 2022-04-26 | 上汽大众汽车有限公司 | Method and system for detecting depth of accumulated water on road surface and vehicle |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114758139A (en) | 2022-07-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106548182B (en) | Pavement crack detection method and device based on deep learning and main cause analysis | |
| CN107038416B (en) | Pedestrian detection method based on binary image improved HOG characteristics | |
| CN111046880A (en) | Infrared target image segmentation method and system, electronic device and storage medium | |
| CN111611643A (en) | Family type vectorization data obtaining method and device, electronic equipment and storage medium | |
| CN110555382A (en) | Finger vein identification method based on deep learning and Wasserstein distance measurement | |
| CN112149543B (en) | Building dust recognition system and method based on computer vision | |
| CN111445459A (en) | Image defect detection method and system based on depth twin network | |
| CN109949209B (en) | Rope detection and removal method based on deep learning | |
| CN111539330B (en) | Transformer substation digital display instrument identification method based on double-SVM multi-classifier | |
| CN112258426A (en) | Automatic scaffold image inclination correction method based on Mask RCNN | |
| CN108961262B (en) | Bar code positioning method in complex scene | |
| CN114758139B (en) | Method for detecting accumulated water in foundation pit | |
| CN110687122A (en) | Method and system for detecting surface cracks of ceramic tile | |
| CN115731493A (en) | Rainfall micro physical characteristic parameter extraction and analysis method based on video image recognition | |
| CN113313107A (en) | Intelligent detection and identification method for multiple types of diseases on cable surface of cable-stayed bridge | |
| CN115841633A (en) | Power tower and power line associated correction power tower and power line detection method | |
| CN111178405A (en) | Similar object identification method fusing multiple neural networks | |
| CN109165592B (en) | Real-time rotatable face detection method based on PICO algorithm | |
| CN111027564A (en) | Low-illumination imaging license plate recognition method and device based on deep learning integration | |
| CN113989604A (en) | Tire DOT information identification method based on end-to-end deep learning | |
| KR20080079798A (en) | Method of face detection and recognition | |
| CN112288726A (en) | Method for detecting foreign matters on belt surface of underground belt conveyor | |
| CN107480648B (en) | Method for detecting characters in natural scene | |
| Khan et al. | Shadow removal from digital images using multi-channel binarization and shadow matting | |
| CN110276260B (en) | Commodity detection method based on depth camera |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |