CN106067021B - A kind of house refuse target identification system of human assistance - Google Patents
A kind of house refuse target identification system of human assistance Download PDFInfo
- Publication number
- CN106067021B CN106067021B CN201610364253.6A CN201610364253A CN106067021B CN 106067021 B CN106067021 B CN 106067021B CN 201610364253 A CN201610364253 A CN 201610364253A CN 106067021 B CN106067021 B CN 106067021B
- Authority
- CN
- China
- Prior art keywords
- information
- human assistance
- identification
- straight line
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/255—Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
A kind of house refuse target identification system of human assistance, human assistance processing module is continuously displayed to every frame image is received, manually straight line, the object local flatness coordinate information and angle information that human assistance processing module identifies are marked on needing the object sorted on the image of display；The information and image information that will identify that simultaneously are sent to computer generalization processing unit；Computer generalization processing unit is shown to received image and is shown the straight line of artificial stroke, and straight line is translated according to known object movement speed；Data information judges whether the information of human assistance processing module identification is accurate based on the received, it will finally confirm that accurate information is transformed under image coordinate system by local coordinate, obtain the information of the object of this identification, reject the information with the duplicate object of last time identification of this identification, the information of final object is obtained, and is highlighted on the object finally identified in display image and marks straight line.
Description
Technical field
The present invention relates to a kind of house refuse target identification systems of human assistance.
Background technique
In machine vision technique, existing image-recognizing method usually extracts certain feature, obtains
Feature is compared with ideal value, the high conduct recognition result of alignment similarity.
Field is sorted in multiple target, the target category for needing to identify, feature type are very more, such as various shape, color
Bottle, jar etc. extract manifold method since the calculation amount of feature extraction is very big and be not able to satisfy in real-time and wanted
It asks；And the accuracy rate of the method for feature extraction is not high always, on automatic industrial manufacturing line, is difficult to meet requirement.
The present invention is directed sorting consumer waste identifies field, house refuse site environment is severe, complicated composition are various,
Object is easy to be blocked by other non-targeted objects and object is easy to mix other compositions by contamination, and object shape is caused to believe
Breath, colouring information do not protrude, cannot be identified using conventional image processing method or material identification method.
Summary of the invention
Technology of the invention solves the problems, such as: overcome the deficiencies in the prior art proposes a kind of house refuse of human assistance
Target identification system.
The technical solution of the invention is as follows: a kind of house refuse target identification system of human assistance, including artificial auxiliary
Help processing module and computer generalization processing unit；
Human assistance processing module real-time reception samples the optical imagery of camera shooting, carries out continuously to every frame image is received
It has been shown that, by the object manually according to the concrete condition of the optical imagery of shooting, manually sorted on the image of display in needs
On mark straight line, human assistance processing module automatically obtains the temporal information under the local coordinate for clicking the screen moment,
The beginning and end of the straight line gone out using artificial stroke calculates what average value was identified as human assistance processing module as starting point
Object local flatness coordinate information, the artificial auxiliary processing module of straight slope angle marked with human assistance identify
Angle information；The information and image information that will identify that simultaneously are sent to computer generalization processing unit；
Computer generalization processing unit is shown to received image and is shown the straight line of artificial stroke, and straight line according to
Known object movement speed is translated；Data information judges the information of human assistance processing module identification based on the received
It is whether accurate, it will finally confirm that accurate information is transformed under image coordinate system by local coordinate, obtain the mesh of this identification
The information for marking object rejects the information with the duplicate object of last time identification of this identification, obtains final object
Information, and highlighted on the object finally identified in display image and mark straight line, what which was identified with human assistance
Straight line is that raw information binding time information is highlighted progress to target tracking.
Specific step is as follows for human assistance processing module:
(1) human assistance processing module acquires in real time and shows house refuse image information；
(2) on the image as far as possible close to the location point on object long side vertex after the interested object of artificial discovery
For starting point, straight line is marked as terminal using the location point as far as possible close to object long side bottom point, during stroke finger not from
Curtain spread its tail until bottom point；
(3) human assistance processing module is to calculate with the terminal of artificial stroke using the starting point of artificial stroke as zequin
Endpoint calculation goes out the object local flatness coordinate information that the equalization point of two o'clock is identified as human assistance；
The specific implementation steps are as follows for computer generalization processing unit:
(1) straight line of artificial stroke is shown and shown to received image, and straight line is moved according to known object
Dynamic speed is translated；
(2) following processing is carried out to the information of received all objects: with received a certain object local flatness
Coordinate information is geometric center, is identified in the region of object average area with twice expection and carries out Hough Straight Line Identification, [0
Degree, 180 degree] in 10 degree of interval statistics straight lines slope angle, calculate the maximum area of the straight slope angle frequency of occurrences
Between, using the average angle in section as the angle of identified object；
(3) angle obtained in step (2) is compared with the angle information that human assistance processing module identifies, such as
Two object angular deviation of fruit is less than preset threshold value, then it is assumed that is that human assistance identification is correct, otherwise pops up dialog box and wait
Manually determine whether that confirmation is clicked correctly, if it is thinks that human assistance identification is correct, otherwise abandon this and identify and cancel
The respective straight of step (1) is shown；Until received all object information all handle completion, the object of this identification is obtained
Information；
(4) on the basis of the target that human assistance clicks on current frame image, a human assistance is successively obtained
Target identical with datum target ordinate on frame image is clicked, according to the movement velocity v and two frame times of object when shooting
Difference calculates the theoretical position of target corresponding with datum target on a upper artificial auxiliary clicking frame image, calculates theoretical position
With the Euclidean distance of all targets on upper one artificial auxiliary clicking frame image, the minimum corresponding target of Euclidean distance is and benchmark
The identical target of target, and the straight line that the target for repeating to click reject while deleting step (1) is shown.
Compared with the prior art, the invention has the advantages that:
(1) the house refuse target identification system of a kind of human assistance of the invention can overcome above-mentioned environmental background multiple
Miscellaneous, the unconspicuous disadvantage of object feature introduces human assistance and knows the artificial separate sorting consumer waste scene that allows otherwise
In the case of complete house refuse in object identification.
(2) human assistance knowledge solves pure Computer Image Processing mode otherwise and is unable to complete complicated non-structural background
The shortcomings that object identifies；
(3) computer that reduces of human assistance identification method is identified to camera, camera lens, luminous environment, computer disposal energy
The requirement of power etc., effectively reduces cost；
(4) human assistance processing module be combined with each other with computer generalization processing module, has both reduced computer identification
Algorithm difficulty solves the problems, such as that pure manual identified is easy error and repeats to identify, system structure more optimizes.
Detailed description of the invention
Fig. 1 is present system block diagram；
Fig. 2 is that physical structure of the present invention implements block diagram；
Fig. 3 is that the present invention manually assists in identifying module flow diagram；
Fig. 4 is computer generalization processing module flow chart of the present invention；
Fig. 5 is that the present invention picks out repetition target flow chart；
Specific embodiment
It elaborates with reference to the accompanying drawings and embodiments to the present invention.
As shown in Figure 1, 2, present system includes human assistance processing module and computer generalization processing module；According to figure
Physical structure in 2, the article on camera captured in real-time conveyer belt obtain visual pattern, and man-machine auxiliary processing module can be with
By the way of currently used touch screen computer, human assistance processing module, computer generalization processing unit are all operated in
On industrial personal computer, the identification to object on conveyer belt is completed in various pieces combination.Every part is described in detail separately below.
(1) human assistance processing module
Human assistance processing module real-time reception samples the optical imagery of camera shooting, carries out continuously to every frame image is received
It has been shown that, by the object manually according to the concrete condition of the optical imagery of shooting, manually sorted on the image of display in needs
On mark straight line, human assistance processing module automatically obtains the temporal information under the local coordinate for clicking the screen moment,
The beginning and end of the straight line gone out using artificial stroke calculates what average value was identified as human assistance processing module as starting point
Object local flatness coordinate information, the artificial auxiliary processing module of straight slope angle marked with human assistance identify
Angle information.The straight line information that manual identified goes out is shown on the image, and is moved accordingly according to known image movement speed
The position of straight line, while it is comprehensive that information and image information that above- mentioned information human assistance identification module identifies is sent to computer
Processing unit is closed, above-mentioned human assistance identification process is as shown in Figure 3.Local coordinate refers to calculating the area of display image screen
The pixel coordinate system in domain, using first, screen upper left corner pixel as coordinate origin, horizontal direction is abscissa, and vertical direction is vertical
Coordinate；Image coordinate system refers to the coordinate system in image recognition region shown in Fig. 2, describes single pixel in entire image
Position coordinates, coordinate origin are located at the pixel in the image upper left corner, and coordinate is (0,0), and image coordinate system horizontal axis is level side
To maximum value 1600, the image coordinate system longitudinal axis is vertical direction, and maximum value 1200, i.e. picture traverse are 1600 × 1200.
(2) computer generalization processing unit
Computer generalization processing unit is shown to received image and is shown the straight line of artificial stroke, and straight line according to
Known object movement speed is translated, and the mobile speed of object is known fixed speed.With above-mentioned human assistance
O'clock angle recognition is carried out in the region of twice of object area centered on the plane coordinates identified, and to the target identified
The object angle information that object angle degree goes out with manual identified compares, and is considered artificial if error is less than the threshold value of setting
Auxiliary identification information is that correctly, which is the object information of principal and interest identification, and otherwise computer generalization handles mould
Block is reminded by way of popping up dialog box manually to be confirmed, it also hold that should if manual confirmation is effective object
Target information is the target information of this identification, and manual confirmation is not that effective object then abandons the identification of this human assistance,
And cancels and being shown about the straight line of the target.Computer generalization treatment process is as shown in Figure 4.To all objects received
Object information after information all carries out above-mentioned processing, after obtaining this recognition and verification；
The method of computer generalization processing module identification angle is, using the image coordinate manually clicked as geometric center, with
Twice expection, which is identified in the region of object average area, carries out Hough Straight Line Identification, 10 areas Du Yige in [0 degree, 180 degree]
Between count the slope angle of straight line, calculate the maximum section of the straight slope angle frequency of occurrences, made with the average angle in section
Manually to assist in identifying angle out, the temporal information that manually assists in identifying at the time of manually to click.
Coordinate information under object information local coordinate after above-mentioned confirmation is transformed by computer generalization processing unit
Under image coordinate system, obtain the information of final object, judge this identification object coordinate information whether last time identify
Object information out has repetition and rejects to duplicate information, and on the object finally identified in display image
Highlighted to mark straight line, the straight line which is identified using human assistance is raw information binding time information to target tracking height
Bright display carries out.The mobile speed of object is known fixed speed.
It is as follows to reject the step of repeating target information: using a target on human assistance click current frame image as base
Standard successively obtains target identical with datum target ordinate on an artificial auxiliary clicking frame image, according to mesh when shooting
Movement velocity v and two frame times for marking object are poor, calculate mesh corresponding with datum target on a upper artificial auxiliary clicking frame image
Target theoretical position calculates the Euclidean distance of theoretical position with all targets on upper one artificial auxiliary clicking frame image, Euclidean
The minimum corresponding target of distance is target identical with datum target, and the target for repeating to click reject while deleting and
Its corresponding straight line is shown.
Rejecting repetition target information can also be carried out using step as shown in Figure 5, specific as follows:
(1) coordinate information, angle information and the temporal information of object are obtained；
(2) identified target target as a comparison is obtained from upper one artificial auxiliary clicking frame image, obtains comparison
Coordinate information, angle information and the temporal information of target；
(3) judge object and compare the difference of the Y-coordinate of target, if difference is less than the 10% of comparison target Y-coordinate value,
Then follow the steps (4)；Otherwise comparison target is reacquired from upper one artificial auxiliary clicking frame image, is re-execute the steps
(3), until comparing completion with all targets on upper one artificial auxiliary clicking frame image；
(4) judge whether the angle difference of object and comparison target is less than the 30% of comparison target angle angle value, if being less than,
(5) are thened follow the steps, comparison target is otherwise reacquired from upper one artificial auxiliary clicking frame image, is re-execute the steps
(3), until comparing completion with all targets on upper one artificial auxiliary clicking frame image；
(5) poor according to the movement velocity v of object when shooting and two frame times, object X-coordinate information is calculated upper one
Frame compares the X-coordinate information at target identification moment, and the X-coordinate information of calculating and the X information of comparison target are made the difference, judge difference
Whether the 20% of comparison target X-coordinate value is less than, if being less than, the object and comparison target are same target, by the target
It carries out rejecting while deleting corresponding straight line and show；Otherwise the reacquisition pair from upper one artificial auxiliary clicking frame image
Than target, (3) are re-execute the steps, until comparing completion with all targets on previous frame image.
For from identification bottle target in the rubbish on conveyer belt, when manually emerging by the image discovery sense on touch screen
When the bottle of interest, bottle corresponding position on artificial stroke touch screen, computer carries out above-mentioned processing according to stroke movement and identifies
The target information of human assistance identification, and shown in a manner of highlighted, human assistance processing module believes corresponding coordinate
Breath is sent to computer generalization processing unit.Computer generalization processing unit calculates object according to above-mentioned angle recognition method
Angle, if the angle and human assistance identification angular error it is larger if display image on pop up dialog box, prompter
Work be confirmed whether being object, if it is confirmed that being, the information that above-mentioned human assistance is identified carries out subsequent changes in coordinates
Equal operations form final object information.
Unspecified part of the present invention belongs to common sense well known to those skilled in the art.
Claims (3)
1. a kind of house refuse target identification system of human assistance, it is characterised in that including human assistance processing module and calculating
Machine integrated treatment unit；
Human assistance processing module real-time reception samples the optical imagery of camera shooting, is continuously shown to received every frame image
Show, by manually according to the concrete condition of the optical imagery of shooting, manually on the image of display on the object for needing to sort
Straight line is marked, human assistance processing module automatically obtains the temporal information under the local coordinate for clicking the screen moment, with
The beginning and end for the straight line that artificial stroke goes out is the mesh that starting point calculates that average value is identified as human assistance processing module
Object local flatness coordinate information is marked, the straight slope angle marked with human assistance is the angle that artificial auxiliary processing module identifies
Spend information；The information and image information that will identify that simultaneously are sent to computer generalization processing unit；
Computer generalization processing unit is shown to received image and is shown the straight line of artificial stroke, and straight line is according to known
Object movement speed translated；Based on the received data information judge human assistance processing module identification information whether
Accurately, it will finally confirm that accurate information is transformed under image coordinate system by local coordinate, obtain the object of this identification
Information, reject this identification with last time identification duplicate object information, obtain the information of final object,
And highlighted on the object finally identified in display image and mark straight line, which is with the straight line that human assistance identifies
Raw information binding time information is highlighted target tracking；
Wherein, data information judges whether the information of human assistance processing module identification is accurate based on the received, to received institute
There is the information of object to be carried out following processing: using received a certain object local flatness coordinate information as geometric center, with
Twice expection, which is identified in the region of object average area, carries out Hough Straight Line Identification, 10 areas Du Yige in [0 degree, 180 degree]
Between count straight line slope angle, the maximum section of the straight slope angle frequency of occurrences is calculated, with the average angle in the section
Angle as identified object；The angle information that the angle of the identified object and human assistance processing module are identified carries out
It compares, if two object angular deviations are less than preset threshold value, then it is assumed that be that human assistance identification is correct.
2. a kind of house refuse target identification system of human assistance according to claim 1, it is characterised in that: artificial auxiliary
Helping processing module, specific step is as follows:
(1) human assistance processing module acquires in real time and shows house refuse image information；
(2) it is with the location point as far as possible close to object long side vertex on the image after the interested object of artificial discovery
Point, to mark straight line as terminal close to the location point of object long side bottom point as far as possible, finger is without departing from screen during stroke
Curtain is until bottom point；
(3) human assistance processing module is to calculate terminal with the terminal of artificial stroke using the starting point of artificial stroke as zequin
Calculate the object local flatness coordinate information that the equalization point of two o'clock is identified as human assistance.
3. a kind of house refuse target identification system of human assistance according to claim 1, it is characterised in that: computer
The specific implementation steps are as follows for integrated treatment unit:
(1) straight line of artificial stroke is shown and shown to received image, and straight line is according to the mobile speed of known object
Degree is translated；
(2) following processing is carried out to the information of received all objects: with received a certain object local flatness coordinate
Information is geometric center, is identified in the region of object average area with twice expection and carries out Hough Straight Line Identification, [0 degree,
180 degree] in 10 degree of interval statistics straight lines slope angle, calculate the maximum section of the straight slope angle frequency of occurrences,
Using the average angle in the section as the angle of identified object；
(3) angle obtained in step (2) is compared with the angle information that human assistance processing module identifies, if two
Object angular deviation is less than preset threshold value, then it is assumed that is that human assistance identification is correct, it is artificial otherwise to pop up dialog box waiting
Determine whether that confirmation is clicked correctly, if it is thinks that human assistance identification is correct, otherwise abandon this identification and cancellation step
(1) respective straight is shown；Until received all object information all handle completion, the letter of the object of this identification is obtained
Breath；
(4) on the basis of the target that human assistance clicks on current frame image, an artificial auxiliary clicking is successively obtained
Target identical with datum target ordinate on frame image, meter poor according to the movement velocity v of object when shooting and two frame times
It counts the theoretical position of target corresponding with datum target on an artificial auxiliary clicking frame image in, calculates theoretical position and upper one
The Euclidean distance of all targets on a artificial auxiliary clicking frame image, the minimum corresponding target of Euclidean distance are and datum target phase
Same target, and the straight line that the target for repeating to click reject while deleting step (1) is shown.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610364253.6A CN106067021B (en) | 2016-05-26 | 2016-05-26 | A kind of house refuse target identification system of human assistance |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610364253.6A CN106067021B (en) | 2016-05-26 | 2016-05-26 | A kind of house refuse target identification system of human assistance |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106067021A CN106067021A (en) | 2016-11-02 |
| CN106067021B true CN106067021B (en) | 2019-05-24 |
Family
ID=57420782
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610364253.6A Active CN106067021B (en) | 2016-05-26 | 2016-05-26 | A kind of house refuse target identification system of human assistance |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106067021B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2000237689A (en) * | 1999-02-18 | 2000-09-05 | Takuma Co Ltd | Shape sorting machine |
| CN102063726A (en) * | 2010-12-31 | 2011-05-18 | 中国科学院计算技术研究所 | Moving target classification method and system |
| CN102609934A (en) * | 2011-12-22 | 2012-07-25 | 中国科学院自动化研究所 | Multi-target segmenting and tracking method based on depth image |
| CN103143509A (en) * | 2013-03-21 | 2013-06-12 | 电子科技大学 | Garbage classification robot and garbage identification and classification method |
| CN103194991A (en) * | 2013-04-03 | 2013-07-10 | 西安电子科技大学 | Road cleaning system and method through intelligent robot |
| CN103522291A (en) * | 2013-10-29 | 2014-01-22 | 中国人民解放军总装备部军械技术研究所 | Target capturing system and method of explosive ordnance disposal robot |
| CN105518702A (en) * | 2014-11-12 | 2016-04-20 | 深圳市大疆创新科技有限公司 | Method, device and robot for detecting target object |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9870617B2 (en) * | 2014-09-19 | 2018-01-16 | Brain Corporation | Apparatus and methods for saliency detection based on color occurrence analysis |
-
2016
- 2016-05-26 CN CN201610364253.6A patent/CN106067021B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2000237689A (en) * | 1999-02-18 | 2000-09-05 | Takuma Co Ltd | Shape sorting machine |
| CN102063726A (en) * | 2010-12-31 | 2011-05-18 | 中国科学院计算技术研究所 | Moving target classification method and system |
| CN102609934A (en) * | 2011-12-22 | 2012-07-25 | 中国科学院自动化研究所 | Multi-target segmenting and tracking method based on depth image |
| CN103143509A (en) * | 2013-03-21 | 2013-06-12 | 电子科技大学 | Garbage classification robot and garbage identification and classification method |
| CN103194991A (en) * | 2013-04-03 | 2013-07-10 | 西安电子科技大学 | Road cleaning system and method through intelligent robot |
| CN103522291A (en) * | 2013-10-29 | 2014-01-22 | 中国人民解放军总装备部军械技术研究所 | Target capturing system and method of explosive ordnance disposal robot |
| CN105518702A (en) * | 2014-11-12 | 2016-04-20 | 深圳市大疆创新科技有限公司 | Method, device and robot for detecting target object |
Non-Patent Citations (2)
| Title |
|---|
| 基于机器视觉的工业机器人分拣技术研究;赵彬;《中国优秀硕士学位论文全文数据库》;20131231;第1-71页 |
| 工业机器人分拣技术的实现;焦恩璋 等;《控制与检测》;20100228(第2期);第84-87页 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106067021A (en) | 2016-11-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109724990B (en) | Method for quickly positioning and detecting code spraying area in label of packaging box | |
| CN111496770B (en) | Intelligent carrying mechanical arm system based on 3D vision and deep learning and use method | |
| CN106022386B (en) | A kind of computer identifies the house refuse target identification system in conjunction with man-machine interactively | |
| CN102542289B (en) | Pedestrian volume statistical method based on plurality of Gaussian counting models | |
| CN105225225B (en) | A kind of leather system for automatic marker making method and apparatus based on machine vision | |
| CN111199556B (en) | Indoor pedestrian detection and tracking method based on camera | |
| CN109785337A (en) | Mammal counting method in a kind of column of Case-based Reasoning partitioning algorithm | |
| CN101727654B (en) | Method realized by parallel pipeline for performing real-time marking and identification on connected domains of point targets | |
| CN106204614A (en) | A kind of workpiece appearance defects detection method based on machine vision | |
| CN112518748B (en) | Automatic grabbing method and system for visual mechanical arm for moving object | |
| CN110108712A (en) | Multifunctional visual sense defect detecting system | |
| CN104458748A (en) | Aluminum profile surface defect detecting method based on machine vision | |
| CN106067031B (en) | Based on artificial mechanism for correcting errors and deep learning network cooperation machine vision recognition system | |
| CN102214291A (en) | Method for quickly and accurately detecting and tracking human face based on video sequence | |
| CN105930795A (en) | Walking state identification method based on space vector between human body skeleton joints | |
| CN104217428A (en) | Video monitoring multi-target tracking method for fusion feature matching and data association | |
| CN104077596A (en) | Landmark-free tracking registering method | |
| CN112102368B (en) | Deep learning-based robot garbage classification and sorting method | |
| CN107146239A (en) | Satellite video moving target detecting method and system | |
| Momeni-k et al. | Height estimation from a single camera view | |
| CN105690393A (en) | Four-axle parallel robot sorting system based on machine vision and sorting method thereof | |
| CN109685827B (en) | Target detection and tracking method based on DSP | |
| CN111715559A (en) | Garbage sorting system based on machine vision | |
| CN102831408A (en) | Human face recognition method | |
| CN104299246A (en) | Production line object part motion detection and tracking method based on videos |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |