CN110852208B - Crowd density estimation method and readable storage medium - Google Patents
Crowd density estimation method and readable storage medium Download PDFInfo
- Publication number
- CN110852208B CN110852208B CN201911037521.3A CN201911037521A CN110852208B CN 110852208 B CN110852208 B CN 110852208B CN 201911037521 A CN201911037521 A CN 201911037521A CN 110852208 B CN110852208 B CN 110852208B
- Authority
- CN
- China
- Prior art keywords
- crowd
- density
- bbox
- pedestrian
- pixel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
- G06V20/53—Recognition of crowd images, e.g. recognition of crowd congestion
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Abstract
The invention discloses a crowd density estimation method, which comprises the following steps: acquiring video images of people; performing target detection on the video images of the crowd, marking a pedestrian by using a Bbox, wherein all the Bboxes form a Bbox set corresponding to the crowd; and according to the Bbox set, the crowd density is obtained by calculating the average occupied area. The beneficial effects of the invention are as follows: by automatically collecting the monitoring video and carrying out target detection, the crowd density is calculated by a method of calculating the average occupied area according to the detection result, so that the crowd density is automatically estimated, the manual workload is reduced, the efficiency and the estimation precision are improved, the number of people is combined with the occupied area, the estimation precision is improved, the comparability of density estimation is enhanced, and the comparison and classification are convenient. The invention also discloses a readable storage medium.
Description
Technical Field
The invention relates to the technical field of security monitoring, in particular to a crowd density estimation method and a readable storage medium.
Background
With the dramatic increase in urban population density, many public infrastructures often come up with short term peak traffic, and high crowding of people is prone to various incidents. Global abnormal behavior (e.g., too high crowd density) that may exist in group motion presents a significant safety hazard. Therefore, it is necessary to estimate the population density in public infrastructure and the like and further manage and coordinate the population density.
The existing crowd density estimation method mainly has the following problems that the manual monitoring method is time-consuming and labor-consuming, and the estimation accuracy cannot be ensured; the influence of the occupied area is ignored by adopting a people counting method, and the precision is low; the density estimation values are not strong in comparability under different scenes, and are inconvenient to compare and grade.
Disclosure of Invention
The present invention is based on the above-mentioned problems, and provides a crowd density estimation method and a readable storage medium.
The technical scheme for solving the technical problems is as follows: a crowd density estimation method comprising the steps of: acquiring video images of people; performing target detection on the video images of the crowd, marking a pedestrian by using a Bbox, wherein all the Bboxes form a Bbox set corresponding to the crowd; and according to the Bbox set, the crowd density is obtained by calculating the average occupied area.
The beneficial effects of the invention are as follows: by automatically collecting the monitoring video and carrying out target detection, the crowd density is calculated by a method of calculating the average occupied area according to the detection result, so that the crowd density is automatically estimated, the manual workload is reduced, the efficiency and the estimation precision are improved, the number of people is combined with the occupied area, the estimation precision is improved, the comparability of density estimation is enhanced, and the comparison and classification are convenient.
The invention also discloses a readable storage medium comprising instructions which, when run on an electronic device, cause the electronic device to execute the crowd density estimation method according to the technical scheme.
The technical scheme has the beneficial effects that the crowd density is calculated by automatically collecting the monitoring video and carrying out target detection and a method for calculating the average occupied area according to the detection result, so that the crowd density is automatically estimated, the manual workload is reduced, the efficiency and the estimation precision are improved, the number of people and the occupied area are combined, the estimation precision is improved, the comparability of density estimation is enhanced, and the comparison and grading are convenient.
Drawings
FIG. 1 shows a flow chart of a crowd density estimation method provided in accordance with an embodiment of the invention;
fig. 2 shows a schematic diagram of an acquired image provided according to an embodiment of the invention.
Detailed Description
In order that the above-recited objects, features and advantages of the present invention will be more clearly understood, a more particular description of the invention will be rendered by reference to the appended drawings and appended detailed description. It should be noted that, without conflict, the embodiments of the present invention, i.e., the features in the embodiments, may be combined with each other.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, however, the present invention may be practiced in other ways than those described herein, and therefore the scope of the present invention is not limited to the specific embodiments disclosed below.
Fig. 1 shows a flow chart of a crowd density estimation method according to an embodiment of the invention.
As shown in fig. 1, in this embodiment, a crowd density estimation method includes the following steps: acquiring video images of people; performing target detection on the video images of the crowd, marking a pedestrian by using a Bbox, wherein all the Bboxes form a Bbox set corresponding to the crowd; and according to the Bbox set, the crowd density is obtained by calculating the average occupied area.
In the embodiment, the monitoring video is automatically collected, the target detection is carried out, the crowd density is calculated by a method of calculating the average occupied area according to the detection result, the crowd density is automatically estimated, the manual workload is reduced, the efficiency and the estimation precision are improved, the number of people is combined with the occupied area, the estimation precision is improved, the comparability of density estimation is enhanced, and the comparison and the grading are convenient.
It should be noted that, the target detection for the video image is a well-known algorithm in the technical field, and the YOLO target detection algorithm may be used. The result of the application target detection is a crowd represented by a set of bboxes, wherein the bboxes represent rectangular boxes corresponding to pedestrians, and the information comprises the positions of the pedestrians, the height and width of the rectangular boxes and the like.
Optionally, according to the Bbox set, obtaining the crowd density by calculating an average occupied area, including:
determining the number N of people in the crowd according to the number of Bboxes in the Bbox set t ；
According to the BboxAggregate calculation crowd's total area A t ；
According to the number N of people t And the total occupied area A t Calculating the density of people
In the above embodiment, the crowd density represented by the average occupied area is obtained by performing statistical calculation on the Bbox corresponding to the crowd, which is convenient and easy to understand.
Optionally, calculating the total occupied area A of the crowd according to the Bbox set t Comprising the steps of, in combination,
according to each Bbox of the crowd, acquiring the occupation area A of each pedestrian i ；
Calculating the total occupation area of the crowd
Namely A t Is N t A union of the footprints of individuals.
Optionally, the method includes obtaining a occupation area A of each pedestrian according to each Bbox of the crowd i Comprising the steps of, in combination,
converting the pixel abscissa (u, v) of the pedestrians of each Bbox of the crowd into the field size abscissa (u ', v') of the pedestrians; wherein the Bbox comprises the pixel abscissa (u, v) of pedestrians;
determining the occupation area A of each pedestrian according to the field size transverse coordinate (u ', v') of the pedestrian i =c (u ', v', R), i.e. the footprint a i And (3) taking the horizontal and vertical coordinates (u ', v') of the field size as the circle center, wherein the circle with the preset radius R is represented by the field size radius corresponding to the round occupied area of the pedestrians in the extremely low density state.
Floor area A of one person i Is an area surrounded by a circle with radius R, when a person alone enjoys the block to occupy the groundThe area is at a very low density level. According to the reality, about 0.6 person/square meter, representing an extremely low population density, represented by a=pi R 2 R=0.73 meters. R may be set to 0.73 meters as desired.
In video monitoring, the coordinates of pedestrians are obtained as coordinates represented by pixels, and the coordinates represented by the pixels need to be converted into coordinates represented by actual dimensions for the convenience of calculation.
And converting the pixel coordinates of the pedestrians into actual size coordinates through a preset radius, and obtaining the occupied area of each pedestrian.
Under the condition that the occupation area of each pedestrian is known to be a circle with the radius of R and the circle center coordinate value is given, the occupation area of all pedestrians is obtained by common knowledge and conventional means in the field, and the calculation can be realized by geometric calculation or computer image fitting.
In the above embodiment, the pixel value monitored by the video is converted into the occupation area of the field size, so that the occupation actual area of people is conveniently obtained, the crowd density estimation standard is unified, and the comparison and classification are convenient.
Optionally, said converting the pixel abscissa (u, v) of the pedestrians of said each Bbox of the crowd to the field size abscissa (u ', v') of the pedestrians, including,
solving the proportional relation K of the horizontal coordinate of the pedestrian pixel and the field size x Wherein K is x The ratio of the size width D of the preset pedestrians to the average value of the pixel widths w of all the rectangular frames in the crowd is equal to the ratio; the Bbox further comprises a rectangular frame (w, h) with a pedestrian contour, wherein w is the pixel width of the rectangular frame, and h is the pixel height of the rectangular frame;
according to each Bbox of the crowd, an included angle alpha between an imaging plane and a horizontal plane is obtained;
solving the proportional relation K of the ordinate of the pedestrian pixel and the field size y ,K y ＝K x /cosα；
Solving for the solid dimension of the pedestrian, the abscissa (u ', v '), u ' =k x *u，v′＝K y *v。
Fig. 2 shows a schematic diagram of an acquired image. The pedestrian width direction is set as the abscissa direction, and the direction perpendicular to the abscissa is set as the ordinate direction. As can be seen from fig. 2, the imaging plane pixel abscissa axis is in parallel relation to the solid-sized abscissa axis. The ratio of the width of the pedestrian size to the pixel width of all rectangular frames in the crowd can be approximately used as the ratio of the horizontal coordinate of the pedestrian pixels to the field size.
In reality, the width D of a person is about 0.6 m, and the average value of the width of the rectangular frame can be conveniently obtained according to the Bbox set, so that the proportional relation K between the abscissa of the pedestrian pixel and the field size can be obtained x . As can be seen from FIG. 2, the ratio of the ordinate of the pedestrian pixel to the field size is also affected by the determination of the angle between the imaging plane and the horizontal plane, and the ratio K of the ordinate of the pedestrian pixel to the field size can be obtained according to the determination of the angle between the imaging plane and the horizontal plane and the ratio of the abscissa of the pedestrian pixel to the field size y 。
After the proportional relation is obtained, the pixel coordinates can be converted into the field coordinates.
In the above embodiment, the estimation is made more accurate by considering the influence of the angle of the positional relationship of the imaging plane on the estimation.
Optionally, the step of calculating an included angle alpha between the imaging plane and the horizontal plane according to each Bbox of the crowd comprises,
In the above-described embodiments, the first and second embodiments,
the product of the average height-width ratio of all rectangular frames of the imaging plane and the width D of the person is the size and height of the person on the pixel plane, the ratio of the size and height of the person on the pixel plane to the height of the preset pedestrian,the sine value of the included angle between the imaging plane and the horizontal plane can be obtained, so that the included angle between the imaging plane and the horizontal plane can be obtained. The height of the preset pedestrians can be set according to the average height of the pedestrians in reality.
Optionally, the dimension height H of the preset pedestrian is 1.65 meters, the dimension width D of the preset pedestrian is 0.6 meters, and the preset radius R is 0.73 meters.
The size height, the size width and the preset radius of the pedestrians are respectively set to be 1.65 meters, 0.6 meter and 0.73 meter, so that the method is relatively close to the actual situation of the pedestrians, a relatively good estimation effect can be achieved, the accuracy of the estimation method is further improved, the uniformity of estimation standards is ensured, and the method is convenient to use and compare in different monitoring scenes.
Optionally, the method further comprises the following steps:
and classifying the crowd density according to a preset crowd density classification standard.
In the embodiment, the crowd density is graded so as to more conveniently prompt and grade early warning.
Optionally, the crowd density grading criteria is,
when the density D of the crowd t Greater than or equal to
And is less than +.>
The crowd density grade is extremely low;
when the density D of the crowd t Greater than or equal to
And is less than +.>
The crowd density grade is low;
when the density D of the crowd t Greater than or equal to
And is less than +.>
The crowd density grade is medium;
when the density D of the crowd t Greater than or equal to
And is less than +.>
The crowd density grade is high;
when the density D of the crowd t Less than time
The crowd density grade is extremely high.
The embodiment of the invention also provides a readable storage medium, which comprises instructions, when the instructions are run on the electronic equipment, the electronic equipment is caused to execute the crowd density estimation method according to the technical scheme.
In the embodiment, the monitoring video is automatically collected, the target detection is carried out, the crowd density is calculated by a method of calculating the average occupied area according to the detection result, the crowd density is automatically estimated, the manual workload is reduced, the efficiency and the estimation precision are improved, the number of people is combined with the occupied area, the estimation precision is improved, the comparability of density estimation is enhanced, and the comparison and the grading are convenient.
According to the requirements and emergency schemes, the density grade of the crowd can be monitored in real time, the crowd density grade reaches high grade or above, and then the crowd density is sent out to give out an early warning.
In the above embodiments, the response is targeted by establishing more reasonable ranking criteria to use the density estimation and ranking information targeted.
The above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (6)
1. The crowd density estimation method is characterized by comprising the following steps of:
acquiring video images of people;
performing target detection on the video images of the crowd, marking a pedestrian by using a Bbox, wherein all the Bboxes form a Bbox set corresponding to the crowd;
according to the Bbox set, the density of the crowd is obtained by calculating the average occupied area;
according to the Bbox set, the crowd density is obtained by calculating the average occupied area, and the method comprises the following steps:
determining the number N of people in the crowd according to the number of Bboxes in the Bbox set t ；
Calculating the total occupied area A of the crowd according to the Bbox set t ；
According to the number N of people t And the total occupied area A t Calculating the density of people
Calculating the total occupied area A of the crowd according to the Bbox set t Comprising the steps of, in combination,
according to each Bbox of the crowd, acquiring the occupation area A of each pedestrian i ；
Calculating the total occupation area of the crowd
Namely A t Is N t A union of the foot print ranges of the individual pedestrians;
acquiring the occupied area A of each pedestrian according to each Bbox of the crowd i Comprising the steps of, in combination,
converting the pixel abscissa (u, v) of the pedestrians of each Bbox of the crowd into the field size abscissa (u ', v') of the pedestrians; wherein the Bbox comprises the pixel abscissa (u, v) of pedestrians;
determining the occupation area A of each pedestrian according to the field size transverse coordinate (u ', v') of the pedestrian i =c (u ', v', R), i.e. the footprint a i The method is characterized in that the method takes the horizontal coordinate (u ', v ') and the vertical coordinate (u ') of the field size as the circle center, the circle with the preset radius R is used for representing, and R is the field size radius corresponding to the round occupation range of pedestrians in the extremely low density state;
said converting the pixel abscissa (u, v) of the pedestrians of said each Bbox of the crowd into the field size abscissa (u ', v') of the pedestrians, including,
solving the proportional relation K of the horizontal coordinate of the pedestrian pixel and the field size x Wherein K is x The ratio of the size width D of the preset pedestrians to the average value of the pixel widths w of all the rectangular frames in the crowd is equal to the ratio; the Bbox further comprises a rectangular frame (w, h) with a pedestrian contour, wherein w is the pixel width of the rectangular frame, and h is the pixel height of the rectangular frame;
according to each Bbox of the crowd, an included angle alpha between an imaging plane and a horizontal plane is obtained;
solving the proportional relation K of the ordinate of the pedestrian pixel and the field size y ,K y ＝K x /cosα；
Solving for the solid dimension of the pedestrian, the abscissa (u ', v '), u ' =k x *u，v′＝K y *v。
2. The method of claim 1, wherein said determining an angle α between the imaging plane and the horizontal plane based on each Bbox of the crowd comprises,
3. The crowd density estimation method of claim 2, wherein the height H of the size of the preset pedestrian is 1.65 meters, the width D of the size of the preset pedestrian is 0.6 meters, and the radius R is 0.73 meters.
4. The method of crowd density estimation according to claim 1, further comprising the steps of:
and classifying the crowd density according to a preset crowd density classification standard.
5. The method of claim 4, wherein the population density classification criteria is,
when the density D of the crowd t Greater than or equal to
And is less than +.>
The crowd density grade is extremely low;
when the density D of the crowd t Greater than or equal to
And is less than +.>
The crowd density grade is low;
when the density D of the crowd t Greater than or equal to
And is less than +.>
The crowd density grade is medium;
when the density D of the crowd t Greater than or equal to
And is less than +.>
The crowd density grade is high;
when the density D of the crowd t Less than time
The crowd density grade is extremely high.
6. A readable storage medium comprising instructions which, when executed on an electronic device, cause the electronic device to perform a crowd density estimation method according to any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911037521.3A CN110852208B (en) | 2019-10-29 | 2019-10-29 | Crowd density estimation method and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911037521.3A CN110852208B (en) | 2019-10-29 | 2019-10-29 | Crowd density estimation method and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110852208A CN110852208A (en) | 2020-02-28 |
| CN110852208B true CN110852208B (en) | 2023-06-02 |
Family
ID=69598933
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911037521.3A Active CN110852208B (en) | 2019-10-29 | 2019-10-29 | Crowd density estimation method and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110852208B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115223102B (en) * | 2022-09-08 | 2022-12-16 | 枫树谷(成都)科技有限责任公司 | Real-time crowd density fusion sensing method and model based on camera cluster |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101727573A (en) * | 2008-10-13 | 2010-06-09 | 汉王科技股份有限公司 | Method and device for estimating crowd density in video image |
| CN101751553A (en) * | 2008-12-03 | 2010-06-23 | 中国科学院自动化研究所 | Method for analyzing and predicting large-scale crowd density |
| CN104484677A (en) * | 2014-12-31 | 2015-04-01 | 贵州东方世纪科技股份有限公司 | Method for analyzing demographic data information by means of satellite images |
| CN105791774A (en) * | 2016-03-31 | 2016-07-20 | 北京工业大学 | Surveillance video transmission method based on video content analysis |
| CN106127109A (en) * | 2016-06-13 | 2016-11-16 | 宁波工程学院 | A kind of pedestrian density computational methods of large-scale public place |
| CN106295557A (en) * | 2016-08-05 | 2017-01-04 | 浙江大华技术股份有限公司 | A kind of method and device of crowd density estimation |
| CN106845344A (en) * | 2016-12-15 | 2017-06-13 | 重庆凯泽科技股份有限公司 | Demographics' method and device |
| CN109697435A (en) * | 2018-12-14 | 2019-04-30 | 重庆中科云从科技有限公司 | Stream of people's quantity monitoring method, device, storage medium and equipment |
| CN112668451A (en) * | 2020-12-24 | 2021-04-16 | 南京泓图人工智能技术研究院有限公司 | Crowd density real-time monitoring method based on YOLOv5 |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105654021B (en) * | 2014-11-12 | 2019-02-01 | 株式会社理光 | Method and apparatus of the detection crowd to target position attention rate |
-
2019
- 2019-10-29 CN CN201911037521.3A patent/CN110852208B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101727573A (en) * | 2008-10-13 | 2010-06-09 | 汉王科技股份有限公司 | Method and device for estimating crowd density in video image |
| CN101751553A (en) * | 2008-12-03 | 2010-06-23 | 中国科学院自动化研究所 | Method for analyzing and predicting large-scale crowd density |
| CN104484677A (en) * | 2014-12-31 | 2015-04-01 | 贵州东方世纪科技股份有限公司 | Method for analyzing demographic data information by means of satellite images |
| CN105791774A (en) * | 2016-03-31 | 2016-07-20 | 北京工业大学 | Surveillance video transmission method based on video content analysis |
| CN106127109A (en) * | 2016-06-13 | 2016-11-16 | 宁波工程学院 | A kind of pedestrian density computational methods of large-scale public place |
| CN106295557A (en) * | 2016-08-05 | 2017-01-04 | 浙江大华技术股份有限公司 | A kind of method and device of crowd density estimation |
| CN106845344A (en) * | 2016-12-15 | 2017-06-13 | 重庆凯泽科技股份有限公司 | Demographics' method and device |
| CN109697435A (en) * | 2018-12-14 | 2019-04-30 | 重庆中科云从科技有限公司 | Stream of people's quantity monitoring method, device, storage medium and equipment |
| CN112668451A (en) * | 2020-12-24 | 2021-04-16 | 南京泓图人工智能技术研究院有限公司 | Crowd density real-time monitoring method based on YOLOv5 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110852208A (en) | 2020-02-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9754160B2 (en) | Method and device for detecting gathering of objects based on stereo vision as well as non-transitory computer-readable medium | |
| CN107633516B (en) | Method and device for identifying road surface deformation diseases | |
| US20160350921A1 (en) | Automatic camera calibration | |
| CN103164711B (en) | The method of region based on pixel and support vector machine artificial abortion's density Estimation | |
| CN109034036B (en) | Video analysis method, teaching quality assessment method and system and computer-readable storage medium | |
| CN109598794B (en) | Construction method of three-dimensional GIS dynamic model | |
| CN112652015B (en) | BIM-based pavement disease marking method and device | |
| CN111626277B (en) | Vehicle tracking method and device based on over-station inter-modulation index analysis | |
| CN111832489A (en) | Subway crowd density estimation method and system based on target detection | |
| JP6653361B2 (en) | Road marking image processing apparatus, road marking image processing method, and road marking image processing program | |
| Börcs et al. | Fast 3-D urban object detection on streaming point clouds | |
| CN109791607A (en) | It is detected from a series of images of video camera by homography matrix and identifying object | |
| CN101236646B (en) | Method and system for detecting distinct strong relevant direction with estimated image | |
| CN113536935A (en) | Safety monitoring method and equipment for engineering site | |
| CN110852208B (en) | Crowd density estimation method and readable storage medium | |
| JP6698191B1 (en) | Road marking failure detection device, road marking failure detection method, and road marking failure detection program | |
| CN111144377A (en) | Dense area early warning method based on crowd counting algorithm | |
| CN109919068A (en) | Intensive scene stream of people method of real-time is adapted to based on video analysis | |
| EP4250245A1 (en) | System and method for determining a viewpoint of a traffic camera | |
| CN113920535B (en) | Electronic region detection method based on YOLOv5 | |
| CN107025450B (en) | Heat map generation method | |
| CN107016349B (en) | Crowd flow analysis method based on depth camera | |
| KR102624378B1 (en) | Crack detection system for road | |
| CN114463973B (en) | Image-based traffic state detection method | |
| CN108363988A (en) | A kind of people counting method of combination characteristics of image and hydrodynamics characteristic |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |