CN112183162A - Face automatic registration and recognition system and method in monitoring scene - Google Patents
Face automatic registration and recognition system and method in monitoring scene Download PDFInfo
- Publication number
- CN112183162A CN112183162A CN201910599019.5A CN201910599019A CN112183162A CN 112183162 A CN112183162 A CN 112183162A CN 201910599019 A CN201910599019 A CN 201910599019A CN 112183162 A CN112183162 A CN 112183162A
- Authority
- CN
- China
- Prior art keywords
- face
- target
- module
- information
- quality
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/5866—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
Abstract
The invention provides a face automatic registration and recognition system and a method under a monitoring scene, which realize the face position determination function through a target detection module, realize the matching of targets in a current frame and a historical frame through a target tracking module, evaluate the face quality through a quality judgment module, filter the face which is not meaningfully detected, acquire the main characteristics of the detected face through an identity recognition module and complete the recognition of personnel in a warehouse, and judge the quality condition of the personnel in the warehouse and compare the characteristics of the newly warehoused warehouse through a face information registration module to realize automatic registration and management. On the basis of the existing face recognition system, the face quality judgment process and the historical face information matching method are optimized, the automatic database building and management process is added, the automatic registration, recognition and management mechanism of the personnel identities in the non-database under the monitoring scene is realized, more target access information under the monitoring scene is recorded and associated, and the face quality judgment method has better practicability and stability.
Description
Technical Field
The invention relates to the technical field of digital image processing, in particular to an algorithm for automatically registering and identifying non-warehoused faces under the condition of a monitoring scene.
Background
The face recognition technology is used for detecting faces appearing in a video acquisition picture, extracting key points, extracting features, carrying out feature ratio peer-to-peer processing and finally completing target identity information recognition according to the use requirements of different scenes. The face recognition technology integrates image processing, pattern recognition and probability statistics theory, and is widely applied to aspects of identity verification, safety monitoring, smart cities and the like. The current usage scenario of face recognition can be summarized as 1: 1 comparison (bank counter, customs, hotel check-in, airport security check, internet bar verification, mobile phone unlocking), 1: n-comparison (query individual identity), M: and N comparison (conference sign-in, district entrance guard and automatic gate).
The current face recognition technology needs to collect face information in an early stage for registration, set a white list or black list mode, and then can identify the target under a video collection scene. The processing method has the following disadvantages in the monitoring scene:
a) a large number of unregistered targets exist in a monitoring scene, and the human face recognition system cannot record the access information of the targets;
b) if the target appearing in the step a) is directly recorded without being put in storage, the related information cannot be established when the target appears next time;
c) when the target in the library is shielded or fuzzy, the target is regarded as an unregistered target, and a higher matching distance is generated between the target and the original features in the library in the later detection, an association mechanism of two labels needs to be established.
Disclosure of Invention
Aiming at the defects, the invention provides a system and a method for automatically registering and identifying a human face in a monitoring scene, so as to solve the problem that the existing human face identification method automatically registers, identifies and manages the identity when people are present in a non-library.
The technical scheme of the invention is as follows:
an automatic face registration and recognition system in a monitoring scene is characterized in that: the system comprises video acquisition equipment, a target detection module, a target tracking module, a quality judgment module, an identity recognition module and a face information registration module; the video acquisition equipment is a plurality of camera equipment which are arranged in different areas of a monitoring scene and is used for acquiring the appearance condition of a target under different angles and positions; the target detection module realizes the detection of a target in a collected video frame and the determination of the position of a human face; the target tracking module is used for realizing the association with the target of the previous frame and determining the newly appeared target and the target which is being identified; the quality judgment module acquires quality information of a target face and ensures the effectiveness of extracted features; the identity recognition module realizes identity confirmation of a target corresponding to the face; the face information registration module is a main part and realizes automatic registration and management of strange targets.
The target detection module comprises pedestrian detection and face detection; the pedestrian detection realizes the determination of pedestrian areas in the monitoring picture through a deep learning model; the face detection acquires face information facing the camera equipment in a monitoring picture and assists in suppressing invalid candidate areas in pedestrian detection.
The target tracking module calculates the state change of the targets in the current frame and the historical frame by adopting pose inertia change and regional color morphological characteristic information, predicts the probability condition of the target track on a time axis through a Viterbi algorithm, reserves the optimal alternative track sequence of the number of the detected targets of the upper frame in each moment time period, determines the position of each moment of the target when the track probability meets the threshold requirement, and realizes the matching of the targets in the current frame and the historical frame.
The face quality judgment module judges facial feature information, pose information and fuzzy degree; the facial feature information judges the distance between two eyes, the eye opening degree and the expression change information according to the extracted key points of the human face; the pose information acquires a face deflection angle according to the extracted face key points; and the fuzzy degree is subjected to effectiveness judgment and overall fuzzy grading according to the facial key point confirmed five sense organ region, and an evaluation value is obtained through weighting calculation.
The target identity recognition module comprises image preprocessing, characteristic batch extraction and characteristic batch comparison; the image preprocessing is used for sorting the images according to the input requirements of the feature extraction deep learning model; the characteristic batch extraction obtains effective characteristics of the human face through a deep learning model; and the characteristic batch comparison realizes the similarity distance values of the plurality of faces and the registered base database characteristics through a parallel computing base.
The face information registration module comprises quality condition re-judgment, new warehousing feature comparison and automatic registration library management; the quality condition judges again that the face quality information obtained by the face quality judging module is filtered through a stricter evaluation threshold value, and an unregistered target capable of being put in storage is determined; the new warehousing characteristic comparison realizes the characteristic comparison between the characteristic to be warehoused and the characteristic of the warehousing target in the near period of time, and avoids multiple warehousing of the same target under multiple camera devices; the automatic registry management realizes functions of registering and warehousing a new target, associating and processing labels in a target tracking process when the labels conflict and storing a registered feature library.
The invention also provides a face automatic registration and identification method in a monitoring scene, which is characterized in that:
(1) when a target enters a monitoring scene camera shooting device area, the system acquires a target body range and a face position through a target detection module;
(2) detecting a target position in a subsequent monitoring video frame, predicting and updating the position of a target tracker, storing historical track information of a target, and selecting an optimal target track according to probability occurrence conditions on a time axis;
(3) extracting key points from the detected face of the frame, acquiring facial features, expression conditions and deflection angles of the target face according to the extracted key point information, and determining a face quality evaluation coefficient;
(4) preprocessing the image of the face passing the quality evaluation, extracting the characteristics, comparing the characteristics with the characteristics in the registered database to determine the most similar identity information, judging whether the face is a person in the database, and outputting an identification result if the face is the person in the database;
(5) for non-in-warehouse personnel, setting a stricter quality evaluation threshold value to screen out the face features capable of being put in a warehouse, and analyzing and comparing the face features with the target historical features and recent new warehousing features to ensure that the features are effective and are not registered from other channels; when the registered target is identified as other targets in the library for multiple times in the tracking process and has higher confidence, judging that the identification tags conflict, performing priority judgment on two tags and changing a low-priority tag into a high-priority tag, and deleting the characteristic when the tag is changed for multiple times; when the user saves the registered feature library, the features of the whole library are compared again, and the effectiveness and the representativeness of the saved files are ensured.
On the basis of the existing face recognition system, the face quality judgment process and the historical face information matching method are optimized, the automatic database building and management process is added, the automatic registration, recognition and management mechanism of the personnel identities in the non-database under the monitoring scene is realized, more target access information under the monitoring scene is recorded and associated, and the face quality judgment method has better practicability and stability.
Drawings
FIG. 1 is a framework of the automatic face registration system of the present invention;
fig. 2 is an automatic registry library management flow diagram of the present invention.
Detailed Description
As shown in fig. 1, the present invention includes: the system comprises video acquisition equipment, a target detection module, a target tracking module, a quality judgment module, an identity recognition module and a human face information registration module. The target detection module is composed of pedestrian detection and face detection, and realizes the face position determination function; the target tracking module consists of position prediction, feature matching and time probability prediction, realizes the matching of targets in the current frame and the historical frame, and determines a newly-appeared target and a target which is being identified; the quality judgment module evaluates the quality of the face according to information such as the five sense organs, the pose, the fuzzy degree and the like, and filters the meaningless detected face; the identity recognition module is composed of image preprocessing, characteristic batch extraction and characteristic batch comparison, and is used for acquiring the main characteristics of the detected face and completing the recognition of the personnel in the library; the human face information registration module judges the quality condition of the non-in-warehouse personnel and compares the new in-warehouse characteristics, thereby realizing automatic registration and management.
The target detection module comprises pedestrian detection and face detection; the pedestrian detection realizes the determination of pedestrian areas in the monitoring picture through a deep learning model; the face detection acquires face information facing the camera equipment in a monitoring picture and assists in suppressing invalid candidate areas in pedestrian detection.
The target tracking module calculates the state change of the targets in the current frame and the historical frame by adopting pose inertia change and regional color morphological characteristic information, predicts the probability condition of the target track on a time axis through a Viterbi algorithm, reserves the optimal alternative track sequence of the number of the detected targets of the upper frame in each moment time period, determines the position of each moment of the target when the track probability meets the threshold requirement, and realizes the matching of the targets in the current frame and the historical frame.
The face quality judgment module judges facial feature information, pose information and fuzzy degree; the facial feature information judges the distance between two eyes, the eye opening degree and the expression change information according to the extracted key points of the human face; the pose information acquires a face deflection angle according to the extracted face key points; and the fuzzy degree is subjected to effectiveness judgment and overall fuzzy grading according to the facial key point confirmed five sense organ region, and an evaluation value is obtained through weighting calculation.
The target identity recognition module comprises image preprocessing, characteristic batch extraction and characteristic batch comparison; the image preprocessing is used for sorting the images according to the input requirements of the feature extraction deep learning model; the characteristic batch extraction obtains effective characteristics of the human face through a deep learning model; and the characteristic batch comparison realizes the similarity distance values of the plurality of faces and the registered base database characteristics through a parallel computing base.
The face information registration module comprises quality condition re-judgment, new warehousing feature comparison and automatic registration library management; the quality condition judges again that the face quality information obtained by the face quality judging module is filtered through a stricter evaluation threshold value, and an unregistered target capable of being put in storage is determined; the new warehousing characteristic comparison realizes the characteristic comparison between the characteristic to be warehoused and the characteristic of the warehousing target in the near period of time, and avoids multiple warehousing of the same target under multiple camera devices; the automatic registry management realizes functions of registering and warehousing a new target, associating and processing labels in a target tracking process when the labels conflict and storing a registered feature library.
As shown in fig. 2, the basic workflow of the present invention includes:
(1) when a target enters a monitoring scene camera shooting device area, the system acquires a target body range and a face position through a target detection module;
(2) detecting a target position in a subsequent monitoring video frame, predicting and updating the position of a target tracker, storing historical track information of a target, and selecting an optimal target track according to probability occurrence conditions on a time axis;
(3) extracting key points from the detected face of the frame, acquiring facial features, expression conditions and deflection angles of the target face according to the extracted key point information, and determining a face quality evaluation coefficient;
(4) preprocessing the image of the face passing the quality evaluation, extracting the characteristics, comparing the characteristics with the characteristics in the registered database to determine the most similar identity information, judging whether the face is a person in the database, and outputting the identification result if the face is the person in the database;
(5) for non-in-warehouse personnel, a stricter quality evaluation threshold needs to be set to screen out the face features capable of being put in a warehouse, and the face features are analyzed and compared with the target historical features and the recent new warehousing features to ensure that the features are effective and are not registered from other channels; when the registered target is identified as other targets in the library for multiple times in the tracking process and has higher confidence, judging that the identification tags conflict, performing priority judgment on two tags and changing a low-priority tag into a high-priority tag, and deleting the characteristic when the tag is changed for multiple times; when the user saves the registered feature library, the system compares the features of the whole library again to ensure the validity and the representativeness of the saved files.
Claims (7)
1. An automatic face registration and recognition system in a monitoring scene is characterized in that: the system comprises video acquisition equipment, a target detection module, a target tracking module, a quality judgment module, an identity recognition module and a face information registration module; the video acquisition equipment is a plurality of camera equipment which are arranged in different areas of a monitoring scene and is used for acquiring the appearance condition of a target under different angles and positions; the target detection module realizes the detection of a target in a collected video frame and the determination of the position of a human face; the target tracking module is used for realizing the association with the target of the previous frame and determining the newly appeared target and the target which is being identified; the quality judgment module acquires quality information of a target face and ensures the effectiveness of extracted features; the identity recognition module realizes identity confirmation of a target corresponding to the face; the face information registration module is a main part and realizes automatic registration and management of strange targets.
2. The system for automatically registering and recognizing the human face under the monitoring scene according to claim 1, characterized in that: the target detection module comprises pedestrian detection and face detection; the pedestrian detection realizes the determination of pedestrian areas in the monitoring picture through a deep learning model; the face detection acquires face information facing the camera equipment in a monitoring picture and assists in suppressing invalid candidate areas in pedestrian detection.
3. The system for automatically registering and recognizing the human face under the monitoring scene according to claim 1, characterized in that: the target tracking module calculates the state change of the targets in the current frame and the historical frame by adopting pose inertia change and regional color morphological characteristic information, predicts the probability condition of the target track on a time axis through a Viterbi algorithm, reserves the optimal alternative track sequence of the number of the detected targets of the upper frame in each moment time period, determines the position of each moment of the target when the track probability meets the threshold requirement, and realizes the matching of the targets in the current frame and the historical frame.
4. The system for automatically registering and recognizing the human face under the monitoring scene according to claim 1, characterized in that: the face quality judgment module judges facial feature information, pose information and fuzzy degree; the facial feature information judges the distance between two eyes, the eye opening degree and the expression change information according to the extracted key points of the human face; the pose information acquires a face deflection angle according to the extracted face key points; and the fuzzy degree is subjected to effectiveness judgment and overall fuzzy grading according to the facial key point confirmed five sense organ region, and an evaluation value is obtained through weighting calculation.
5. The system for automatically registering and recognizing the human face under the monitoring scene according to claim 1, characterized in that: the target identity recognition module comprises image preprocessing, characteristic batch extraction and characteristic batch comparison; the image preprocessing is used for sorting the images according to the input requirements of the feature extraction deep learning model; the characteristic batch extraction obtains effective characteristics of the human face through a deep learning model; and the characteristic batch comparison realizes the similarity distance values of the plurality of faces and the registered base database characteristics through a parallel computing base.
6. The system for automatically registering and recognizing the human face under the monitoring scene according to claim 1, characterized in that: the face information registration module comprises quality condition re-judgment, new warehousing feature comparison and automatic registration library management; the quality condition judges again that the face quality information obtained by the face quality judging module is filtered through a stricter evaluation threshold value, and an unregistered target capable of being put in storage is determined; the new warehousing characteristic comparison realizes the characteristic comparison between the characteristic to be warehoused and the characteristic of the warehousing target in the near period of time, and avoids multiple warehousing of the same target under multiple camera devices; the automatic registry management realizes functions of registering and warehousing a new target, associating and processing labels in a target tracking process when the labels conflict and storing a registered feature library.
7. A face automatic registration and identification method in a monitoring scene is characterized in that:
(1) when a target enters a monitoring scene camera shooting device area, the system acquires a target body range and a face position through a target detection module;
(2) detecting a target position in a subsequent monitoring video frame, predicting and updating the position of a target tracker, storing historical track information of a target, and selecting an optimal target track according to probability occurrence conditions on a time axis;
(3) extracting key points from the detected face of the frame, acquiring facial features, expression conditions and deflection angles of the target face according to the extracted key point information, and determining a face quality evaluation coefficient;
(4) preprocessing the image of the face passing the quality evaluation, extracting the characteristics, comparing the characteristics with the characteristics in the registered database to determine the most similar identity information, judging whether the face is a person in the database, and outputting an identification result if the face is the person in the database;
(5) for non-in-warehouse personnel, setting a stricter quality evaluation threshold value to screen out the face features capable of being put in a warehouse, and analyzing and comparing the face features with the target historical features and recent new warehousing features to ensure that the features are effective and are not registered from other channels; when the registered target is identified as other targets in the library for multiple times in the tracking process and has higher confidence, judging that the identification tags conflict, performing priority judgment on two tags and changing a low-priority tag into a high-priority tag, and deleting the characteristic when the tag is changed for multiple times; when the user saves the registered feature library, the features of the whole library are compared again, and the effectiveness and the representativeness of the saved files are ensured.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910599019.5A CN112183162A (en) | 2019-07-04 | 2019-07-04 | Face automatic registration and recognition system and method in monitoring scene |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910599019.5A CN112183162A (en) | 2019-07-04 | 2019-07-04 | Face automatic registration and recognition system and method in monitoring scene |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112183162A true CN112183162A (en) | 2021-01-05 |
Family
ID=73915395
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910599019.5A Pending CN112183162A (en) | 2019-07-04 | 2019-07-04 | Face automatic registration and recognition system and method in monitoring scene |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112183162A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113283319A (en) * | 2021-05-13 | 2021-08-20 | Oppo广东移动通信有限公司 | Method and device for evaluating face ambiguity, medium and electronic equipment |
| CN113822211A (en) * | 2021-09-27 | 2021-12-21 | 山东睿思奥图智能科技有限公司 | Interactive person information acquisition method |
| CN114140864A (en) * | 2022-01-29 | 2022-03-04 | 深圳市中讯网联科技有限公司 | Trajectory tracking method and device, storage medium and electronic equipment |
| CN116110100A (en) * | 2023-01-14 | 2023-05-12 | 深圳市大数据研究院 | Face recognition method, device, computer equipment and storage medium |
| WO2023207582A1 (en) * | 2022-04-28 | 2023-11-02 | 北京字跳网络技术有限公司 | Multi-object tracking method and apparatus, device, and readable storage medium |
| CN117079351A (en) * | 2023-10-12 | 2023-11-17 | 成都崇信大数据服务有限公司 | Method and system for analyzing personnel behaviors in key areas |
-
2019
- 2019-07-04 CN CN201910599019.5A patent/CN112183162A/en active Pending
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113283319A (en) * | 2021-05-13 | 2021-08-20 | Oppo广东移动通信有限公司 | Method and device for evaluating face ambiguity, medium and electronic equipment |
| CN113822211A (en) * | 2021-09-27 | 2021-12-21 | 山东睿思奥图智能科技有限公司 | Interactive person information acquisition method |
| CN113822211B (en) * | 2021-09-27 | 2023-04-11 | 山东睿思奥图智能科技有限公司 | Interactive person information acquisition method |
| CN114140864A (en) * | 2022-01-29 | 2022-03-04 | 深圳市中讯网联科技有限公司 | Trajectory tracking method and device, storage medium and electronic equipment |
| CN114140864B (en) * | 2022-01-29 | 2022-07-05 | 深圳市中讯网联科技有限公司 | Trajectory tracking method and device, storage medium and electronic equipment |
| WO2023207582A1 (en) * | 2022-04-28 | 2023-11-02 | 北京字跳网络技术有限公司 | Multi-object tracking method and apparatus, device, and readable storage medium |
| CN116110100A (en) * | 2023-01-14 | 2023-05-12 | 深圳市大数据研究院 | Face recognition method, device, computer equipment and storage medium |
| CN116110100B (en) * | 2023-01-14 | 2023-11-14 | 深圳市大数据研究院 | Face recognition method, device, computer equipment and storage medium |
| CN117079351A (en) * | 2023-10-12 | 2023-11-17 | 成都崇信大数据服务有限公司 | Method and system for analyzing personnel behaviors in key areas |
| CN117079351B (en) * | 2023-10-12 | 2024-01-30 | 成都崇信大数据服务有限公司 | Method and system for analyzing personnel behaviors in key areas |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112183162A (en) | Face automatic registration and recognition system and method in monitoring scene | |
| CN106203274B (en) | Real-time pedestrian detection system and method in video monitoring | |
| CN109711370B (en) | Data fusion method based on WIFI detection and face clustering | |
| CN108256459B (en) | Security check door face recognition and face automatic library building algorithm based on multi-camera fusion | |
| US7310442B2 (en) | Scene analysis surveillance system | |
| US10839228B2 (en) | Method and system for tracking an object in a defined area | |
| CN106203458B (en) | Crowd video analysis method and system | |
| CN104751136B (en) | A kind of multi-camera video event back jump tracking method based on recognition of face | |
| KR100974293B1 (en) | METHOD AND SYSTEM FOR AUTOMATED FACE DETECTION and Recognition | |
| CN109635146B (en) | Target query method and system based on image characteristics | |
| CN109376639B (en) | Accompanying personnel early warning system and method based on portrait recognition | |
| CN107292240A (en) | It is a kind of that people's method and system are looked for based on face and human bioequivalence | |
| CN105868689A (en) | Cascaded convolutional neural network based human face occlusion detection method | |
| CN105574506A (en) | Intelligent face tracking system and method based on depth learning and large-scale clustering | |
| CN110674761B (en) | Regional behavior early warning method and system | |
| KR101957677B1 (en) | System for learning based real time guidance through face recognition and the method thereof | |
| CN112132048A (en) | Community patrol analysis method and system based on computer vision | |
| JP2006236260A (en) | Face authentication device, face authentication method, and entrance/exit management device | |
| CN111353338A (en) | Energy efficiency improvement method based on business hall video monitoring | |
| Patil et al. | Suspicious movement detection and tracking based on color histogram | |
| KR101515214B1 (en) | Identification method using face recognition and entrance control system and method thereof using the identification method | |
| RU2315352C2 (en) | Method and system for automatically finding three-dimensional images | |
| CN111507232B (en) | Stranger identification method and system based on multi-mode multi-strategy fusion | |
| Manikandan et al. | A neural network aided attuned scheme for gun detection in video surveillance images | |
| CN116798176A (en) | Data management system based on big data and intelligent security |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |