CN113378651A - Distributed rapid pedestrian re-identification system - Google Patents
Distributed rapid pedestrian re-identification system Download PDFInfo
- Publication number
- CN113378651A CN113378651A CN202110551778.1A CN202110551778A CN113378651A CN 113378651 A CN113378651 A CN 113378651A CN 202110551778 A CN202110551778 A CN 202110551778A CN 113378651 A CN113378651 A CN 113378651A
- Authority
- CN
- China
- Prior art keywords
- pedestrian
- identification
- frame
- edge computing
- storage device
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 claims description 21
- 230000005540 biological transmission Effects 0.000 claims description 12
- 238000004364 calculation method Methods 0.000 description 30
- 238000000034 method Methods 0.000 description 10
- 238000005516 engineering process Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 239000007787 solid Substances 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Analysis (AREA)
Abstract
本发明提供一种分布式快速行人重识别系统，涉及行人重识别技术领域。本发明中边缘计算装置与摄像头连接；外部存储装置与边缘计算装置连接；终端计算机用于通过边缘计算装置调取外部存储装置内的文件信息和录像存储装置内的原始视频录像，实现随时目标行人重识别。由于整套系统中边缘计算装置与摄像头一一对应，分布式完成提取实时接收的录像视频流中帧画面的行人特征，将目标行人从各个摄像头摄制的视频录像中寻找出来，高效快速实现行人重识别工作。
The invention provides a distributed fast pedestrian re-identification system, which relates to the technical field of pedestrian re-identification. In the present invention, the edge computing device is connected with the camera; the external storage device is connected with the edge computing device; the terminal computer is used to retrieve the file information in the external storage device and the original video recording in the video storage device through the edge computing device, so as to realize the target pedestrian at any time. Re-identify. Due to the one-to-one correspondence between the edge computing devices and the cameras in the whole system, the pedestrian features of the frames in the video stream received in real time can be extracted in a distributed manner, and the target pedestrians can be found from the video recordings recorded by each camera, thus realizing pedestrian re-identification efficiently and quickly. Work.
Description
Technical Field
The invention relates to the technical field of pedestrian re-identification, in particular to a distributed rapid pedestrian re-identification system.
Background
With the development of deep learning, the neural network technology is applied to more and more scenes, and the re-identification of pedestrians is more and more emphasized as a popular research direction in the field of computer vision. The pedestrian re-identification technology is a technology for tracking pedestrians across mirrors, and a pedestrian is searched from video recordings of all angles according to the posture characteristics of the pedestrian, the color characteristics of clothes and the like, and the tracks of the pedestrian are spliced.
At present, many experts and scholars propose various pedestrian re-identification technologies and pedestrian trajectory reconstruction technologies. For example, pedestrian detection techniques are developed to extract pedestrians from video, wherein more common techniques include YOLOV3 or YOLOV4 pedestrian detection techniques; and developing a pedestrian recognition technology, and sending the picture to the edge device to calculate the characteristic vector of the pedestrian.
However, when the existing pedestrian re-identification scheme performs calculation of pedestrian re-identification, the acquisition frame rate of the video is far higher than the processing frame rate of the video without consideration, so that the re-identification efficiency is low. Although the pedestrian characteristics in the video can be calculated by using a plurality of servers, the detection cannot be synchronous, the cost is high, and the requirement of continuous detection in public places cannot be met due to the defects that the machine is easy to crash after long-term work.
Disclosure of Invention
Technical problem to be solved
Aiming at the defects of the prior art, the invention provides a distributed rapid pedestrian re-identification system, which solves the technical problem of low efficiency of pedestrian re-identification.
(II) technical scheme
In order to achieve the purpose, the invention is realized by the following technical scheme:
a distributed rapid pedestrian re-identification system, comprising: the device comprises a video storage device, an edge calculation device, an external storage device, a camera and a terminal computer;
the video storage device is connected with the camera and used for receiving an original video acquired by the camera;
the edge calculation device is connected with the camera and used for receiving the original video record and acquiring the position of a pedestrian in a frame picture and the characteristic vector of the pedestrian in the frame picture;
the external storage device is connected with the edge calculation device and is used for receiving single file information which is sent by the edge calculation device and consists of the position of the pedestrian in a frame picture, the feature vector of the pedestrian in the frame picture, the time information of the frame picture and the camera number of the source of the frame picture;
and the terminal computer is used for calling file information in the external storage device and the original video in the video storage device through the edge computing device to realize the re-identification of the target pedestrian.
Preferably, the distributed rapid pedestrian re-identification system further comprises a transmission device, and the terminal computer is connected with the edge calculation device through the transmission device.
Preferably, the terminal computer is specifically configured to call the file information and the original video record according to the target pedestrian feature vector and a target time range to be queried, so as to realize target pedestrian re-identification.
Preferably, the edge calculation means comprises a pedestrian detection sub-means and a pedestrian identification sub-means connected to each other;
the pedestrian detection sub-device is used for receiving the original video record and acquiring the position of the pedestrian in a frame picture; the pedestrian re-identification sub-device is used for receiving the position information of the pedestrian in the frame picture and acquiring the feature vector of the pedestrian in the frame picture.
Preferably, the distributed rapid pedestrian re-identification system further comprises a data merging device, and the data merging device is arranged between the terminal computer and the plurality of transmission devices.
Preferably, the data merging device includes several stages, and the data merging device at the next stage is used for integrating the data transmitted by the data merging device at the previous stage.
Preferably, the pedestrian detection sub-device carries a pedestrian detection model, and the pedestrian detection model comprises a YOLOV3 model or a YOLOV4 model; the pedestrian identification sub-device carries a pedestrian re-identification model.
Preferably, the external storage device includes a usb disk, a solid state disk, or a removable disk.
(III) advantageous effects
The invention provides a distributed rapid pedestrian re-identification system. Compared with the prior art, the method has the following beneficial effects:
the edge calculating device is connected with the camera and used for receiving the original video and acquiring the position of a pedestrian in a frame picture and the characteristic vector of the pedestrian in the frame picture; the external storage device is connected with the edge calculation device and is used for receiving single file information which is sent by the edge calculation device and consists of the position of the pedestrian in the frame picture, the feature vector of the pedestrian in the frame picture, the time information of the frame picture and the camera number of the source of the frame picture; and the terminal computer is used for calling file information in the external storage device and the original video in the video storage device through the edge computing device so as to realize the re-identification of the target pedestrian at any time. Because the edge computing devices in the whole set of system correspond to the cameras one to one, the pedestrian characteristics of frame pictures in the video stream received in real time are extracted in a distributed mode, the target pedestrians are searched from the video videos shot by the cameras, and the pedestrian re-identification work is efficiently and quickly realized.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a block diagram of a distributed rapid pedestrian re-identification system according to an embodiment of the present invention;
fig. 2 is a block diagram of a distributed rapid pedestrian re-identification system according to an embodiment of the present invention (including a data merging device);
fig. 3 is a block diagram of a distributed rapid pedestrian re-identification system according to an embodiment of the present invention (including several stages of data merging devices);
fig. 4 is a block diagram of an edge computing device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are clearly and completely described, and it is obvious that the described embodiments are a part of the embodiments of the present invention, but not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The embodiment of the application solves the technical problem of low efficiency of pedestrian re-identification by providing the distributed rapid pedestrian re-identification system.
In order to solve the technical problems, the general idea of the embodiment of the application is as follows:
the edge calculating device in the embodiment of the invention is connected with the camera and used for receiving the original video recording and acquiring the position of a pedestrian in a frame picture and the characteristic vector of the pedestrian in the frame picture; the external storage device is connected with the edge calculation device and is used for receiving single file information which is sent by the edge calculation device and consists of the position of the pedestrian in the frame picture, the feature vector of the pedestrian in the frame picture, the time information of the frame picture and the camera number of the source of the frame picture; and the terminal computer is used for calling file information in the external storage device and the original video in the video storage device through the edge computing device so as to realize the re-identification of the target pedestrian at any time. Because the edge computing devices in the whole set of system correspond to the cameras one to one, the pedestrian characteristics of frame pictures in the video stream received in real time are extracted in a distributed mode, the target pedestrians are searched from the video videos shot by the cameras, and the pedestrian re-identification work is efficiently and quickly realized.
In order to better understand the technical solution, the technical solution will be described in detail with reference to the drawings and the specific embodiments.
Example 1:
as shown in fig. 1 to 3, an embodiment of the present invention provides a distributed rapid pedestrian re-identification system, including: the device comprises a video recording storage device, an edge calculation device, an external storage device, a camera, a transmission device and a terminal computer.
The video storage device is connected with the camera and used for receiving the original video acquired by the camera.
The edge calculation device is connected with the camera and used for receiving the original video record and acquiring the position of the pedestrian in the frame picture and the characteristic vector of the pedestrian in the frame picture.
As shown in fig. 4, the edge calculation means includes a pedestrian detection sub-means and a pedestrian recognition sub-means connected to each other; the pedestrian detection sub-device is used for receiving the original video record and acquiring the position of the pedestrian in a frame picture; the pedestrian re-identification sub-device is used for receiving the position information of the pedestrian in the frame picture and acquiring the feature vector of the pedestrian in the frame picture.
The pedestrian detection sub-device carries a pedestrian detection model, and the pedestrian detection model comprises a Yolov3 model or a Yolov4 model; the pedestrian identification sub-device carries a pedestrian re-identification model.
The external storage device is connected with the edge calculation device and is used for receiving single file information which is sent by the edge calculation device and consists of the position of the pedestrian in the frame picture, the feature vector of the pedestrian in the frame picture, the time information of the frame picture and the camera number of the source of the frame picture.
The external storage device comprises a U disk, a solid state disk or a mobile hard disk.
The terminal computer is connected with the edge computing device through the transmission device; the device is used for calling file information in the external storage device and an original video in the video storage device through the edge computing device to realize target pedestrian re-identification.
Specifically, the terminal computer is specifically configured to call the file information and the original video record according to the target pedestrian feature vector and a target time range to be queried, so as to realize target pedestrian re-identification.
As shown in fig. 2, the distributed rapid pedestrian re-identification system according to the embodiment of the present invention further includes a data merging device, where the data merging device is disposed between the terminal computer and the plurality of transmission devices.
As shown in fig. 3, the data merging device includes several stages, and the data merging device at the next stage is used for integrating the data transmitted by the data merging device at the previous stage.
The working principle of the distributed rapid pedestrian re-identification system provided by the embodiment of the invention is as follows:
an edge computing device is deployed and installed in each camera of a city or each place with pedestrian re-identification requirements, a pre-trained pedestrian detection and pedestrian re-identification model is carried on the edge computing device, the device is provided with a pluggable storage interface, and the device can be connected with a U disk, a solid state disk, a mobile hard disk and the like, and is a large-capacity storage device.
And determining the optimal capacity of the external storage equipment according to the time length of the original video of the camera which can be stored. The edge computing device needs to be connected to a video access port of the camera, that is, video data captured by the camera needs to be transmitted to the original video recording storage device and also needs to be transmitted to the edge device.
The purpose of the edge calculation device is as follows: continuously detecting each frame of picture in a video stream, firstly calculating the pedestrian in the video frame through a pedestrian re-identification model carried in the video stream, wherein the pedestrian detection model can use a YOLOV3 model or a YOLOV4 model, then sending the separated pedestrian into the carried pedestrian re-identification model, and calculating the feature vector of the pedestrian. Therefore, the position of a pedestrian and the number of the pedestrian are obtained, and the time when the frame of picture appears and the current number of the camera are difficult to obtain due to the real-time input of the camera video. The four are packaged into a file to be stored in an external storage device of the edge computing device for later inquiry.
The file contains these four items of information: the position of the pedestrian in the frame picture, the feature vector of the pedestrian in the frame picture, the time information of the frame picture and the camera number of the source of the frame picture. It can be easily found that the position of a pedestrian in the video recording can be accurately determined through the four items of information. And because the four items of information are simple digital information, the occupied storage space is very small, each tone of information only needs a space of a few or dozens of KB approximately, and a slightly larger storage medium can completely meet the requirement of pedestrian information recording of one month or even up to a month.
Of course, for some industrial cameras, if 70 frames of data are acquired in 1 second, the calculation load of the edge calculation device is large. However, for scenes in a city, it takes at least 1 to 2 seconds even if a pedestrian runs out of the camera screen. Therefore, it is not necessary to store every frame of picture collected in the camera at all, and the invention is set to collect 5-10 frames of pictures within 1 second. If 5 frames are acquired in one second, namely 1 frame is acquired in 0.2 second; if 10 frames are acquired in 1 second, 1 frame is acquired in 0.1 second. Therefore, the calculation amount of the edge calculation device is greatly reduced, and a common edge calculation device can also complete corresponding tasks.
The edge computing device stores the pedestrians detected in the monitoring picture at any moment in the storage device. If the pedestrian re-identification is needed, the pedestrian re-identification can be quickly completed only by comparing the pedestrian information stored in each edge device. Since each edge device is networked, it can be connected to the central network, i.e. the terminal computer, by wireless or wired transmission means.
Therefore, when a user needs to inquire a pedestrian in a certain time period, only one pedestrian needs to be inquired at the time, and a common computer (host) can complete the feature extraction of the pedestrian. And the user uses the terminal computer to send the feature vector of the target pedestrian and the target time range to be inquired to each edge device through the network. At this time, each edge device receives the instruction, reads data in the external storage device, screens out pedestrian features within the target time range, and calculates the distances between all pedestrian features and the target pedestrian features in the edge calculation device by calculating the Euclidean distance or some other distances so as to judge whether the pedestrian features are the same pedestrian. And if the pedestrian is the same, sending the file information corresponding to the pedestrian and the original video back to the terminal computer for summarizing.
Because the external storage device stores the file information, the edge computing device can search the pedestrian picture from the video storage device through the file information. After receiving the information of each edge computing device, the terminal computer sequences all the information through the time of each piece of information, and then matches the position information and the time information in the information with the original video, thereby efficiently, quickly and real-timely realizing the re-identification of the pedestrian.
Particularly, when the number of the cameras is very large, if each edge computing device summarizes the information and directly feeds the information back to the terminal computer for sequencing and integration, the processing pressure of the whole system is still high, and the efficiency cannot be guaranteed.
Correspondingly, the embodiment of the invention also provides a data merging device, and the data merging device is arranged between the terminal computer and the plurality of transmission devices. The data merging device is used for integrating the results of calculation and inquiry of a certain number of edge calculation devices or an edge calculation device in a block of area, sorting the inquiry results according to time sequence, and then continuously sending the integrated information to the upper level.
If the area is very large, as shown in fig. 3, a multi-stage data merging device may be added, that is, the previous stage of the data merging device is also the data merging device, and each stage of the data merging device is used to sequentially integrate the data sent by the next stage of the data merging device according to time and send the integrated data back to the previous stage. And finally, sending all integrated information back to the terminal computer, wherein the terminal computer directly searches for a target in the original video through pedestrian information which is arranged in sequence from front to back according to time.
Through the above process, the distributed rapid pedestrian re-identification system provided by the embodiment of the invention further comprises a data merging device to realize rapid, real-time and high-response-speed pedestrian re-identification retrieval function.
In summary, compared with the prior art, the method has the following beneficial effects:
1. the edge calculating device in the embodiment of the invention is connected with the camera and used for receiving the original video recording and acquiring the position of a pedestrian in a frame picture and the characteristic vector of the pedestrian in the frame picture; the external storage device is connected with the edge calculation device and is used for receiving single file information which is sent by the edge calculation device and consists of the position of the pedestrian in the frame picture, the feature vector of the pedestrian in the frame picture, the time information of the frame picture and the camera number of the source of the frame picture; and the terminal computer is used for calling file information in the external storage device and the original video in the video storage device through the edge computing device so as to realize the re-identification of the target pedestrian at any time. Because the edge computing devices in the whole set of system correspond to the cameras one to one, the pedestrian characteristics of frame pictures in the video stream received in real time are extracted in a distributed mode, the target pedestrians are searched from the video videos shot by the cameras, and the pedestrian re-identification work is efficiently and quickly realized.
2. The embodiment of the invention also provides a multi-stage data merging device, and the data merging device is arranged between the terminal computer and the plurality of transmission devices. The data merging device has the functions of integrating the results of calculation and query of a certain number of edge calculation devices or one region, sorting the query results according to time, and then continuously sending the integrated information to the upper stage, so that the finishing efficiency of pedestrian re-identification is further improved.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (8)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110551778.1A CN113378651A (en) | 2021-05-20 | 2021-05-20 | Distributed rapid pedestrian re-identification system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110551778.1A CN113378651A (en) | 2021-05-20 | 2021-05-20 | Distributed rapid pedestrian re-identification system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113378651A true CN113378651A (en) | 2021-09-10 |
Family
ID=77571524
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110551778.1A Pending CN113378651A (en) | 2021-05-20 | 2021-05-20 | Distributed rapid pedestrian re-identification system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113378651A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111401109A (en) * | 2019-01-03 | 2020-07-10 | 芜湖潜思智能科技有限公司 | External face recording and inquiring device of monitoring camera |
| CN111639509A (en) * | 2019-03-01 | 2020-09-08 | 芜湖潜思智能科技有限公司 | Action track recording and inquiring method based on face recognition |
| CN112200106A (en) * | 2020-10-16 | 2021-01-08 | 中国计量大学 | Cross-camera pedestrian re-identification and tracking method |
| CN112507893A (en) * | 2020-12-14 | 2021-03-16 | 华南理工大学 | Distributed unsupervised pedestrian re-identification method based on edge calculation |
| CN112613496A (en) * | 2021-03-08 | 2021-04-06 | 深圳市安软科技股份有限公司 | Pedestrian re-identification method and device, electronic equipment and storage medium |
-
2021
- 2021-05-20 CN CN202110551778.1A patent/CN113378651A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111401109A (en) * | 2019-01-03 | 2020-07-10 | 芜湖潜思智能科技有限公司 | External face recording and inquiring device of monitoring camera |
| CN111639509A (en) * | 2019-03-01 | 2020-09-08 | 芜湖潜思智能科技有限公司 | Action track recording and inquiring method based on face recognition |
| CN112200106A (en) * | 2020-10-16 | 2021-01-08 | 中国计量大学 | Cross-camera pedestrian re-identification and tracking method |
| CN112507893A (en) * | 2020-12-14 | 2021-03-16 | 华南理工大学 | Distributed unsupervised pedestrian re-identification method based on edge calculation |
| CN112613496A (en) * | 2021-03-08 | 2021-04-06 | 深圳市安软科技股份有限公司 | Pedestrian re-identification method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8314854B2 (en) | Apparatus and method for image recognition of facial areas in photographic images from a digital camera | |
| CN111291633B (en) | A real-time pedestrian re-identification method and device | |
| KR20230013243A (en) | Maintain a fixed size for the target object in the frame | |
| US7630561B2 (en) | Image processing | |
| WO2020057355A1 (en) | Three-dimensional modeling method and device | |
| US8867891B2 (en) | Video concept classification using audio-visual grouplets | |
| US7636453B2 (en) | Object detection | |
| US8699852B2 (en) | Video concept classification using video similarity scores | |
| CN101095149B (en) | Image comparison apparatus and method | |
| US20190325205A1 (en) | Method and apparatus for human behavior recognition, and storage medium | |
| GB2409028A (en) | Face detection | |
| EP1374096A2 (en) | Camera meta-data for content categorization | |
| CN113935358A (en) | Pedestrian tracking method, equipment and storage medium | |
| US11200683B2 (en) | Image processing device and image processing method | |
| US11069073B2 (en) | On-shelf commodity detection method and system | |
| CN111241932A (en) | Automobile exhibition room passenger flow detection and analysis system, method and storage medium | |
| CN110796074A (en) | Pedestrian re-identification method based on space-time data fusion | |
| CN111291646A (en) | A method, device, equipment and storage medium for people flow statistics | |
| CN110866418B (en) | Image base generation method, device, equipment, system and storage medium | |
| CN112949539A (en) | Pedestrian re-identification interactive retrieval method and system based on camera position | |
| CN114926757A (en) | Method and system for retrieving lost article, electronic equipment and computer storage medium | |
| KR102097768B1 (en) | Method and Apparatus for Searching Image by Using Adjacent Distance Reference and Computer-Readable Recording Medium with Program | |
| CN113378651A (en) | Distributed rapid pedestrian re-identification system | |
| CN113409056A (en) | Payment method and device, local identification equipment, face payment system and equipment | |
| CN107742115A (en) | A method and system for detecting and tracking moving objects based on video surveillance |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210910 |
|
| RJ01 | Rejection of invention patent application after publication |