CN110009661A - A kind of method of video frequency object tracking - Google Patents
A kind of method of video frequency object tracking Download PDFInfo
- Publication number
- CN110009661A CN110009661A CN201910249323.7A CN201910249323A CN110009661A CN 110009661 A CN110009661 A CN 110009661A CN 201910249323 A CN201910249323 A CN 201910249323A CN 110009661 A CN110009661 A CN 110009661A
- Authority
- CN
- China
- Prior art keywords
- network
- classifier network
- classifier
- frame
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention belongs to image/video target following technical fields, provide a kind of method of video frequency object tracking, can carry out continuing tracking to single target specific in video, be related to the relevant knowledge of image procossing.Firstly, we are mutually learnt using depth and method one quick target tracker of training of knowledge distillation.Secondly, coming temporarily, many particles to be spread around previous frame, the distribution of particle is random in each frame.Then we choose a big image-region, can include all particles.Target tracker is sent into the relative position of image-region and particle, obtains score to the end, chooses the result of highest scoring.Final result is regard as final result by surrounding after frame returns.Finally, online updating tracker after every secondary tracking failure or certain time.Benefit of the invention is that changing traditional method of sampling, will sample in image layer becomes sampling in characteristic layer, greatly improves speed, improves speed under conditions of guaranteeing precision.
Description
Technical field
The invention belongs to image/video target following technical field, single target specific in video can be carried out persistently with
Track is related to the relevant knowledge of image procossing.
Background technique
With the continuous development of image processing techniques, video frequency object tracking rises emphatically in daily life because of the practicality
The effect wanted.
Video frequency object tracking is broadly divided into two major classes: particle filter method and correlation filter method.Correlation filter
Method is that relevant matches are carried out around previous frame target using target signature, obtained in final corresponding highest place
For present frame target position.This method is because J.F.Henriques et al. was delivered in PAMI periodical in 2014
' the side of the circular matrix proposed in High-speed tracking with kernelized correlation filters '
Calculating can be transformed into Fourier, calculating speed is caused to greatly speed up by method, and speed real-time is stronger.Then Danelljan
The Learning Spatially Regularized Correlation that M et al. was delivered in ICCV meeting in 2015
Filters for Visual Tracking, this article inhibit the marginal information of filter, enable filter is more acurrate to look for
To target position, to further increase precision.Danelljan M in 2016 et al. has also been proposed Beyond in ECCV meeting
Correlation Filters:Learning Continuous Convolution Operators for Visual
The characteristic pattern of different resolution is interpolated into continuous space domain, reapplies Hessian by Tracking, this article by cube interpolation
Matrix can be in the hope of the target position of sub-pixel precision.Danelljan M in 2017 et al. is further mentioned in CVPR meeting
ECO:Efficient Convolution Operators for Tracking is gone out, this article is grasped using the convolution of factorization
Make, and simplified in feature extraction, finally obtains faster stronger tracker.And particle filter method is in previous frame
A large amount of particles are spread around target position, learn target current location by judging whether it is target to image block in particle.Because
The particle spread is enough, so obtained location information is more accurate, the precision of particle filter method is generally all very high.But
Since it is desired that spreading a large amount of particles, cause calculation amount larger, the speed of this method is generally all slow, cause its scalability compared with
Difference.Typical Representative is the Learning Multi-Domain that Nam H et al. was published in CVPR meeting in 2016
Convolutional Neural Networks for Visual Tracking。
Although current particle filter algorithm achieves good results, need to solve there are still Railway Project.It is first
First, the power in precision of particle filter algorithm, which has, does not capture, and due to greatly developing for correlation filter algorithm, particle filter is calculated
Declining trend is also presented in the precision aspect regarded as a pride in method.Secondly, traditional particle filter is due to needing each image block will
Judged, and in order to keep precision, particle has to enough, causes particle filter algorithm speed very slow.
Summary of the invention
The technical problem to be solved by the present invention is just knowing that first frame gave for any one given video
Target position is kept in subsequent video sequence without that can carry out continuing tracking to target under conditions of other any information
Tracking.Moreover, which, which will also be capable of handling, occurs complex scene in video, light variation, similar purpose, blocks
Situation, it is meant that even if the case where target is in more complicated scene, and target is blocked by similar purpose appearance, we are remained to
Enough trace into target.
The technical scheme is that according to a conclusion observed: in video frequency object tracking, target is in front and back two
Great variety can't occur for the positional shift in frame image, and change in shape will not be apparent.Thus we can lead to
Grain scattering around target is crossed in previous frame image, by judging image block in particle, finds mesh in current image frame
Target position, to carry out continuing tracking to target.Moreover, the computation complexity of particle filter is very high, will lead to
Track speed is very slow, can only there is the speed of 1FPS, we change traditional method of sampling, and will sample in image layer becomes in feature
Layer sampling, greatly improves speed, improves speed under conditions of guaranteeing precision.Specific step is as follows:
A kind of method of video frequency object tracking, steps are as follows:
One, off-line training step:
Step 1: utilizing disclosed one classifier network of database training, the input of classifier network is image block, defeated
It is out the score of image block, wherein image block is that target prospect then export as 1, and image block is that background output is then 0;Classifier net
Network is target prospect or background for differentiating input picture block, and is given a mark to image block;
Step 2: the method mutually learnt using depth is trained simultaneously with two identical classifier networks, and tied
Network layer before fruit establishes connection, is mutually supervised with KL divergence, so that two classifier networks acquisitions are more powerful
Classification capacity;
Step 3: the method distilled using knowledge, using trained obtained classifier network as counselor, to refer to
Lead one new classifier network of training;The input of new classifier network is the coordinate of image block and particle, shape, and it is each for exporting
The score of particle frame;In the training process, each layer of classifier network all establishes connection, is supervised mutually using MSE loss
It superintends and directs, so that new classifier network acquires the ability of former classifier network;
Step 4: being similar to step 2, mutually learn to instruct by depth is carried out based on new classifier network that step 3 learns
Practice, the network layer before result establishes connection, mutually to supervise, so that having outstanding speed in new classifier network
More quasi- precision is obtained while spending, and obtains classifier network to the end;
Two, online tracking phase:
Step 5: to given first frame image, many particles are taken around target true value, utilize the target true value of acquirement
The trained classifier network of trim step 4, enables classifier to better conform to this video;Simultaneously according to these target true values
The encirclement frame of one fine-tuning final result of training returns device, and the input for surrounding frame recurrence device is the feature of particle, and output is to adjust
Target position after whole;
Step 6: spreading a large amount of particle in the first frame image peripheral of given video, the size shape of particle is different;Due to
Suddenly change can't occur for the target position in adjacent two frame, so always having some particles to surround mesh very well in the particle spread
Mark object;
Step 7: the coordinate of one comprising all particles big image block and particle being inputted into classifier network, is obtained every
The score of a particle obtains point highest five particles, such as formula (8), in formula,Indicate video sequence t frame i-th
The score of a particle selects the multiple particles of highest scoringTake its average value;Average value is sent into and surrounds frame recurrence device, most
Frame will be surrounded afterwards returns the output result of device as tracking result;
Step 8: the classifier network characterization of the output result obtained every time being saved, when classifier network score is lower than
When 0.5, it just is finely adjusted classifier network using the feature of storage, and expand resampling;Due to this classifier network score
When lower, illustrate that classifier network is not suitable for this frame at this time, need re -training to better conform to target, it is also possible to target position
It is larger to set movement, just needs to sample to get target position at this time;
Step 9: being also finely adjusted output result using the network characterization of storage every 20 frames;Due to the time too long after,
The change in shape of target is larger, cannot be tracked again with most starting trained result, needs that new classifier network is trained
Better conform to target.
Beneficial effects of the present invention: this method is be relatively accurate and quickly to track single goal, even if in extraneous ring
Also there is outstanding representation when border is in bad order.Compared to ordinary particle filter, this patent can in the case where precision is similar,
Speed is greatly improved, real-time is protected.
Detailed description of the invention
Fig. 1 is the block diagram of off-line training.Fig. 1 (a) is that two target trackers carry out the mutual learning trainings of depth, upper and lower two
Network is identical.Fig. 1 (b) is using trained target tracker (top target tracker) as teacher, using knowledge
The method of distillation instructs quick target tracker (lower section target tracker) training.Fig. 1 (c) be two quick targets with
Track device carries out the mutual learning training of depth, and upper and lower two networks are identical.
Fig. 2 is result of the target tracker on some videos.Every the first picture of a line is video first frame.It is wherein green
It is true value that color, which surrounds frame, and it is the tracking result that we invent that red, which surrounds frame,.
Specific embodiment
Below in conjunction with technical solution, a specific embodiment of the invention is further illustrated.
A kind of method of video frequency object tracking, steps are as follows:
One, off-line training step:
Step 1: utilizing disclosed one classifier network of database training, the input of classifier network is image block, defeated
It is out the score of image block, wherein image block is that target prospect then export as 1, and image block is that background output is then 0;Classifier net
Network is target prospect or background for differentiating input picture block, and is given a mark to image block;Such as formula (1), wherein
It indicates in i-th of image block that the t frame of video sequence is got,Representative image blockThe m class in classifier network 1
Feature, m value 1,2;In presentation class device network 1 in softmax layers m class output;Formula (2) presentation class
The supervision of device network is lost, in formula, Indicate t frame obtains in video sequence i-th
The true value label of a image block, when true value label and classification output phase simultaneously, then on the contrary result is 1, then be 0;It indicates to divide
Class device network 1 takes the loss of N number of image block in image sequence;
Step 2: the method mutually learnt using depth is trained simultaneously with two identical classifier networks, and tied
Network layer before fruit establishes connection, is mutually supervised with KL divergence, so that two classifier networks acquisitions are more powerful
Classification capacity;Formula (3) DKL(p2‖p1) represent and mutually supervised with KL divergence, in formula,Respectively
In presentation class device network 1, classifier network 2 in softmax layers m class output, KL divergence be taken in image sequence it is N number of
Image block is calculated;Formula (4)The final loss of classifier network 1, classifier network 2 is respectively indicated,
InRespectively indicate classifier network 1, classifier network 2 takes the loss of N number of image block in image sequence;λ1、λ2
It is hyper parameter, for adjusting the relationship between loss;
Step 3: the method distilled using knowledge, using trained obtained classifier network as counselor, to instruct
One new classifier network of training;The input of new classifier network is the coordinate of image block and particle, shape, is exported as each particle
The score of frame;In the training process, each layer of classifier network all establishes connection, is supervised mutually using MSE loss, so that
New classifier network acquires the ability of former classifier network;Such as formula (5), in formula,Point
It Biao Shi not image blockIn classifier network Θ1、Θ2In in l layers coordinate k output, W, H are respectively indicated in classifier network
The width and height of l layers of output,The MSE loss function that l layers of network of presentation class device;Formula (6) Respectively indicate classifier network Θ1、Θ2L layers of superposition are lost obtained network MSE loss, and α, β are hyper parameters, in order to
Regulation loss ratio;Following formula (7) is lost in final supervision, is the superposition of Classification Loss and MSE loss;
Step 4: being similar to step 2, mutually learn to instruct by depth is carried out based on new classifier network that step 3 learns
Practice, the network layer before result establishes connection, mutually to supervise, so that having outstanding speed in new classifier network
More quasi- precision is obtained while spending, and obtains classifier network to the end;
Two, online tracking phase:
Step 5: to given first frame image, many particles are taken around target true value, utilize the target true value of acquirement
The trained classifier network of trim step 4, enables classifier to better conform to this video;Simultaneously according to these target true values
The encirclement frame of one fine-tuning final result of training returns device, and the input for surrounding frame recurrence device is the feature of particle, and output is to adjust
Target position after whole;
Step 6: spreading a large amount of particle in the first frame image peripheral of given video, the size shape of particle is different;Due to
Suddenly change can't occur for the target position in adjacent two frame, so always having some particles to surround mesh very well in the particle spread
Mark object;
Step 7: the coordinate of one comprising all particles big image block and particle being inputted into classifier network, is obtained every
The score of a particle obtains point highest five particles, such as formula (8), in formula,Indicate video sequence t frame i-th
The score of a particle selects the multiple particles of highest scoringTake its average value;Average value is sent into and surrounds frame recurrence device, most
Frame will be surrounded afterwards returns the output result of device as tracking result;
Step 8: the classifier network characterization of the output result obtained every time being saved, when classifier network score is lower than
When 0.5, it just is finely adjusted classifier network using the feature of storage, and expand resampling;Due to this classifier network score
When lower, illustrate that classifier network is not suitable for this frame at this time, need re -training to better conform to target, it is also possible to target position
It is larger to set movement, just needs to sample to get target position at this time;
Step 9: being also finely adjusted output result using the network characterization of storage every 20 frames;Due to the time too long after,
The change in shape of target is larger, cannot be tracked again with most starting trained result, needs that new classifier network is trained
Better conform to target.
Claims (1)
1. a kind of method of video frequency object tracking, which is characterized in that steps are as follows:
One, off-line training step:
Step 1: utilizing disclosed one classifier network of database training, the input of classifier network is image block, exports and is
The score of image block, wherein image block is that target prospect then exports as 1, and image block is that background output is then 0;Classifier network is used
It is target prospect or background in differentiating input picture block, and gives a mark to image block;Such as formula (1), whereinIt indicates
In i-th of image block that the t frame of video sequence is got,Representative image blockThe spy of m class in classifier network 1
Sign, m value 1,2;In presentation class device network 1 in softmax layers m class output;Formula (2) presentation class device
The supervision of network is lost, in formula, Indicate t frame obtains in video sequence i-th
The true value label of image block, when true value label and classification output phase simultaneously, then on the contrary result is 1, then be 0;Presentation class device
Network 1 takes the loss of N number of image block in image sequence;
Step 2: the method mutually learnt using depth is trained simultaneously with two identical classifier networks, and result it
Preceding network layer establishes connection, is mutually supervised with KL divergence, so that two classifier networks obtain more powerful point
Class ability;Formula (3) DKL(p2‖p1) represent and mutually supervised with KL divergence, in formula,It respectively indicates point
In class device network 1, classifier network 2 in softmax layers m class output, KL divergence is that N number of image block is taken in image sequence
It is calculated;Formula (4)The final loss of classifier network 1, classifier network 2 is respectively indicated, whereinRespectively indicate classifier network 1, classifier network 2 takes the loss of N number of image block in image sequence;λ1、λ2It is
Hyper parameter, for adjusting the relationship between loss;
Step 3: the method distilled using knowledge, using trained obtained classifier network as counselor, to instruct to instruct
Practice a new classifier network;The input of new classifier network is the coordinate of image block and particle, shape, is exported as each particle
The score of frame;In the training process, each layer of classifier network all establishes connection, is supervised mutually, is made using MSE loss
Obtain the ability that new classifier network acquires former classifier network;Such as formula (5), in formula,
Respectively indicate image blockIn classifier network Θ1、Θ2In in l layers coordinate k output, W, H respectively indicate classifier network
In l layers output width and height,The MSE loss function that l layers of network of presentation class device;Formula (6)Respectively indicate classifier network Θ1、Θ2L layers of superposition are lost obtained network MSE loss, and α, β are super
Parameter, for regulation loss ratio;Following formula (7) is lost in final supervision, is the superposition of Classification Loss and MSE loss;
Step 4: it is similar to step 2, carries out the mutual learning training of depth based on the new classifier network that step 3 is learnt,
As a result the network layer before establishes connection, mutually to supervise, so that having outstanding speed in new classifier network
More quasi- precision is obtained simultaneously, obtains classifier network to the end;
Two, online tracking phase:
Step 5: to given first frame image, many particles are taken around target true value, are finely tuned using the target true value of acquirement
Step 4 trained classifier network, enables classifier to better conform to this video;Simultaneously according to the training of these target true values
The encirclement frame of one fine-tuning final result returns device, surround frame return device input be particle feature, after output is adjustment
Target position;
Step 6: spreading a large amount of particle in the first frame image peripheral of given video, the size shape of particle is different;Due to adjacent
Suddenly change can't occur for the target position in two frames, so always having some particles to surround object very well in the particle spread
Body;
Step 7: the coordinate of one comprising all particles big image block and particle being inputted into classifier network, obtains each grain
The score of son obtains point highest five particles, such as formula (8), in formula,Indicate video sequence t frame i-th
The score of son, selects the multiple particles of highest scoringTake its average value;Average value is sent into and surrounds frame recurrence device, finally will
It surrounds frame and returns the output result of device as tracking result;
Step 8: the classifier network characterization of the output result obtained every time is saved, when classifier network score is lower than 0.5,
Classifier network is finely adjusted with regard to the feature using storage, and expands resampling;When lower due to this classifier network score,
Illustrate that classifier network is not suitable for this frame at this time, need re -training to better conform to target, it is also possible to which target position is mobile
It is larger, it just needs to sample to get target position at this time;
Step 9: being also finely adjusted output result using the network characterization of storage every 20 frames;Due to the time too long after, target
Change in shape it is larger, cannot be tracked again with most starting trained result, need that new classifier network is trained more preferable
Adapt to target.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910249323.7A CN110009661B (en) | 2019-03-29 | 2019-03-29 | Video target tracking method |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910249323.7A CN110009661B (en) | 2019-03-29 | 2019-03-29 | Video target tracking method |
Publications (2)
Publication Number | Publication Date |
---|---|
CN110009661A true CN110009661A (en) | 2019-07-12 |
CN110009661B CN110009661B (en) | 2022-03-29 |
Family
ID=67168857
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201910249323.7A Active CN110009661B (en) | 2019-03-29 | 2019-03-29 | Video target tracking method |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN110009661B (en) |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102750540A (en) * | 2012-06-12 | 2012-10-24 | 大连理工大学 | Morphological filtering enhancement-based maximally stable extremal region (MSER) video text detection method |
WO2015016787A2 (en) * | 2013-07-29 | 2015-02-05 | Galbavy Vladimir | Board game for teaching body transformation principles |
CN107452025A (en) * | 2017-08-18 | 2017-12-08 | 成都通甲优博科技有限责任公司 | Method for tracking target, device and electronic equipment |
US20180268203A1 (en) * | 2017-03-17 | 2018-09-20 | Nec Laboratories America, Inc. | Face recognition system for face recognition in unlabeled videos with domain adversarial learning and knowledge distillation |
CN109389621A (en) * | 2018-09-11 | 2019-02-26 | 淮阴工学院 | RGB-D method for tracking target based on the fusion of multi-mode depth characteristic |
-
2019
- 2019-03-29 CN CN201910249323.7A patent/CN110009661B/en active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102750540A (en) * | 2012-06-12 | 2012-10-24 | 大连理工大学 | Morphological filtering enhancement-based maximally stable extremal region (MSER) video text detection method |
WO2015016787A2 (en) * | 2013-07-29 | 2015-02-05 | Galbavy Vladimir | Board game for teaching body transformation principles |
US20180268203A1 (en) * | 2017-03-17 | 2018-09-20 | Nec Laboratories America, Inc. | Face recognition system for face recognition in unlabeled videos with domain adversarial learning and knowledge distillation |
CN107452025A (en) * | 2017-08-18 | 2017-12-08 | 成都通甲优博科技有限责任公司 | Method for tracking target, device and electronic equipment |
CN109389621A (en) * | 2018-09-11 | 2019-02-26 | 淮阴工学院 | RGB-D method for tracking target based on the fusion of multi-mode depth characteristic |
Non-Patent Citations (2)
Title |
---|
ZHIZHEN CHI ETAL.: "Dual Deep Network for Visual Tracking", 《IEEE TRANSACTIONS ON IMAGE PROCESSING》 * |
罗海波等: "基于深度学习的目标跟踪方法研究现状与展望", 《红外与激光工程》 * |
Also Published As
Publication number | Publication date |
---|---|
CN110009661B (en) | 2022-03-29 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Xu et al. | Learn to scale: Generating multipolar normalized density maps for crowd counting | |
CN106874914B (en) | A kind of industrial machinery arm visual spatial attention method based on depth convolutional neural networks | |
CN108520206B (en) | Fungus microscopic image identification method based on full convolution neural network | |
CN104867161B (en) | A kind of method for processing video frequency and device | |
CN111696137B (en) | Target tracking method based on multilayer feature mixing and attention mechanism | |
Li et al. | ADTrack: Target-aware dual filter learning for real-time anti-dark UAV tracking | |
CN110378943A (en) | Image processing method, device, electronic equipment and storage medium | |
Lin et al. | BiCF: Learning bidirectional incongruity-aware correlation filter for efficient UAV object tracking | |
CN106981080A (en) | Night unmanned vehicle scene depth method of estimation based on infrared image and radar data | |
CN107909081A (en) | The quick obtaining and quick calibrating method of image data set in a kind of deep learning | |
Ni et al. | Controlling the rain: From removal to rendering | |
CN109712165A (en) | A kind of similar foreground picture image set dividing method based on convolutional neural networks | |
CN110222696A (en) | A method of plant leaf blade disease identification is carried out using GPCNNs and ELM | |
Ward et al. | Scalable learning for bridging the species gap in image-based plant phenotyping | |
CN107480676A (en) | A kind of vehicle color identification method, device and electronic equipment | |
CN111539483B (en) | False image identification system based on GAN network and construction method | |
Xie et al. | Small low-contrast target detection: Data-driven spatiotemporal feature fusion and implementation | |
CN105025201B (en) | A kind of video background restorative procedure of space and time continuous | |
CN110009661A (en) | A kind of method of video frequency object tracking | |
Zhang et al. | Boosting transferability of physical attack against detectors by redistributing separable attention | |
Ruhil et al. | Detection of changes from Satellite Images Using Fused Differene Images and Hybrid Kohonen Fuzzy C-Means Sigma | |
Chun et al. | Human action recognition based on improved motion history image and deep convolutional neural networks | |
Liu et al. | Super-Resolution Based on Residual Dense Network for Agricultural Image | |
Li et al. | Detecting Plant Leaves Based on Vision Transformer Enhanced YOLOv5 | |
Ghorban et al. | Improving fm-gan through mixup manifold regularization |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |