CN107797262B - Microscope different multiples camera lens based on image texture combines focus method - Google Patents
Microscope different multiples camera lens based on image texture combines focus method Download PDFInfo
- Publication number
- CN107797262B CN107797262B CN201711068903.3A CN201711068903A CN107797262B CN 107797262 B CN107797262 B CN 107797262B CN 201711068903 A CN201711068903 A CN 201711068903A CN 107797262 B CN107797262 B CN 107797262B
- Authority
- CN
- China
- Prior art keywords
- image
- clarity
- objective table
- objective
- block
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/24—Base structure
- G02B21/241—Devices for focusing
- G02B21/244—Devices for focusing using image analysis techniques
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/50—Constructional details
- H04N23/55—Optical parts specially adapted for electronic image sensors; Mounting thereof
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/667—Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- General Physics & Mathematics (AREA)
- Optics & Photonics (AREA)
- Microscoopes, Condenser (AREA)
- Automatic Focus Adjustment (AREA)
Abstract
The present invention discloses a kind of method that the microscope different multiples camera lens joint based on image texture focuses, comprising the following steps: uses low power objective；Initialize Z axis position；It enables objective table move down N step, while calculating camera image clarity at equal intervals, record utmost sharpness and its place Z axis position；Objective table is moved upwards up at current location and the 90% of utmost sharpness position distance；Objective table moving step length is determined according to image definition at this time, repeats this step until clarity reaches the within a certain error range of utmost sharpness；Effective coverage on glass slide is divided into m × n block, calculates the definition values of each sub-block, the maximum sub-block center of definition values is moved to immediately below object lens；Object lens are changed to high power objective and more high power objective, repeat above-mentioned focusing, joint is finally completed and focuses.The present invention is effectively prevented from the problem of falling into local extremum in existing focus method, also solves the predicament that can not be focused due in the visual field without target.
Description
Technical field
The present invention relates to optical focus technical field, in particular to a kind of microscope different multiples mirror based on image texture
Head joint focus method.
Background technique
Series has gradually been moved towards in development based on Digital Image Processing and digital imaging system, the development of full microscope
Change, is integrated and intelligent, increasing effect is played in every field such as life science, material science, national defense safeties.
Cervical arthroplasty needs operator to adjust focussing plane repeatedly, the time-consuming more and labor intensive cost of whole process, focusing it is clear
Degree is easy to bring error by personal subjective assessment, therefore, Full automatic digital is microscopical focus automatically it is imperative.
Flourishing for image processing techniques and reaching its maturity as the automatic of digital microscope for image automated collection systems
It focuses and realizes that the technology that provides may.Digital Image Processing theory thinks that the energy of image is largely focused on the low of amplitude spectrum
Frequency and Mid Frequency, but the acutance of image outline and details richness depend on the radio-frequency component of image.Therefore, various detection images
Marginal information or calculate image high fdrequency component auto focusing method come into being.Detect the letter to focusedimage high fdrequency component
Number is referred to as autofocus evaluation function, and the process focused automatically is exactly to find the process of autofocus evaluation function maximum value, is widely used
Be search by hill climbing method to find best focal point.However in practical applications, autofocus evaluation function curve is that have local maximum
Value, search by hill climbing method is easy to be mistakenly considered when encountering local extremum has found focal plane.In addition, existing automatic focusing
Method is mostly to find focal plane in field of excursion region, if no or rare focus objects in present viewing field region, then assemble
Coke failure.
Summary of the invention
The purpose of the present invention is to provide a kind of, and the microscope different multiples camera lens based on image texture combines focus method,
This method can effectively improve the accuracy focused automatically under high power objective.
Realize the technical scheme is that
A kind of microscope different multiples camera lens joint focus method based on image texture, method includes the following steps:
Step 1: first under low power objective, the effective coverage on glass slide is first placed in micro objective by motorized subject table
Underface；
Step 2:Z axis uses step motor control, and step pitch is fixed, and fixed step number N is selected, so that the N step number of stepper motor
Corresponding distance is greater than twice of the sum of the return difference of stepper motor and the thickness error of glass slide, and it is moveable to be less than Z axis
Maximum magnitude；Initialization Z axis position is above focal plane and apart from focal plane within step number N；
Step 3: enabling objective table move down N step, acquisition camera image and calculate image clearly at equal intervals at the same time
Degree records utmost sharpness and its place Z axis position；
Step 4: objective table is moved quickly into upwards at current location and the 90% of utmost sharpness position distance；
Step 5: acquisition camera image simultaneously calculates clarity, and the degree for deviateing utmost sharpness according to it determines objective table
Step-length, step-length closer from utmost sharpness is smaller, and remoter step-length is bigger；Objective table moves up the step distance, repeats this step
Suddenly until clarity reaches in the error range of utmost sharpness, then success is focused, if objective table is more than initial position, focused
Failure；
Step 6: if focusing successfully, the effective coverage on glass slide is divided into m × n block, and m × n son is obtained under this focal plane
The image information in block region calculates the definition values of each sub-block, the mobile X of control motorized subject table and Y-axis, clarity maximum
Sub-block center be placed in the underface at object lens center；
Step 7: object lens being changed to high power objective, repeat step 2 to step 6；
Step 8: object lens being changed to more high power objective, repeat step 2 to step 5；It focuses and completes；
Wherein, this method obtains the clarity of image using autofocus evaluation function.
The autofocus evaluation function specifically:
Color image is switched into gray level image；
Gaussian Blur is carried out to gray level image, eliminates the influence of noise；
Using canny operator extraction image gradient, and non-zero gradient number of pixels is counted, is returned divided by the total pixel of image
One changes, and obtains the clarity of image.
The step 3 specifically:
Objective table is enabled to move down N step at a slow speed, while timing acquiring and storing camera image, to prevent timer from returning
Letter of transfer number executes overlong time, can not obtain image in time, and another new thread of opening takes the image in memory to calculate clarity, record
Clarity maximum value and its place Z axis position discharge memory headroom shared by image until the calculating of all images finishes.
The step 4 specifically:
90% at a distance from the utmost sharpness place Z axis position that current Z axis position is recorded with step 3 is calculated, control carries
Object platform fast moves this distance upwards, makes objective table close to position of focal plane and is located under focal plane.
The step 5 specifically:
The clarity that current camera image is calculated with autofocus evaluation function, compared with clarity maximum value.It is clear when beginning
Clear degree very little, objective table are moved up with larger step size, and as clarity is close to maximum value, step-length gradually halves, and repeat this step
It is rapid until: (1) (relative error range can be set as 5%- in a certain range for clarity and the relative error of utmost sharpness
10%) it, returns and focuses successfully；(2) moving up total step number is more than N and one for estimating the sum of the constant of stepping return difference, then
Focal plane is not found, is returned and is focused failure.
The step 6 specifically:
If step 5 focuses the clear image successfully obtained under this visual field, it is evenly divided into m × n sub-block, and right
Each sub-block calculates clarity, takes the center of the maximum sub-block of clarity as target position, remembers target position distance view
The horizontal distance of the Yezhong heart is dx, vertical range dy；
By above-mentioned dx, dy is corresponding to the horizontal distance and vertical range on glass slide according to amplification factor, controls objective table
Target position is moved to central region by this mobile distance.
The beneficial effects of the present invention are: a kind of microscope different multiples camera lens connection based on image texture proposed by the present invention
Close focus method, obtain the priori knowledge of image value of maximum articulation first by autofocus evaluation function, after according to this priori knowledge
The problem of constantly adjusting focussing plane, falling into local extremum when being effectively prevented from focusing.In entire focusing, constantly from low
Times object lens are switched to more high power objective, gradually by slightly hitting exactly to content target area the most abundant carefully is moved to the visual field automatically
Centre, also solve because in the visual field without target due to the predicament that can not focus.
Detailed description of the invention
Fig. 1 is focus mechanism schematic diagram of the present invention under low power, high power or more high power objective；
Fig. 2 is flow diagram of the invention；
Fig. 3 is the sector division schematic diagram for judging clarity Yu maximum value degree of closeness；
Fig. 4 is the effective coverage schematic diagram of glass slide；
Fig. 5 is that the effective coverage of glass slide is divided into the schematic diagram of m × n sub-block.
Specific embodiment
Illustrate technological means, technological improvement and beneficial effect of the present invention in order to be more clearly understood, ties below
Closing attached drawing, the present invention will be described in detail.
Automatic microscopy platform mostly uses the movement of step motor control objective table, theoretically looks for the maximum position of image definition
Setting only needs to calculate image definition while objective table is mobile, find its since it is small increase to peak value again reduction when, rear dephasign
Should distance make its reach maximum value at can focus success.However since stepper motor in the characteristic of precision aspect and is moving
Existing return difference problem in the process, when retrogressing, can not accurately return at original maximum value, micro- for high-power microscope
Small error will lead to the huge difference of image image formation state.Therefore focus issues are directed to, one kind provided by the present invention is based on
The microscope different multiples camera lens of image texture combines focus method, referring to fig. 2, comprising the following specific steps
S101: glass slide used referring to fig. 4, detection applying material in the effective coverage of glass slide.In low power objective
Under, motorized subject table is first placed in the central point O of the effective coverage on glass slide the underface of micro objective.
S102:Z axis uses step motor control, and step pitch is fixed, and fixed step number N is selected, so that the N step number institute of stepper motor
Corresponding distance is greater than twice of the return difference of stepper motor and the sum of the thickness error of glass slide, and is less than Z axis movably most
On a large scale；Initialization Z axis position is above focal plane and apart from focal plane within step number N.
S103: it enables objective table move down N step, acquisition camera image and calculates image clearly at equal intervals at the same time
Degree records utmost sharpness and its place Z axis position.
The step specifically:
Objective table is enabled to move down N step at a slow speed, while opening timing device is used for acquisition camera image, to prevent timing
Device call back function executes overlong time, can not obtain image in time, and another new thread of opening is for calculating clarity；
Timer acquires and stores camera image when objective table is mobile, when objective table stops movement, waiting new thread
End of run, Off Timer.
New thread takes image to calculate clarity, records clarity maximum value rateMax and its place Z axis position zmax, until
All image calculating finish, and discharge memory headroom shared by image.
S104: objective table is moved quickly into upwards at current location and the 90% of utmost sharpness position distance.
The step specifically:
If current location is zcurrent, objective table is enabled to move up 0.9 × (zmax-zcurrent) step, make objective table close to coke
Face position is simultaneously located under focal plane.
S105: acquisition camera image simultaneously calculates clarity with autofocus evaluation function, deviates utmost sharpness according to it
Degree determines objective table step-length, and step-length closer from utmost sharpness is smaller, and remoter step-length is bigger；Objective table moves up the step-length
Distance, this step of repetition reach in the error range of utmost sharpness up to clarity, then focus success, if objective table is more than just
Beginning position, then focus failure.
The step specifically:
The clarity rate for calculating current camera image, compared with clarity maximum value rateMax, according to where rate
Section (referring to Fig. 3) determine the step-length that objective table moves up.Rate very little when beginning, that is, fall in section 3, objective table with compared with
Big step-length step3 is moved up and (is set step3 herein as 64 steps), and as objective table is close to position of focal plane, rate sequentially enters area
Section 2, section 1, section 0, step-length successively halve therewith；
Repeat S105 until: (1) clarity reaches or approaches utmost sharpness, that is, falls into section 0, and return focuses successfully；
(2) moving up total step number is more than N and one for estimating the sum of constant C of stepper motor return difference, then does not find focal plane, return
Focus failure.
S106: being divided into m × n sub-block for the effective coverage on glass slide, and m × n sub-block region is obtained under this focal plane
Image information, calculate the definition values of every piece of sub-block, the mobile X of control motorized subject table and Y-axis, texture information is the most abundant
Sub-block region is placed in the underface at object lens center.
The step specifically:
If step 5 focuses successfully, referring to Fig. 5, the effective coverage on glass slide is evenly divided into m × n sub-block, and right
Each sub-block (S11、S12、S13……Snm) definition values are calculated, take the maximum sub-block of clarity (to be assumed to be Skl) center
(x, y) is used as target position, and target position is apart from central region (x0, y0) horizontal distance dx=x-x0, vertical range dy=
y-y0；
By above-mentioned dx, dy is corresponding to the horizontal distance and vertical range on glass slide according to amplification factor, controls objective table
Target position is moved to central region by this mobile distance.
S107: being changed to high power objective for object lens, and in this process, effective coverage is that clarity is maximum in S106 step
Sub-block Skl, S102 is repeated to S106.
S108: object lens are changed to more high power objective, repeat S102 to S105.
In conclusion a kind of microscope different multiples camera lens based on image texture proposed by the invention combines focusing side
Method in such a way that preparatory moving stage scans in a certain range, obtains focal plane imaging figure under a fixed multiple object lens
The priori knowledge of image sharpness value；Then make one step of objective table close at focal plane and below the focal plane, then with from large to small
Step-length moves closer to focal plane, improves the speed of focusing, moves horizontally objective table after obtaining the focal plane under this multiple object lens, makes content
Region abundant is located at visual field center.Constantly switching more high power objective repeats the above process, and finally obtains high power objective downward view
Abundant in content clear image.Effective image high-definition is provided using the method for the present invention for subsequent Micrograph image processing.Root
According to different application backgrounds, the present invention is equally applicable to the automatic focusing of other imaging devices by modification appropriate.
The foregoing is merely presently preferred embodiments of the present invention, is not intended to limit the invention, it is all in spirit of the invention and
Within principle, any modification, equivalent replacement, improvement and so on be should all be included in the protection scope of the present invention.
Claims (2)
1. a kind of method that the microscope different multiples camera lens joint based on image texture focuses, which is characterized in that this method packet
Include step in detail below:
Step 1: first under low power objective, motorized subject table the effective coverage on glass slide be placed in low power objective just under
Side；
Step 2:Z axis uses step motor control, and step pitch is fixed, and selectes fixed step number N, so that the N step number institute of stepper motor is right
The distance answered is greater than twice of the sum of the return difference of stepper motor and the thickness error of glass slide, and it is movably maximum to be less than Z axis
Range；Initialization Z axis position is above focal plane and apart from focal plane within step number N；
Step 3: enabling objective table move down N step, acquisition camera image and calculate image definition at equal intervals at the same time, remember
Record utmost sharpness and its place Z axis position；
Step 4: objective table is moved quickly into upwards at current location and the 90% of utmost sharpness position distance；
Step 5: acquisition camera image simultaneously calculates clarity, determines that objective table walks according to its degree for deviateing utmost sharpness
Long, step-length closer from utmost sharpness is smaller, and remoter step-length is bigger；Objective table moves up the step distance, repeats this step
Until clarity reaches in the error range of utmost sharpness, then success is focused, if objective table is more than initial position, focuses mistake
It loses；
Step 6: if focusing successfully, the effective coverage on glass slide is divided into m × n block, and m × n sub-block area is obtained under this focal plane
The image information in domain calculates the definition values of each sub-block, the mobile X of control motorized subject table and Y-axis, the maximum son of clarity
Block center is placed in the underface at object lens center；
Step 7: object lens being changed to high power objective, repeat step 2 to step 6；
Step 8: object lens being changed to more high power objective, repeat step 2 to step 5；It completes to focus；
Wherein, the calculating image definition uses autofocus evaluation function；
The step 3 specifically:
Objective table is enabled to move down N step at a slow speed, while timing acquiring and storing camera image, to prevent timer callback letter
Number executes overlong time, can not obtain image in time, and another new thread of opening takes the image in memory to calculate clarity, and record is clear
Maximum value and its place Z axis position are spent, until the calculating of all images finishes, discharges memory headroom shared by image；
The step 4 specifically:
Where calculating the utmost sharpness that current Z axis position records with step 3 at a distance from Z axis position 90%, control objective table to
On fast move this distance；
The step 5 specifically:
The clarity that current camera image is calculated with autofocus evaluation function, compared with clarity maximum value；Clarity when beginning
Very little, objective table are moved up with larger step size, and as clarity is close to maximum value, step-length gradually halves, and it is straight to repeat this step
Extremely: (1) clarity and the relative error of utmost sharpness are that relative error range is set as 5%-10% in a certain range, are returned poly-
Coke success；(2) moving up total step number is more than N and the sum of constant for being used for stepper motor return difference, then does not find focal plane, return
It returns, focuses failure；
The step 6 specifically:
If step 5 focuses the clear image successfully obtained under this visual field, it is evenly divided into m × n sub-block, and to each
Sub-block calculates definition values, takes the center of the maximum sub-block of clarity as target position, remembers target position apart from the visual field
The horizontal distance at center is dx,Vertical range is dy；
By above-mentioned dx, dy is corresponding to the horizontal distance and vertical range on glass slide according to amplification factor, and it is mobile to control objective table
Target position is moved to central region by this distance.
2. the method that microscope different multiples camera lens joint according to claim 1 focuses, which is characterized in that the focusing
Evaluation function specifically:
Color image is switched into gray level image；
Gaussian Blur is carried out to gray level image, eliminates the influence of noise；
Using canny operator extraction image gradient, and non-zero gradient number of pixels is counted, carries out normalizing divided by the total pixel of image
Change, obtains the clarity of image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711068903.3A CN107797262B (en) | 2017-11-03 | 2017-11-03 | Microscope different multiples camera lens based on image texture combines focus method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711068903.3A CN107797262B (en) | 2017-11-03 | 2017-11-03 | Microscope different multiples camera lens based on image texture combines focus method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107797262A CN107797262A (en) | 2018-03-13 |
| CN107797262B true CN107797262B (en) | 2019-09-10 |
Family
ID=61549089
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711068903.3A Active CN107797262B (en) | 2017-11-03 | 2017-11-03 | Microscope different multiples camera lens based on image texture combines focus method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107797262B (en) |
Families Citing this family (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109031638B (en) * | 2018-08-23 | 2021-05-14 | 深圳市瑞图生物技术有限公司 | Quick automatic focusing method of biological microscope |
| CN110361382A (en) * | 2018-12-29 | 2019-10-22 | 上海北昂医药科技股份有限公司 | A kind of chromosome scanning imaging system |
| CN109995998B (en) * | 2019-01-03 | 2020-06-12 | 中国科学院生物物理研究所 | Automatic focusing method suitable for scanning/transmission electron microscope imaging |
| CN110232680B (en) * | 2019-05-30 | 2021-04-27 | 广智微芯(扬州)有限公司 | Image ambiguity evaluation method and device |
| CN113676661B (en) * | 2019-11-09 | 2023-08-01 | 深圳市瑞沃德生命科技有限公司 | Focusing searching method and system thereof |
| CN110987886B (en) * | 2019-11-28 | 2022-09-09 | 上海纳奥生物科技有限公司 | Full-automatic microscopic image fluorescence scanning system |
| CN111665617A (en) * | 2020-06-24 | 2020-09-15 | 武汉中纪生物科技有限公司 | Focusing method and system |
| CN113960778B (en) * | 2021-09-29 | 2024-07-30 | 成都西图科技有限公司 | Dynamic step length focusing method based on intermediate frequency filtering |
| CN114785959B (en) * | 2022-06-16 | 2022-10-14 | 江苏美克医学技术有限公司 | Automatic focusing method and device for fluorescence microscope, storage medium and electronic equipment |
| CN116358841B (en) * | 2023-06-01 | 2023-08-08 | 南京木木西里科技有限公司 | Microscope lens self-identification calibration system |
| CN117672914A (en) * | 2023-11-30 | 2024-03-08 | 魅杰光电科技(上海)有限公司 | Focusing system for wafer detection, wafer detection system and focusing method |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101706609B (en) * | 2009-11-23 | 2012-05-02 | 常州超媒体与感知技术研究所有限公司 | Image processing based fast automatic focusing method of microscope |
| EP2508841B1 (en) * | 2011-04-06 | 2016-03-23 | Agfa HealthCare N.V. | System and method for optical coherence tomography |
| CN105306825A (en) * | 2015-11-18 | 2016-02-03 | 成都中昊英孚科技有限公司 | Novel infrared image focusing system and use method thereof |
| CN105938243B (en) * | 2016-06-29 | 2018-04-13 | 华南理工大学 | More multiplying power microscope quick focusing methods in a kind of TFT LCD detections |
-
2017
- 2017-11-03 CN CN201711068903.3A patent/CN107797262B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN107797262A (en) | 2018-03-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107797262B (en) | Microscope different multiples camera lens based on image texture combines focus method | |
| CN109714519B (en) | Method and system for automatically adjusting image frame | |
| CN106324795B (en) | A kind of automatic quick focusing method in more visuals field of detector microscope | |
| CN113109936B (en) | Microscope automatic focusing method and device based on image definition evaluation | |
| CN106534676B (en) | Autofocus adjustment method towards zooming camera system | |
| CN101408709B (en) | Image viewfinding device and automatic focusing method thereof | |
| CN105430279B (en) | Camera fast automatic focusing method and device | |
| CN110531484B (en) | Microscope automatic focusing method with settable focusing process model | |
| CN108345085A (en) | Focus method and focusing system | |
| CN110824689B (en) | Full-automatic microscopic image depth of field expanding system and method thereof | |
| DE102007011222A1 (en) | Autofocus method and autofocus device for its implementation | |
| CN108259753A (en) | A kind of camera auto-focusing method and device that climbing method is improved based on defocus estimation | |
| CN110793000B (en) | Intelligent focusing method for head shaking computer lamp based on machine vision | |
| CN106019550B (en) | Dynamic focusing device and focusing tracking for the micro- scanning of high speed | |
| CN109031638B (en) | Quick automatic focusing method of biological microscope | |
| CN105430277B (en) | Autofocus control method and device | |
| CN110210333A (en) | A kind of focusing iris image acquiring method and device automatically | |
| CN106506946B (en) | A kind of camera automatic focusing method and video camera | |
| CN115308876B (en) | Microscope rapid focusing method, device, medium and product based on reference focal plane | |
| CN106791353B (en) | The methods, devices and systems of auto-focusing | |
| RU2389050C1 (en) | Automatic focusing method | |
| CN114785959B (en) | Automatic focusing method and device for fluorescence microscope, storage medium and electronic equipment | |
| CN114185164B (en) | Quick automatic focusing method of microscope | |
| CN104834081B (en) | Rapid automatic focusing method for stereoscopic microscope | |
| CN105845534A (en) | Automatic focusing method of electron microscope |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |