CN111427383B - Control method for variable base line of binocular cradle head - Google Patents
Control method for variable base line of binocular cradle head Download PDFInfo
- Publication number
- CN111427383B CN111427383B CN202010189504.8A CN202010189504A CN111427383B CN 111427383 B CN111427383 B CN 111427383B CN 202010189504 A CN202010189504 A CN 202010189504A CN 111427383 B CN111427383 B CN 111427383B
- Authority
- CN
- China
- Prior art keywords
- ball screw
- length
- target
- baseline
- camera
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D3/00—Control of position or direction
- G05D3/12—Control of position or direction using feedback
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Automation & Control Theory (AREA)
- Length Measuring Devices By Optical Means (AREA)
- Stereoscopic And Panoramic Photography (AREA)
Abstract
The invention discloses a control method of a binocular cradle head variable baseline, which comprises the following steps: step one, calculating the vertical distance between the current target and the camera; step two, an ideal baseline length formula of the camera at different target distances is established; step three, calculating the vertical distance from the current target to the camera after moving; step four, calculating the ideal baseline length of the current target after moving according to an ideal baseline length formula; fifthly, determining the fluctuation of the base line length and the rotation angle of the ball screw; and step six, compensating the return error of the ball screw, generating a driving signal, and driving the ball screw to rotate by a driving motor to adjust the length of the base line. The control method disclosed by the invention can realize automatic adjustment of the base line length according to the target distance, and can compensate the return error of the ball screw, thereby improving the accuracy of the base line length.
Description
Technical Field
The invention relates to a control method of a variable baseline of a binocular cradle head.
Background
The binocular cradle head can obtain three-dimensional information from two plane images by utilizing the parallax principle, and a three-dimensional view is constructed.
When the length of the base line of the camera is shorter, the common visual range of the two cameras is large, but the parallax of a distant object is smaller, and the three-dimensional imaging effect on the distant object is poor; when the base line length of the camera is large, the common visible range of the two cameras is small, which is beneficial to three-dimensional imaging of distant objects, but the union of the fields of view of the two cameras is small, which is not beneficial to searching and tracking targets. Therefore, there is a need for a control method for a variable baseline binocular head.
However, the requirement of the binocular camera on the baseline precision is higher, the existing binocular camera baseline control does not compensate for errors on a mechanical structure, the control precision of the baseline distance is not high, and the motion precision, the rapidity and the stability of the cradle head can be reduced due to frequent change of the baseline.
Disclosure of Invention
In order to solve the technical problems, the invention provides a control method of a variable baseline of a binocular cradle head, which aims to realize automatic adjustment of the length of the baseline according to a target distance, compensate return errors of a ball screw and improve the accuracy of the length of the baseline.
In order to achieve the above purpose, the technical scheme of the invention is as follows:
a control method of a binocular cradle head variable baseline comprises the following steps:
step one, calculating the vertical distance between the current target and the camera;
step two, an ideal baseline length formula of the camera at different target distances is established;
step three, calculating the vertical distance from the current target to the camera after moving;
step four, calculating the ideal baseline length of the current target after moving according to an ideal baseline length formula;
fifthly, determining the fluctuation of the base line length and the rotation angle of the ball screw;
and step six, compensating the return error of the ball screw, generating a driving signal, and driving the ball screw to rotate by a driving motor to adjust the length of the base line.
In the above scheme, the specific method of the first step is as follows:
the two lenses of the binocular camera are respectively fixed on left and right identical ball screws, the rotation directions of the two ball screws are opposite, the motor is controlled by the control system to drive the ball screws to rotate respectively, the two lenses are mutually close or far away, the focal length of the two lenses is a fixed value f, and the two lenses of the binocular camera are O l ,O r The distance T between the two is the base line length of the camera;
the method comprises the steps of shooting the same target A through a left camera and a right camera to obtain two images, eliminating distortion and binocular correction, wherein the target A only has difference in horizontal direction in the two images, and finding out the target position in the two images through a target recognition algorithm to obtain the parallax u of the current target l -u r I.e. theThe pixel position u of the target in the images shot by the left camera and the right camera l And u r The difference value in the horizontal direction obtains the vertical distance Z from the current sampling target point A to the binocular camera through the similar triangle principle c :
In the above scheme, the specific method of the second step is as follows:
(1) According to different camera internal parameters and resolutions, combining industry experience, selecting parallax u l -u r Is within a suitable range Deltau 1 ~Δu 2 ;
(2) In the range of the base line length T min ~T max Internally select a plurality of proper baseline length values T min ,T 1 ……T max As an ideal baseline length, the selection of the baseline length value is generated by adopting a mode of a formula (2);
(3) The obtained baseline length value T i Upper and lower limits Deltau of parallax 1 ,Δu 2 Substituting the formula (1) to obtain
[Z i1 ,Z i2 ]I.e. the base length T i The range of the corresponding target distance is T i As an ideal base line length, the method can be used for obtaining
……
This results in an ideal baseline length formula for different target distances:
in the above scheme, the specific method of the third step is as follows:
after the current target moves, the parallax u 'of the target in the next frame of image is acquired' l -u′ r Obtaining the distance Z from the target point A' to the binocular camera after movement c ′:
In the above scheme, the specific method of the fourth step is as follows:
the calculated distance Z between the moving target point and the binocular camera c ' substituting equation (5) to obtain the ideal baseline length T ' at that distance ' ideal 。
In the above scheme, the specific method in the fifth step is as follows:
calculation of ideal baseline Length T' ideal The difference Δt=t 'from the current baseline length T' ideal -T;
Obtaining the rotation angle theta of each ball screw according to the difference delta T
Wherein S is the lead of the ball screw.
In the above scheme, the specific method in the sixth step is as follows:
the two cameras start to move from the initial position accurately positioned on the ball screw, the return error of the ball screw does not exist in the first movement, and the first baseline length variation delta T is obtained according to the previous steps 0 Then, the rotation angle theta of the ball screw is calculated by the formula (7) 0 When the change amount delta T of the base line length is obtained at a certain time i Then, in order to eliminate the return error of the ball screw, it is first determined whether the ball screw is rotated in the reverse direction, that is, the baseline variation Δt i The same sign as the previous variation is directly driven by the control system to rotate the angle theta i The method comprises the steps of carrying out a first treatment on the surface of the When the rotation angles of the left ball screw and the right ball screw are different, the rotation angles of the left ball screw and the right ball screw are respectively increased by delta theta il ,Δθ ir ;
θ il ′=θ i +Δθ il (8)
θ ir ′=θ i +Δθ ir (9)
Then the control system drives the two ball screws to rotate by an angle theta respectively il ' and theta ir ′,Δθ il ,Δθ ir The return error compensation angles of the left ball screw and the right ball screw are respectively.
In a further technical scheme, the return error compensation angle is obtained as follows:
the angle sensor is used for recording the rotation angle of the ball screw, and the laser interferometer is used for judging whether the ball screw starts to drive, so that the angle from the start of rotation to the occurrence of drive recorded at a certain point on the ball screw is the return error compensation angle at the point;
the ideal base line length T of each of the formulas (5) can be obtained after the detection of the left ball screw and the right ball screw according to the method i The corresponding return error compensation angle delta theta il ,Δθ ir 。
Through the technical scheme, the binocular cradle head variable baseline control method provided by the invention realizes the automatic adjustment of the baseline length according to the target distance by establishing an ideal baseline length formula; by compensating the return error of the ball screw, the high-precision ball screw is driven to move, so that the accuracy of the length of the base line of the two phases is improved, and the target is not lost.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below.
FIG. 1 is a schematic diagram of a binocular camera imaging principle disclosed in an embodiment of the present invention;
fig. 2 is an enlarged schematic view of the clearance of the ball screw disclosed in the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.
The invention provides a control method of a variable baseline of a binocular cradle head, which comprises the following steps:
step one, calculating the vertical distance between the current target and the camera;
as shown in fig. 1, two lenses of the binocular camera are respectively fixed on two left and right identical ball screws, the rotation directions of the two ball screws are opposite, the two ball screws are respectively controlled by a control system to drive a motor to rotate, so that the two lenses are mutually close to or far away from each other, the focal length of the two lenses is a fixed value f, and the two lenses of the binocular camera are O l ,O r The distance T between the two is the base line length of the camera;
the method comprises the steps of shooting the same target A through a left camera and a right camera to obtain two images, eliminating distortion and binocular correction, wherein the target A only has difference in horizontal direction in the two images, and finding out the target position in the two images through a target recognition algorithm to obtain the parallax u of the current target l -u r I.e. the position u of the target in the image taken by the left and right cameras l And u r The difference in the horizontal direction is obtained by the principle of similar triangleVertical distance Z of current sample target point A to binocular camera c :
Step two, an ideal baseline length formula of the camera at different target distances is established;
in order to enable the binocular head to adjust proper base line length according to targets at different distances, and meanwhile, adjustment of base lines does not frequently interfere with motion precision and stability of imaging of the binocular camera and pitching and rotating of the binocular head, the embodiment adopts a segmentation function mode to segment the target distance Z into [ Z ] min ,Z 1 ],[Z 1 ,Z 2 ]……[Z n ,Z max ]And the like. Each segment corresponds to a fixed baseline length. The method comprises the following 3 steps:
(1) According to different camera internal parameters and resolutions, combining industry experience, selecting parallax u l -u r Is within a suitable range Deltau 1 ~Δu 2 ;
(2) In the range of the base line length T min ~T max Internally select a plurality of proper baseline length values T min ,T 1 ……T max As an ideal baseline length, the selection of the baseline length value is generated by adopting a mode of a formula (2);
(3) The obtained baseline length value T i Upper and lower limits Deltau of parallax 1 ,Δu 2 Substituting the formula (1) to obtain
[Z i1 ,Z i2 ]I.e. the base length T i The range of the corresponding target distance is T i As an ideal base line length, the method can be used for obtaining
……
This results in an ideal baseline length formula for different target distances:
step three, calculating the vertical distance from the current target to the camera after moving;
after the current target moves, the parallax u 'of the target in the next frame of image is acquired' l -u′ r Obtaining the distance Z from the target point A' to the binocular camera after movement c ′:
Step four, calculating the ideal baseline length of the current target after moving according to an ideal baseline length formula;
the calculated distance Z between the moving target point and the binocular camera c ' substituting equation (5) to obtain the ideal baseline length T ' at that distance ' ideal 。
Fifthly, determining the fluctuation of the base line length and the rotation angle of the ball screw;
calculation of ideal baseline Length T' ideal The difference Δt=t 'from the current baseline length T' ideal -T;
Obtaining the rotation angle theta of each ball screw according to the difference delta T
Wherein S is the lead of the ball screw.
And step six, compensating the return error of the ball screw, generating a driving signal, and driving the ball screw to rotate by a driving motor to adjust the length of the base line.
As shown in fig. 2, since there is a gap Δε=b-2R between the balls and the ball screw, when the rotational direction of the ball screw is changed, there is a period of idle stroke that rotates only without transmission, and the baseline distance is not changed, so that the actually required rotational angle of the ball screw is larger than the theoretical rotational angle θ, and therefore, the return error of the ball screw needs to be compensated, which is specifically as follows:
the two cameras start to move from the initial position accurately positioned on the ball screw, the return error of the ball screw does not exist in the first movement, and the first baseline length variation delta T is obtained according to the previous steps 0 Then, the rotation angle theta of the ball screw is calculated by the formula (7) 0 When the change amount delta T of the base line length is obtained at a certain time i Then, in order to eliminate the return error of the ball screw, it is first determined whether the ball screw is rotated in the reverse direction, that is, the baseline variation Δt i The same sign as the previous variation is directly driven by the control system to rotate the angle theta i The method comprises the steps of carrying out a first treatment on the surface of the When the rotation angles of the left ball screw and the right ball screw are different, the rotation angles of the left ball screw and the right ball screw are respectively increased by delta theta il ,Δθ ir ;
θ il ′=θ i +Δθ il (8)
θ ir ′=θ i +Δθ ir (9)
Then the control system drives the two ball screws to rotate by an angle theta respectively il ' and theta ir ′,Δθ il ,Δθ ir The return error compensation angles of the left ball screw and the right ball screw are respectively.
The return error compensation angle is obtained as follows:
the angle sensor is used for recording the rotation angle of the ball screw, and the laser interferometer is used for judging whether the ball screw starts to drive, so that the angle from the start of rotation to the occurrence of drive recorded at a certain point on the ball screw is the return error compensation angle at the point;
the ideal base line length T of each of the formulas (5) can be obtained after the detection of the left ball screw and the right ball screw according to the method i The corresponding return error compensation angle delta theta il ,Δθ ir 。
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (7)
1. The control method of the binocular cradle head variable base line is characterized by comprising the following steps of:
step one, calculating the vertical distance between the current target and the camera;
step two, an ideal baseline length formula of the camera at different target distances is established;
step three, calculating the vertical distance from the current target to the camera after moving;
step four, calculating the ideal baseline length of the current target after moving according to an ideal baseline length formula;
fifthly, determining the fluctuation of the base line length and the rotation angle of the ball screw;
step six, compensating the return error of the ball screw, generating a driving signal, and driving a motor to push the ball screw to rotate so as to adjust the length of the base line;
the specific method of the second step is as follows:
(1) According to different camera internal parameters and resolutions, combining industry experience, selecting parallax u l -u r Is within a suitable range Deltau 1 ~Δu 2 The method comprises the steps of carrying out a first treatment on the surface of the Parallax u l -u r Pixel position u in the images taken by the left and right cameras for the current target l And u r Difference in horizontal direction;
(2) In the range of the base line length T min ~T max Internally select a plurality of proper baseline length values T min ,T 1 ……T max As an ideal baseline length, the selection of the baseline length value is generated by adopting a mode of a formula (2);
(3) The obtained baseline length value T i Upper and lower limits Deltau of parallax 1 ,Δu 2 Substituting the formula (1) to obtain
[Z i1 ,Z i2 ]I.e. the base length T i The range of the corresponding target distance is T i As an ideal base line length, the method can be used for obtaining
……
This results in an ideal baseline length formula for different target distances:
2. the control method of a variable baseline of a binocular cradle head according to claim 1, wherein the specific method of the first step is as follows:
the two lenses of the binocular camera are respectively fixed on left and right identical ball screws, the rotation directions of the two ball screws are opposite, the motor is controlled by the control system to drive the ball screws to rotate respectively, the two lenses are mutually close or far away, the focal length of the two lenses is a fixed value f, and the two lenses of the binocular camera are O l ,O r The distance T between the two is the base line length of the camera;
the method comprises the steps of shooting the same target A through a left camera and a right camera to obtain two images, eliminating distortion and binocular correction, wherein the target A only has difference in horizontal direction in the two images, and finding out the target position in the two images through a target recognition algorithm to obtain the parallax u of the current target l -u r The vertical distance Z between the current sampling target point A and the binocular camera is obtained through the similar triangle principle c :
3. The control method of a variable baseline of a binocular cradle head according to claim 1, wherein the specific method of the third step is as follows:
after the current target moves, the parallax u of the target in the next frame of image is acquired l ′-u′ r Obtaining the distance Z from the target point A' to the binocular camera after movement c ′:
4. The control method of a variable baseline of a binocular cradle head according to claim 3, wherein the specific method of the fourth step is as follows:
the calculated distance Z between the moving target point and the binocular camera c ' substituting equation (5) to obtain the ideal baseline length T at that distance i ′ deal 。
5. The control method of a variable baseline of a binocular cradle head according to claim 4, wherein the specific method of the fifth step is as follows:
calculation of ideal baseline Length T i ′ deal Difference Δt=t from current baseline length T i ′ deal -T;
Obtaining the rotation angle theta of each ball screw according to the difference delta T
Wherein S is the lead of the ball screw.
6. The control method of a variable baseline of a binocular cradle head according to claim 5, wherein the specific method of the sixth step is as follows:
two ofThe camera starts to move from a starting position accurately positioned on the ball screw, the return error of the ball screw does not exist in the first movement, and the first baseline length variation delta T is obtained according to the previous steps 0 Then, the rotation angle theta of the ball screw is calculated by the formula (7) 0 When the change amount delta T of the base line length is obtained at a certain time i Then, in order to eliminate the return error of the ball screw, it is first determined whether the ball screw is rotated in the reverse direction, that is, the baseline variation Δt i The same sign as the previous variation is directly driven by the control system to rotate the angle theta i The method comprises the steps of carrying out a first treatment on the surface of the When the rotation angles of the left ball screw and the right ball screw are different, the rotation angles of the left ball screw and the right ball screw are respectively increased by delta theta il ,Δθ ir ;
θil′=θ i +Δθ il (8)
θir′=θ i +Δθ ir (9)
Then the control system drives the two ball screws to rotate by an angle theta respectively il ' and theta ir ′,Δθ il ,Δθ ir The return error compensation angles of the left ball screw and the right ball screw are respectively.
7. The method for controlling a variable baseline of a binocular cradle head according to claim 6, wherein the return error compensation angle is obtained by:
the angle sensor is used for recording the rotation angle of the ball screw, and the laser interferometer is used for judging whether the ball screw starts to drive, so that the angle from the start of rotation to the occurrence of drive recorded at a certain point on the ball screw is the return error compensation angle at the point;
the ideal base line length T of each of the formulas (5) can be obtained after the detection of the left ball screw and the right ball screw according to the method i The corresponding return error compensation angle delta theta il ,Δθ ir 。
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010189504.8A CN111427383B (en) | 2020-03-18 | 2020-03-18 | Control method for variable base line of binocular cradle head |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010189504.8A CN111427383B (en) | 2020-03-18 | 2020-03-18 | Control method for variable base line of binocular cradle head |
Publications (2)
Publication Number | Publication Date |
---|---|
CN111427383A CN111427383A (en) | 2020-07-17 |
CN111427383B true CN111427383B (en) | 2023-04-25 |
Family
ID=71548307
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202010189504.8A Active CN111427383B (en) | 2020-03-18 | 2020-03-18 | Control method for variable base line of binocular cradle head |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN111427383B (en) |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114942009B (en) * | 2022-06-20 | 2024-08-20 | 山东信通电子股份有限公司 | Hidden danger early warning method, device, equipment and medium based on power transmission line |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103292710A (en) * | 2013-05-27 | 2013-09-11 | 华南理工大学 | Distance measuring method applying binocular visual parallax error distance-measuring principle |
CN105823468A (en) * | 2016-03-01 | 2016-08-03 | 乐卡汽车智能科技(北京)有限公司 | Baseline distance-adjustable binocular barrier detection method, and apparatus and system thereof |
CN108282649A (en) * | 2018-03-23 | 2018-07-13 | 天津商业大学 | A kind of binocular vision experiment holder that baseline position is freely adjustable |
CN108827246A (en) * | 2018-03-20 | 2018-11-16 | 哈尔滨工程大学 | A kind of binocular vision device that can accurately adjust |
CN109151334A (en) * | 2018-09-21 | 2019-01-04 | 中国计量大学 | A kind of unmanned vehicle camera system |
CN109343578A (en) * | 2018-11-16 | 2019-02-15 | 北京理工大学 | A kind of TDI-CCD camera binocular vision system visual field alignment schemes of view-based access control model feedback |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10645366B2 (en) * | 2016-06-10 | 2020-05-05 | Lucid VR, Inc. | Real time re-calibration of stereo cameras |
-
2020
- 2020-03-18 CN CN202010189504.8A patent/CN111427383B/en active Active
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103292710A (en) * | 2013-05-27 | 2013-09-11 | 华南理工大学 | Distance measuring method applying binocular visual parallax error distance-measuring principle |
CN105823468A (en) * | 2016-03-01 | 2016-08-03 | 乐卡汽车智能科技(北京)有限公司 | Baseline distance-adjustable binocular barrier detection method, and apparatus and system thereof |
CN108827246A (en) * | 2018-03-20 | 2018-11-16 | 哈尔滨工程大学 | A kind of binocular vision device that can accurately adjust |
CN108282649A (en) * | 2018-03-23 | 2018-07-13 | 天津商业大学 | A kind of binocular vision experiment holder that baseline position is freely adjustable |
CN109151334A (en) * | 2018-09-21 | 2019-01-04 | 中国计量大学 | A kind of unmanned vehicle camera system |
CN109343578A (en) * | 2018-11-16 | 2019-02-15 | 北京理工大学 | A kind of TDI-CCD camera binocular vision system visual field alignment schemes of view-based access control model feedback |
Also Published As
Publication number | Publication date |
---|---|
CN111427383A (en) | 2020-07-17 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11032464B2 (en) | Methods and apparatus for absolute and relative depth measurements using camera focus distance | |
CN102256109B (en) | Automatic tracking camera system for multiple targets and focusing method for system | |
CN106233330B (en) | For identifying the method and control equipment of opposite yaw angle change in the three-dimensional video-frequency system for vehicle | |
CN104469169A (en) | Phase ranging camera shooting module and automatic focusing method thereof | |
CN106896343B (en) | Servo follow-up machine vision device and dynamic tracking ranging method | |
CN111612826A (en) | High-precision three-dimensional motion track acquisition positioning and motion process reproduction method based on binocular video sensor | |
CN111427383B (en) | Control method for variable base line of binocular cradle head | |
WO2017117749A1 (en) | Follow focus system and method based on multiple ranging approaches, and photographing system | |
CN108681024A (en) | Moving target surface focusing mechanism and target surface inclination and visual axis run-out detection method thereof | |
US6954589B2 (en) | Lens control apparatus, lens control method and camera | |
CN113155084A (en) | Binocular vision distance measuring device and method based on laser cross standard line assistance | |
CN106210548B (en) | A kind of method and device of rapid focus | |
CN111207685A (en) | Full-automatic calibration system for structured light depth measurement | |
CN202111787U (en) | Automatic multi-target tracking picture pick-up system | |
Lu et al. | Distance and angle measurement of distant objects on an oblique plane based on pixel variation of CCD image | |
JP7229719B2 (en) | Image processing system | |
US20070140673A1 (en) | Multifunction dual lens matching device for stereoscopic 3D camera system | |
CN113689474A (en) | Tracking method based on gaze tracking one-dimensional high-speed moving target system | |
CN112272272B (en) | Imaging method and device | |
WO2017167033A1 (en) | Image processing method and system for vision system | |
CA2976010A1 (en) | Method for readjusting a parallactic or azimuthal mounting | |
CN107505798A (en) | Aero remote sensor based on linear fit mode real-time at the uniform velocity oblique distance focus adjustment method and device | |
JP2020190453A (en) | Tracking device | |
CN111043485B (en) | Two-axis full-automatic tracking holder and tracking method thereof | |
CN112435302B (en) | Remote large-view-field fisheye camera calibration method based on high-precision turntable and parallel light pipes |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |