CN113029041B - Measuring method for dislocation interpolation of multiple laser lines - Google Patents

Measuring method for dislocation interpolation of multiple laser lines Download PDF

Info

Publication number
CN113029041B
CN113029041B CN202110360540.0A CN202110360540A CN113029041B CN 113029041 B CN113029041 B CN 113029041B CN 202110360540 A CN202110360540 A CN 202110360540A CN 113029041 B CN113029041 B CN 113029041B
Authority
CN
China
Prior art keywords
laser
speed
interval
line
maximum
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110360540.0A
Other languages
Chinese (zh)
Other versions
CN113029041A (en
Inventor
张甘露
王逸濠
吴传昊
赵雪磊
徐永安
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Yangzhou University
Original Assignee
Yangzhou University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Yangzhou University filed Critical Yangzhou University
Priority to CN202110360540.0A priority Critical patent/CN113029041B/en
Publication of CN113029041A publication Critical patent/CN113029041A/en
Application granted granted Critical
Publication of CN113029041B publication Critical patent/CN113029041B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G01MEASURING; TESTING
    • G01BMEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00Measuring arrangements characterised by the use of optical techniques
    • G01B11/24Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
    • G01B11/25Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object
    • G01B11/254Projection of a pattern, viewing through a pattern, e.g. moiré

Abstract

The invention discloses a measuring method for dislocation interpolation of a plurality of laser lines in the technical field of tunnel detection, which comprises the following steps: (1) Installing a plurality of line laser generators and a high-speed digital camera on a mobile platform together; (2) starting a line laser generator; (3) starting the high-speed digital camera; (4) Start shiftA movable platform; (5) Splicing the shot laser line images and then importing the spliced laser line images into a computer; (6) dividing the spliced laser line image; (7) Calculating all interval values in the effective measurement range, and finding out a maximum value; (8) If the maximum interval is less than or equal to s, the speed multiple is reserved; otherwise, the speed multiple is not reserved; (9) Returning to the step (4) until the speed multiple VnReaching a set threshold; (10) Determining the range of selectable speed multiples according to the reserved speed multiples, and finally selecting the maximum speed multiple Vmax(ii) a (11) Calculating the maximum velocity vmax(ii) a The invention can improve the detection efficiency.

Description

Measuring method for dislocation interpolation of multiple laser lines
Technical Field
The invention belongs to the technical field of tunnel detection, and particularly relates to a measuring method for dislocation interpolation of multiple laser lines.
Background
A structured light three-dimensional vision technology belongs to an active optical measurement method, one or more encoding patterns are projected to a measurement scene, a camera is used for collecting a scene projection image at a position forming a certain angle with the projection direction, and then the three-dimensional information of the scene is calculated by matching the projection image with corresponding points of the encoding patterns and utilizing the triangulation principle. Laser points, laser lines, gratings are three main structured lights. As an effective and reliable three-dimensional reconstruction and active measurement means, the structured light three-dimensional vision technology is increasingly important to be applied in the fields of industrial automatic measurement, robot navigation, three-dimensional scene reconstruction and the like due to the characteristics of low cost, high precision, large visual field, good real-time performance, strong anti-interference capability and the like.
The laser spot measurement technology adopts a laser spot generator to project laser spots, a linear array camera collects laser spot images on the surface of an object, the center of the laser spot images is extracted, and the depth coordinate of the surface of the object is calculated according to the triangulation principle, and the laser spot measurement technology is characterized in that one spot is collected at one time, the measurement speed is low, but the measurement precision is high and can reach +/-0.01 mm. In order to improve the measuring speed, a laser line measuring technology is adopted for measuring, a laser line generator is adopted in the existing laser line measuring technology to project a laser line, an area array camera is used for collecting laser line images on the surface of an object, the center of the laser line images is extracted, the depth coordinate of the surface of the object is calculated according to the triangulation principle, one line is collected at a time, the measuring speed is relatively improved, the measuring speed is limited at sampling intervals under the condition that the frame frequency of the camera is fixed, the effect of measuring speed can be achieved only by improving the frame frequency of the camera, and the cost is high.
Disclosure of Invention
The invention aims to overcome the defects in the prior art, provides a measuring method for dislocation interpolation of multiple laser lines, solves the technical problem of low measuring speed in the prior art, and can increase the measuring speed by times and improve the detection efficiency under the condition that the frame frequency of a camera is fixed and the measuring interval meets the specified requirements.
The purpose of the invention is realized by the following steps: a measuring method for dislocation interpolation of a plurality of laser lines comprises the following steps:
(1) Installing a plurality of line laser generators and a high-speed digital camera on a mobile platform together;
(2) Starting a line laser generator, and keeping a plurality of laser lines projected to the surface of the tunnel in a shooting visual field range of a high-speed digital camera;
(3) Starting a high-speed digital camera, and collecting laser line images according to a certain frame frequency f;
(4) Starting the mobile platform, and driving for a period of time t at a certain speed v to enable the plurality of laser lines to be staggered and interpolated;
(5) Splicing laser line images shot by a high-speed digital camera and then leading the laser line images into a computer;
(6) Dividing the spliced laser line image in the step (5) into a starting stage, an effective measuring range and an ending stage;
(7) Calculating the intervals between all adjacent laser lines in the effective measurement range in the image, and finding out the maximum value;
(8) Judging whether the maximum interval is smaller than a specified interval s, and if the maximum interval is smaller than or equal to s, keeping the speed multiple under the condition; if the maximum interval is greater than s, the speed multiple under the condition is not reserved;
(9) Repeating the steps (4) to (8) until the speed multiple VnReaching a set threshold;
(10) According to the reserved speed multiple, determining the range of selectable speed multiple, and finally selecting the maximum speed multiple Vmax
(11) Utilizing the maximum speed multiple V selected in the step (10)maxCalculating the maximum velocity v measured by a plurality of laser linesmax
(12) Maximum velocity v measured from multiple laser linesmaxThe method is applied to a three-dimensional laser detection system of the subway tunnel for actual detection.
As a further improvement of the invention, in the step (1), the specific treatment steps are,
(101) Selecting k line laser generators;
(102) The k line laser generators are uniformly assembled according to a certain interval, so that the interval between the head laser line and the tail laser line is D, and the interval between the adjacent laser lines is D
Figure GDA0003781738240000031
(103) Installing the assembled line laser generator in the step (102) and the high-speed digital camera on a mobile platform together;
wherein k is more than or equal to 2, and the unit of the interval D ismm
As a further improvement of the present invention, in the step (4), v = v0*Vn,v0Is the initial velocity, v0Has units of mm/s, VnN is the multiplication times, delta is the increment of the multiple, and 1+
Figure GDA0003781738240000032
According to the driving time t, the shooting times m = t × f of the high-speed digital camera can be calculated, and m > k.
In order to further realize the division of the laser line image, in the step (6), the specific step of dividing the laser line image is,
(601) For the same laser line, the distance between two adjacent positions of the laser line shot by the high-speed digital camera is as follows:
Figure GDA0003781738240000041
(602) Let i be an element [1, k ]],j∈[1,m],XijThe coordinate of the j-th position of the ith laser line shot by the high-speed digital camera is represented by the following calculation formula:
Figure GDA0003781738240000042
(603) Dividing the laser line image:
division X1,1~Xk,1For the start phase, Xk,1~X1,mFor effective measurement range, X1,m~Xk,mThe end phase is completed.
In order to further realize the calculation of all the intervals, in the step (7), the specific step of calculating the intervals between all the adjacent laser lines in the effective measurement range in the image is,
(701) Is provided with
Figure GDA0003781738240000043
(702)
Figure GDA0003781738240000044
Time, effective measurement range is generated
Figure GDA0003781738240000045
Variety of interval values distrThe calculation formula is as follows,
Figure GDA0003781738240000046
(703)
Figure GDA0003781738240000047
time, effective measurement range is generated
Figure GDA0003781738240000048
The seed interval value is calculated by the formula,
Figure GDA0003781738240000051
above provide
Figure GDA0003781738240000052
Another way of calculating the interval value is as follows,
firstly, taking (m-k + 1) group coordinate values,
X1,k,X2,(k-1),...,X(k-1),2,Xk,1
X1,(k+1),X2,k,...,X(k-1),3,Xk,2
X1,m,X2,(m-1),...,X(k-1),(m-k),Xk,(m-k+1)
then, the absolute value of the difference value is taken from the adjacent coordinates in each group of coordinate values, and the interval value is obtained
Figure GDA0003781738240000053
As a further improvement of the invention, in the step (11), the maximum speed of measurement is calculated by the formula,
vmax=v0*Vmax
compared with the prior art, the invention has the following technical effects: the aim of improving the laser line measuring speed is achieved through dislocation interpolation in the process of movement of a plurality of lasers under the condition that the frame frequency of the camera is not improved and the measuring interval is ensured to meet the specified requirements, the cost is saved, and the measuring speed can be doubled.
Drawings
FIG. 1 is a perspective view of a multi-laser-line misalignment interpolation according to the present invention.
FIG. 2 is a diagram of the uniform arrangement of 6 laser lines in the present invention.
FIG. 3 is a graph of the sampling interval for 6 laser lines of the present invention operating at 4.75 times the speed.
FIG. 4 shows the present invention
Figure GDA0003781738240000061
The maximum interval distribution map in the effective measurement range of different speed multiples.
Wherein, 1 line laser generator, 2 high-speed digital camera.
Detailed Description
The invention is further described below with reference to the accompanying drawings.
A measuring method for dislocation interpolation of a plurality of laser lines comprises the following steps:
(1) Installing a plurality of line laser generators and a high-speed digital camera on a mobile platform together;
(2) Starting a line laser generator, and keeping a plurality of laser lines projected to the surface of the tunnel in a shooting visual field range of the high-speed digital camera;
(3) Starting a high-speed digital camera, and collecting laser line images according to a certain frame frequency f;
(4) Starting the mobile platform, and driving for a period of time t at a certain speed v to enable the plurality of laser lines to be staggered and interpolated;
(5) Splicing laser line images shot by a high-speed digital camera and then leading the laser line images into a computer;
(6) Dividing the spliced laser line image in the step (5) into a starting stage, an effective measuring range and an ending stage;
(7) Calculating the intervals between all adjacent laser lines in the effective measurement range in the image, and finding out the maximum value;
(8) Judging whether the maximum interval is smaller than a specified interval s, and if the maximum interval is smaller than or equal to s, keeping the speed multiple under the condition; if the maximum interval is greater than s, the speed multiple under the condition is not reserved;
(9) Repeating the steps (4) to (8) until the speed multiple VnReaching a set threshold;
(10) According to the reserved speed multiple, determining the range of selectable speed multiple, and finally selecting the maximum speed multiple Vmax
(11) Utilizing the maximum speed multiple V selected in the step (10)maxCalculating the maximum velocity v measured by a plurality of laser linesmax
(12) Measuring the maximum velocity v of a plurality of laser linesmaxThe method is applied to a three-dimensional laser detection system of the subway tunnel for actual detection.
Wherein in the step (1), the concrete processing steps are,
(101) Selecting k line laser generators;
(102) The k line laser generators are uniformly assembled according to a certain interval, so that the interval between the head laser line and the tail laser line is D, and the interval between the adjacent laser lines is D
Figure GDA0003781738240000071
(103) Installing the assembled line laser generator in the step (102) and the high-speed digital camera on a mobile platform together;
wherein k is more than or equal to 2, and the unit of the interval D is mm.
In step (4), v = v0*Vn,v0Is the initial velocity, v0Has units of mm/s, VnN is the multiplication times, delta is the increment of the multiple, and 1+
Figure GDA0003781738240000072
According to the driving time t (unit is s), the shooting times m = t × f of the high-speed digital camera can be calculated, and m > k.
In order to further realize the division of the laser line image, in the step (6), the specific step of dividing the laser line image is,
(601) For the same laser line, the distance between two adjacent positions of the laser line shot by the high-speed digital camera is as follows:
Figure GDA0003781738240000081
(602) Let i be an element [1, k ]],j∈[1,m],XijThe coordinate of the j-th position of the ith laser line shot by the high-speed digital camera is represented by the following calculation formula:
Figure GDA0003781738240000082
(603) Dividing the laser line image:
division X1,1~Xk,1For the start phase, Xk,1~X1,mFor effective measurement range, X1,m~Xk,mThe end phase is completed.
In order to further realize the calculation of all intervals, in the step (7), the specific steps of calculating the intervals between all adjacent laser lines in the effective measurement range in the image are,
(701) Is provided with
Figure GDA0003781738240000083
(702)
Figure GDA0003781738240000084
Time, effective measurement range is generated
Figure GDA0003781738240000085
Variety of interval values distrThe calculation formula is as follows,
Figure GDA0003781738240000086
(703)
Figure GDA0003781738240000091
time, effective measurement range is generated
Figure GDA0003781738240000092
Variety of interval values distrThe calculation formula is as follows,
Figure GDA0003781738240000093
above provide
Figure GDA0003781738240000094
Another way of calculating the interval value is as follows,
firstly, taking (m-k + 1) group coordinate values,
X1,k,X2,(k-1),...,X(k-1),2,Xk,1
X1,(k+1),X2,k,...,X(k-1),3,Xk,2
X1,m,X2,(m-1),...,X(k-1),(m-k),Xk,(m-k+1)
then, the absolute value of the difference value is taken from the adjacent coordinates in each group of coordinate values, and the interval value is obtained
Figure GDA0003781738240000095
In step (11), the maximum speed of measurement is calculated by the formula,
vmax=v0*Vmax
in the present embodiment, it is specified that the laser line measurement interval is not allowed to be greater than 2mm, as shown in fig. 1, the present embodiment provides that 6 laser lines are uniformly distributed at an interval of 11mm (as shown in fig. 2), the maximum measurement speed is 4.75 times, the 6 laser lines are staggered and interpolated to make the measurement interval within the effective measurement range not greater than 2mm (as shown in fig. 3), specifically, for 6 laser lines uniformly arranged at the interval as shown in fig. 2, the camera samples at 500 frames per second, and the initial speed is 3.6km ÷h (1000 mm/s) and a speed multiple of
Figure GDA0003781738240000101
Each increment being 0.05 (i.e. choosing δ to be 0.05), for each speed multiple VnEach laser line is pressed by dnGenerating a sequence at intervals, sequencing the sequence of 6 laser lines according to the distance relative to the initial position, and detecting the maximum interval between adjacent laser lines in the effective measurement range, as shown in fig. 4, wherein the maximum interval in the effective measurement range which is not more than 2mm is an optional speed multiple, and the optional speed multiple range is as follows: 1.15-1.75 times, 1.9-2.15 times, 2.25-2.55 times, 2.95-3.25 times, 3.35-3.5 times, 3.9-4.0 times and 4.5-4.75 times, and for 500-frame cameras per second, the maximum measuring speed is 17.1 kilometers per hour, and the speed multiple of the embodiment reaches 4.75.
The invention achieves the purpose of improving the laser line measuring speed by the dislocation interpolation in the process of multiple laser movement under the condition that the frame frequency of the camera is fixed and the measuring interval is ensured to meet the regulation.
The present invention is not limited to the above embodiments, the number of laser lines, the installation manner of the laser, the measurement interval, and the speed multiple can be designed according to actual requirements, and based on the technical solutions disclosed in the present invention, those skilled in the art can make some substitutions and modifications to some technical features without creative efforts according to the disclosed technical contents, and these substitutions and modifications are all within the protection scope of the present invention.

Claims (6)

1. A measuring method for dislocation interpolation of a plurality of laser lines is characterized by comprising the following steps:
(1) Installing a plurality of line laser generators and a high-speed digital camera on a mobile platform together;
(2) Starting a line laser generator, and keeping a plurality of laser lines projected to the surface of the tunnel in a shooting visual field range of the high-speed digital camera;
(3) Starting a high-speed digital camera, and collecting laser line images according to a certain frame frequency f;
(4) Starting the mobile platform, and driving for a period of time t at a certain speed v to enable the plurality of laser lines to be staggered and interpolated;
(5) Splicing laser line images shot by a high-speed digital camera and then importing the laser line images into a computer;
(6) Dividing the spliced laser line image in the step (5) into a starting stage, an effective measuring range and an ending stage;
(7) Calculating the intervals between all adjacent laser lines in the effective measurement range in the image, and finding out the maximum value;
(8) Judging whether the maximum interval is smaller than a specified interval s, and if the maximum interval is smaller than or equal to s, keeping the speed multiple under the condition; if the maximum interval is greater than s, the speed multiple under the condition is not reserved;
(9) Repeating the steps (4) to (8) until the speed multiple VnReaching a set threshold;
(10) According to the reserved speed multiple, determining the range of selectable speed multiple, and finally selecting the maximum speed multiple Vmax
(11) Utilizing the maximum speed multiple V selected in the step (10)maxCalculating the maximum velocity v measured by a plurality of laser linesmax
(12) Maximum velocity v measured from multiple laser linesmaxThe method is applied to a three-dimensional laser detection system of the subway tunnel for actual detection.
2. The method as claimed in claim 1, wherein the step (1) is carried out by the following steps,
(101) Selecting k line laser generators;
(102) The k line laser generators are uniformly assembled according to a certain interval, so that the interval between the head laser line and the tail laser line is D, and the interval between the adjacent laser lines is D
Figure FDA0003781738230000021
(103) Installing the assembled line laser generator in the step (102) and the high-speed digital camera on a mobile platform together;
wherein k is more than or equal to 2, and the unit of the interval D is mm.
3. The method as claimed in claim 2, wherein in step (4), v = v0*Vn,v0Is the initial velocity, v0Has units of mm/s, VnN is the number of multiplication, δ is the increment of multiple, and 1+ n is delta
Figure FDA0003781738230000022
According to the running time t, the shooting times m = t × f of the high-speed digital camera can be calculated, m > k, and f is a frame frequency.
4. The method as claimed in claim 2, wherein the step (6) of dividing the laser line image comprises the steps of,
(601) For the same laser line, the distance between two adjacent positions of the laser line shot by the high-speed digital camera is as follows:
Figure FDA0003781738230000031
v0for initial velocity, f is frame frequency, VnIs a speed multiple;
(602) Let i be an element [1, k ]],j∈[1,m],XijThe coordinate of the j-th position of the ith laser line shot by the high-speed digital camera is represented by the following calculation formula:
Figure FDA0003781738230000032
(603) Dividing the laser line image:
division X1,1~Xk,1To start the orderSegment, Xk,1~X1,mFor effective measurement range, X1,m~Xk,mThe end phase is completed.
5. The method as claimed in claim 2, wherein the step (7) of calculating the spacing between all the adjacent laser lines in the effective measurement range in the image comprises the steps of,
(701) Is provided with
Figure FDA0003781738230000033
dnThe distance between two adjacent laser lines is calculated;
(702)
Figure FDA0003781738230000034
time, effective measurement range is generated
Figure FDA0003781738230000035
Variety of interval values distrThe calculation formula is as follows,
Figure FDA0003781738230000036
(703)
Figure FDA0003781738230000037
time, effective measurement range is generated
Figure FDA0003781738230000038
The seed interval value is calculated by the formula,
Figure FDA0003781738230000039
above provide
Figure FDA0003781738230000041
Another way of calculating the interval value is as follows,
firstly, taking (m-k + 1) group coordinate values,
X1,k,X2,(k-1),...,X(k-1),2,Xk,1
X1,(k+1),X2,k,...,X(k-1),3,Xk,2
X1,m,X2,(m-1),...,X(k-1),(m-k),Xk,(m-k+1)
then, the absolute value of the difference value is taken from the adjacent coordinates in each group of coordinate values, and the interval value is obtained
Figure FDA0003781738230000042
6. The method as claimed in claim 2, wherein in step (11), the maximum speed is calculated as,
vmax=v0*Vmax,v0is the initial velocity.
CN202110360540.0A 2021-04-02 2021-04-02 Measuring method for dislocation interpolation of multiple laser lines Active CN113029041B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110360540.0A CN113029041B (en) 2021-04-02 2021-04-02 Measuring method for dislocation interpolation of multiple laser lines

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110360540.0A CN113029041B (en) 2021-04-02 2021-04-02 Measuring method for dislocation interpolation of multiple laser lines

Publications (2)

Publication Number Publication Date
CN113029041A CN113029041A (en) 2021-06-25
CN113029041B true CN113029041B (en) 2022-11-01

Family

ID=76454218

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110360540.0A Active CN113029041B (en) 2021-04-02 2021-04-02 Measuring method for dislocation interpolation of multiple laser lines

Country Status (1)

Country Link
CN (1) CN113029041B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114577112A (en) * 2022-01-19 2022-06-03 格力电器(芜湖)有限公司 Chassis bolt position detection method and detection device

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2012098265A (en) * 2010-11-02 2012-05-24 Beru Techno:Kk Measuring device of weight, shape, and other property
CN105091782A (en) * 2015-05-29 2015-11-25 南京邮电大学 Multilane laser light plane calibration method based on binocular vision
CN105222724B (en) * 2015-09-10 2018-09-18 北京天远三维科技股份有限公司 Multi-thread array laser 3 D scanning system and multi-thread array laser 3-D scanning method
CN109186491A (en) * 2018-09-30 2019-01-11 南京航空航天大学 Parallel multi-thread laser measurement system and measurement method based on homography matrix
CN112161574B (en) * 2020-10-12 2021-07-16 昆明理工大学 Three-dimensional measurement system and measurement method based on divergent multi-line laser projection

Also Published As

Publication number Publication date
CN113029041A (en) 2021-06-25

Similar Documents

Publication Publication Date Title
JP3600230B2 (en) Architectural and civil engineering structure measurement and analysis system
CN109060821B (en) Tunnel disease detection method and tunnel disease detection device based on laser detection
CN110887460B (en) Tunnel deformation monitoring method and system based on laser scanner and odometer
KR20200098480A (en) Multi-resolution, simultaneous position measurement and mapping based on 3-D LIDAR measurements
CN109522804B (en) Road edge identification method and system
CN102713509B (en) Stereo camera device and correction method
CN113280798B (en) Geometric correction method for vehicle-mounted scanning point cloud under tunnel GNSS rejection environment
CN106959697B (en) Automatic indoor map construction system for long straight corridor environment
CN113029041B (en) Measuring method for dislocation interpolation of multiple laser lines
US9134117B2 (en) Distance measuring system and distance measuring method
CN104966399B (en) A kind of vehicle-speed detecting device and method
CN109635816B (en) Lane line generation method, apparatus, device, and storage medium
CN102155923A (en) Splicing measuring method and system based on three-dimensional target
JPH11252587A (en) Object tracking device
CN110532582B (en) Method for modeling scanning distance sensor
JPH0467607B2 (en)
CN101373560A (en) Method for measuring position and speed of vehicle on highway based on linear array CCD
CN111208542B (en) Motion trail control system, device and method for automatic mobile traffic facility
CN110595390B (en) Stripe projection device based on rectangular pyramid reflector and three-dimensional reconstruction imaging system
JPWO2017158964A1 (en) Image processing apparatus, mobile device control system, image processing program, and image processing method
CN114495512B (en) Vehicle information detection method, system, electronic equipment and readable storage medium
CN112099042A (en) Vehicle tracking method and system
JPH10187974A (en) Physical distribution measuring instrument
JPH1026526A (en) Triangulation type range finding method
JP3390515B2 (en) Position matching method, vehicle speed calculation method, and vehicle speed calculation device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant