CN113483879B - Small satellite flutter high-speed video measurement method - Google Patents

Small satellite flutter high-speed video measurement method Download PDF

Info

Publication number
CN113483879B
CN113483879B CN202110718419.0A CN202110718419A CN113483879B CN 113483879 B CN113483879 B CN 113483879B CN 202110718419 A CN202110718419 A CN 202110718419A CN 113483879 B CN113483879 B CN 113483879B
Authority
CN
China
Prior art keywords
speed
small satellite
flutter
satellite
speed camera
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110718419.0A
Other languages
Chinese (zh)
Other versions
CN113483879A (en
Inventor
刘世杰
陈鹏
童小华
汪本康
谢欢
冯永玖
金雁敏
许雄
柳思聪
叶真
王超
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tongji University
Original Assignee
Tongji University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tongji University filed Critical Tongji University
Priority to CN202110718419.0A priority Critical patent/CN113483879B/en
Publication of CN113483879A publication Critical patent/CN113483879A/en
Application granted granted Critical
Publication of CN113483879B publication Critical patent/CN113483879B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G01MEASURING; TESTING
    • G01HMEASUREMENT OF MECHANICAL VIBRATIONS OR ULTRASONIC, SONIC OR INFRASONIC WAVES
    • G01H9/00Measuring mechanical vibrations or ultrasonic, sonic or infrasonic waves by using radiation-sensitive means, e.g. optical means
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01BMEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00Measuring arrangements characterised by the use of optical techniques
    • G01B11/02Measuring arrangements characterised by the use of optical techniques for measuring length, width or thickness
    • G01B11/03Measuring arrangements characterised by the use of optical techniques for measuring length, width or thickness by measuring coordinates of points
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02TCLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T90/00Enabling technologies or technologies with a potential or indirect contribution to GHG emissions mitigation

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Image Analysis (AREA)

Abstract

The invention relates to a method for measuring a small satellite flutter high-speed video, which comprises the following steps: 1) Constructing a high-speed video measurement system; 2) And shooting the vibrating small satellite by a high-speed camera, tracking, matching and three-dimensional reconstructing the target point stuck on the surface of the small satellite to obtain a time-course displacement curve of the target point, and calculating according to the displacement to obtain the flutter frequency of the small satellite. Compared with the prior art, the invention has the advantages of simplicity, accuracy and the like.

Description

Small satellite flutter high-speed video measurement method
Technical Field
The invention relates to the field of satellite video data processing, in particular to a method for measuring a small satellite flutter high-speed video.
Background
The satellite flutter refers to the gesture shake generated by the influence of factors such as a dynamic structure and gesture control on the satellite during the in-orbit running of the satellite, and the higher the resolution and agility of the satellite, the more remarkable the flutter influence of the platform. The flutter is a complex phenomenon commonly existing in the on-orbit operation of a small satellite platform, influences the imaging quality, geometric positioning and mapping precision of the satellite, is a key problem to be solved in the development of the small satellite, and an on-line detection method for the satellite flutter is not available at present.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provide a method for measuring the flutter high-speed video of a small satellite.
The aim of the invention can be achieved by the following technical scheme:
a method for measuring a small satellite flutter high-speed video comprises the following steps:
1) Constructing a high-speed video measurement system;
the high-speed video measurement system comprises an upper computer, a high-speed camera, a high-speed image acquisition card, a memory card, a synchronous controller and a total station, wherein the number of the high-speed cameras is two to form a binocular measurement network, the high-speed camera sends acquired small satellite vibration sequence images to the upper computer through the high-speed image acquisition card and the memory card, and the synchronous controller is used for realizing synchronous shooting of the two high-speed cameras;
the small satellite is hung below the bracket and is used for simulating the flutter condition of the satellite in the weightless environment; in the experimental process, the rotation speed of a momentum wheel arranged in a small satellite is controlled, so that the satellite flutters to a certain extent;
2) And shooting the vibrating small satellite by a high-speed camera, tracking, matching and three-dimensional reconstructing the target point stuck on the surface of the small satellite to obtain a time-course displacement curve of the target point, and calculating according to the displacement to obtain the flutter frequency of the small satellite.
The high-speed camera is a CameRecord CL600x2 type high-speed camera.
The high-speed video measurement system adopts a disk array consisting of 8 SSD hard disks as a long-term storage medium.
The network construction of the high-speed video measurement system comprises the following steps:
11 Design of a manual sign:
the artificial mark consists of a white circle and a black boundary and is stuck on the satellite surface;
12 The control network is arranged for three-dimensional calibration.
The step 2) specifically comprises the following steps:
21 Acquiring internal and external azimuth parameters of the high-speed camera through three-dimensional calibration;
22 For the same-name image point on the small satellite image shot by the high-speed camera, calculating to obtain the three-dimensional coordinate of the target point through a front intersection algorithm according to the two-dimensional coordinate of the target point and the internal and external azimuth parameters, and further obtaining a time-course displacement curve of the target point;
23 Obtaining the flutter frequency spectrum of the target point time-course displacement curve, and obtaining the flutter frequency of the minisatellite through calculation.
In the step 21), the internal azimuth parameters of the high-speed camera are obtained by adopting a calibration method of a plane calibration plate, and the external azimuth parameters of the high-speed camera are obtained through space translation and rotation transformation between the high-speed camera and the calibration plate.
In the step 21), the calibrated high-speed camera is unchanged in position in the shooting process.
In the step 22), three unknowns of the three-dimensional coordinates are solved by establishing four linear equations for a pair of image points with the same name according to the internal and external azimuth parameters of each high-speed camera, and in the calculation process, the final coordinate result in the three-dimensional reconstruction is obtained by a linear least square method.
In the step 23), the time-course displacement curve is used as the sequence data signal f (x), and the expression of the flutter spectrum of the time-course displacement curve is obtained as follows:
Figure GDA0003820383790000021
wherein a is 0 2 is the DC component, a n Fourier coefficients, b, being cosine terms n For the Fourier coefficient of the sine term, each frequency n corresponds to an amplitude of
Figure GDA0003820383790000022
Compared with the prior art, the invention has the following advantages:
the invention provides a small satellite flutter high-speed video measurement method, which uses a high-speed camera to shoot the vibration process of a small satellite in a weightless environment, and calculates the displacement of a target point stuck on the surface of the small satellite based on the high-speed video measurement method, so as to calculate the flutter frequency and amplitude of the satellite, and is simple and accurate.
Drawings
Fig. 1 is a schematic layout of a high-speed video measurement system.
FIG. 2 is a schematic illustration of an artificial landmark annotation.
Fig. 3 shows displacement in the X direction.
Fig. 4 shows the Y-direction displacement.
Fig. 5 shows Z-direction displacement.
Fig. 6 is the total displacement.
Fig. 7 shows the spectrum analysis result, wherein fig. 7a shows the time domain analysis result, and fig. 7b shows the frequency domain analysis result.
Fig. 8 is a flow chart of the method of the present invention.
Detailed Description
The invention will now be described in detail with reference to the drawings and specific examples.
The invention provides a method for measuring a small satellite flutter high-speed video, which comprises the following steps:
1) Constructing a high-speed video measurement system according to a small satellite flutter ground experiment scene;
2) Obtaining displacement of the small satellite surface mark point in three directions (X, Y, Z) by using a high-speed photogrammetry technology;
3) And representing the calculated displacement information of the target point in the time domain in the frequency domain through Fourier transform, and solving the flutter main frequency.
Finally, the method is verified in a vibration simulation test of the minisatellite under the weightless environment,
the specific description of each step is as follows:
step 1: high-speed video measurement system construction
In the process of satellite vibration, a high-speed camera is used for shooting the satellite, a time-course displacement curve of a target point is obtained by tracking and matching and three-position reconstruction of the target point stuck on the surface of the small satellite, and finally the flutter frequency of the satellite is calculated according to the displacement.
In this example, a high-speed video measurement system is constructed, 2 cameras with the camera record CL600x2 high-speed are used to form a stereoscopic vision combination for shooting, the imaging range is about 2m×2m, the distance between the cameras and the satellite is about 1.5m, and the specific layout positions of the cameras are shown in fig. 1.
1.1 hardware configuration of high-speed video measurement System
(1) High-speed camera
The camera parameters used in this experiment are shown in table 1.
Table 1 CL600x2 camera parameters
Figure GDA0003820383790000041
The maximum resolution of the CameRecord CL600x2 high-speed camera from Optronis is 1280 x 1024 pixels and the frame rate is 500fps.
(2) High-speed image acquisition card and memory card
The computer system adopts a disk array consisting of 8 SSD hard disks as a long-term storage medium, the writing speed is about 10000Mbps, and the real-time acquisition and storage of high-speed images are supported. The memory of the computer system is set to be 1TB, the longest running time of single acquisition is 149s, and the longest single acquisition is not more than 605s under the longest working condition.
(3) Synchronous controller
So that the synchronization accuracy of the synchronization line does not exceed 100ns.
(4) High-precision total station
The point position measuring precision is better than 0.5mm.
1.2 network construction of high-speed video measurement System
(1) Artificial mark design
The circular mark consists of white circle and black boundary, and is mainly used for three-dimensional measurement of accurate point location, and the circle center mark point can be stuck on the satellite surface, and the position of the mark point is shown in figure 2.
(2) Control net layout
Two orientation schemes are provided in this example:
the relative orientation is not controlled, namely, the three-dimensional calibration is carried out by using a calibration plate;
before the experiment, the position of the control point is measured, and the following needs to be satisfied: the control points are uniformly distributed in the image captured by the high-speed camera. (the total station is required to measure the coordinates of the space point before the experiment)
Step two: small satellite platform flutter parameter resolution
2.1 three-dimensional coordinate reconstruction of marker points
In this example, a calibration method based on a planar calibration plate is used to calculate the internal azimuth parameters of the stereo camera, and this type of calibration method has the advantages of flexibility, reliability and low cost. To obtain these high-precision internal azimuth parameters, the higher order radial lens distortion and tangential lens distortion parameters should be further considered. However, unlike conventional single-camera calibration, stereo camera calibration also requires the determination of external direction parameters for each camera. Therefore, the cameras for simultaneous measurement should synchronously collect the images of the calibration plates, and the external azimuth parameters of the stereo camera can be calculated through the space translation and rotation transformation between each camera and the calibration plate. In addition, it should be noted that the calibrated camera remains stationary during the experiment. In the subsequent data processing, each time the two-dimensional coordinates of any image point of the same name are obtained, the three-dimensional coordinates of the target point can be calculated by the front intersection algorithm, and the collinearly conditional equation can be obtained by a series of mathematical transformations as shown in the following formula:
Figure GDA0003820383790000051
in the above formula, (x, y) represents the corrected image coordinates of the corresponding point; f is the camera image distance; (X) S ,Y S ,Z S ) Line parameters as external orientation to represent three-dimensional space coordinates of the perspective center; (a) 1 ,a 2 ,a 3 ,b 1 ,b 2 ,b 3 ,c 1 ,c 2 ,c 3 ) Are elements of a rotation matrix, and are formed by three angles of external orientationDegree element composition; (X) A ,Y A ,Z A ) Unknown spatial coordinates of the non-target point. With accurate stereo calibration, the inside and outside orientation parameters of each industrial camera have been considered as known values. For a pair of homonymous points, four linear equations may be established to solve for the three unknowns. In this calculation process, the final coordinate result in the three-dimensional reconstruction can be directly calculated by a linear least square method.
As the most basic structural dynamics parameters, the three-dimensional displacement of the full-field target point can be obtained by the coordinate difference of the initial coordinate and the subsequent coordinate.
2.2 Flutter Spectrum analysis
The vibration spectrum parameter is also one of parameters describing the vibration characteristics of the small satellite platform, and the simple harmonic component of the sequence data (or signal) f (x) under different frequencies can be obtained by carrying out Fourier transformation on the single-dimensional sequence data. The amplitude of each component represents the intensity of the component, so that the intensity of all frequency components is displayed, and the frequency spectrum of the signal can be obtained, and the method comprises the following steps:
Figure GDA0003820383790000052
in the above formula, assuming that the function period is 2pi, the fourier series of the general function f (x) contains both a sine term and a cosine term; a, a 0 2 is a function DC component, a n Fourier coefficients, b, being cosine terms n Fourier coefficients of sinusoidal terms, and therefore the amplitude of each frequency is
Figure GDA0003820383790000061
Examples
Test analysis
(1) Experimental scenario
The small satellite in the experiment is hung below the bracket and is used for simulating the flutter condition of the satellite in the weightless environment.
In the experimental process, the rotation speed of a momentum wheel arranged in a small satellite is controlled, so that the satellite is dithered to a certain extent.
(2) Experimental results
1. Three-dimensional space coordinate calculation result
The experiment uses a beam method adjustment method to respectively process the sequence images shot under the working condition. The position of the target point on each frame of image is obtained by matching and tracking the target point attached to the vibration table, the coordinate of the control point is measured according to the total station, the displacement of the calculated target point in the XYZ three directions on each frame of image is based on a collineation equation, as shown in figures 3-4, the displacement about 0.03mm is measured in the X and Z directions by the high-speed video measuring method, and the maximum displacement is 0.08mm in the Y direction as shown in figure 5.
2. Spectral analysis results
And representing the calculated displacement information of the target point in the time domain in the frequency domain through Fourier transform to solve the flutter main frequency, wherein the calculation result is shown in figure 7.
3. Precision analysis
In order to evaluate the resolving precision of the experimental result, 9 control points are brought into the adjustment of the beam method to solve the three-dimensional space coordinates of the target point in the resolving process, and 4 control point coordinates which are involved in resolving are used for checking the resolving precision. Comparing the three-dimensional space calculation result of the 4 points with the three-dimensional space result measured by the total station, as shown in Table 2
Table 2 comparison of the three-dimensional space solution results with the three-dimensional space results measured by the total station
Figure GDA0003820383790000062
Figure GDA0003820383790000071
It can be seen from table 2 that the static three-dimensional reconstruction error can reach sub-millimeter accuracy with a maximum error of 0.357 millimeters.

Claims (6)

1. The method for measuring the flutter high-speed video of the small satellite is characterized by comprising the following steps of:
1) Constructing a high-speed video measurement system;
the high-speed video measurement system comprises an upper computer, a high-speed camera, a high-speed image acquisition card, a memory card, a synchronous controller and a total station, wherein the number of the high-speed cameras is two to form a binocular measurement network, the high-speed camera sends acquired small satellite vibration sequence images to the upper computer through the high-speed image acquisition card and the memory card, and the synchronous controller is used for realizing synchronous shooting of the two high-speed cameras;
the small satellite is hung below the bracket and is used for simulating the flutter condition of the satellite in the weightless environment; in the experimental process, the rotation speed of a momentum wheel arranged in a small satellite is controlled, so that the satellite flutters to a certain extent;
2) Shooting a vibrating small satellite by a high-speed camera, and obtaining a time-course displacement curve of a target point by tracking, matching and three-dimensional reconstruction of the target point stuck on the surface of the small satellite, and calculating the flutter frequency of the small satellite according to the displacement, wherein the method specifically comprises the following steps:
21 Acquiring internal and external azimuth parameters of the high-speed camera through three-dimensional calibration;
22 For the same-name image point on the small satellite image shot by the high-speed camera, calculating to obtain the three-dimensional coordinate of the target point through a front intersection algorithm according to the two-dimensional coordinate of the target point and the internal and external azimuth parameters, and further obtaining a time-course displacement curve of the target point;
23 Obtaining the flutter frequency spectrum of the target point time-course displacement curve, and calculating to obtain the flutter frequency of the minisatellite, wherein the flutter frequency is specifically as follows: taking the time-course displacement curve as a sequence data signal f (x), and acquiring the expression of the flutter frequency spectrum of the time-course displacement curve as follows:
Figure FDA0004146769020000011
wherein a is 0 And/2 is DCComponent, a n Fourier coefficients, b, being cosine terms n For the Fourier coefficient of the sine term, each frequency n corresponds to an amplitude of
Figure FDA0004146769020000012
The high-speed camera is a CameRecord CL600x2 type high-speed camera.
2. The method for measuring the flutter high-speed video of the small satellite according to claim 1, wherein the high-speed video measuring system adopts a disk array consisting of 8 SSD hard disks as a long-term storage medium.
3. The method for measuring the flutter high-speed video of a small satellite according to claim 1, wherein the network construction of the high-speed video measuring system comprises the following steps:
11 Design of a manual sign:
the artificial mark consists of a white circle and a black boundary and is stuck on the satellite surface;
12 The control network is arranged for three-dimensional calibration.
4. The method for measuring the flutter high-speed video of the small satellite according to claim 1, wherein in the step 21), the internal azimuth parameters of the high-speed camera are obtained by adopting a calibration method of a plane calibration plate, and the external azimuth parameters of the high-speed camera are obtained by space translation and rotation transformation between the high-speed camera and the calibration plate.
5. The method for measuring the flutter high-speed video of a small satellite according to claim 1, wherein in the step 21), the calibrated high-speed camera is unchanged in position during the shooting process.
6. The method of claim 1, wherein in the step 22), three unknowns of three-dimensional coordinates are solved by establishing four linear equations for a pair of image points of the same name according to the internal and external azimuth parameters of each high-speed camera, and in the calculation process, the final coordinate result in three-dimensional reconstruction is obtained by a linear least square method.
CN202110718419.0A 2021-06-28 2021-06-28 Small satellite flutter high-speed video measurement method Active CN113483879B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110718419.0A CN113483879B (en) 2021-06-28 2021-06-28 Small satellite flutter high-speed video measurement method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110718419.0A CN113483879B (en) 2021-06-28 2021-06-28 Small satellite flutter high-speed video measurement method

Publications (2)

Publication Number Publication Date
CN113483879A CN113483879A (en) 2021-10-08
CN113483879B true CN113483879B (en) 2023-06-02

Family

ID=77936106

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110718419.0A Active CN113483879B (en) 2021-06-28 2021-06-28 Small satellite flutter high-speed video measurement method

Country Status (1)

Country Link
CN (1) CN113483879B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114545959B (en) * 2022-02-24 2024-01-12 中国人民解放军战略支援部队航天工程大学 Remote sensing satellite platform control and image correction method based on flutter information

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106989812A (en) * 2017-05-03 2017-07-28 湖南科技大学 Large fan blade modal method of testing based on photogrammetric technology
CN111207895A (en) * 2020-01-13 2020-05-29 中国科学院微小卫星创新研究院 Ground micro-vibration experiment system and method for remote sensing micro-nano satellite

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103482088B (en) * 2013-08-12 2015-07-15 上海卫星工程研究所 Satellite micro-vibration test multi-point suspension system and design method thereof
CN103983343B (en) * 2014-05-29 2016-05-11 武汉大学 A kind of satellite platform based on multispectral image tremble detection method and system
CN105424350B (en) * 2015-12-19 2017-10-31 湖南科技大学 Thin-wall part mode testing method and system based on machine vision
US9922428B2 (en) * 2016-08-19 2018-03-20 Crystal Instruments Corporation Vibration image acquisition and processing
CN107228748A (en) * 2017-06-16 2017-10-03 华南理工大学 Satellite antenna structural vibration measurement apparatus and method based on non-contact measurement
CN110047110B (en) * 2019-03-11 2021-06-11 北京空间飞行器总体设计部 Flexible satellite-borne antenna on-orbit vibration measurement method based on sequence image
CN111649857A (en) * 2020-04-23 2020-09-11 河海大学 Inhaul cable modal measurement method for target matching analysis
CN111854632B (en) * 2020-06-22 2021-12-14 新拓三维技术(深圳)有限公司 Image measuring method of high-speed moving object and computer readable storage medium

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106989812A (en) * 2017-05-03 2017-07-28 湖南科技大学 Large fan blade modal method of testing based on photogrammetric technology
CN111207895A (en) * 2020-01-13 2020-05-29 中国科学院微小卫星创新研究院 Ground micro-vibration experiment system and method for remote sensing micro-nano satellite

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
《双目立体工业摄影测量关键技术研究与应用》;于英;《中国优秀硕士学位论文全文数据库 信息科技辑》;20120215(第2期);第7页 *

Also Published As

Publication number Publication date
CN113483879A (en) 2021-10-08

Similar Documents

Publication Publication Date Title
CN110296691A (en) Merge the binocular stereo vision measurement method and system of IMU calibration
Gorjup et al. Still-camera multiview Spectral Optical Flow Imaging for 3D operating-deflection-shape identification
CN108535097A (en) A kind of method of triaxial test sample cylindrical distortion measurement of full field
Liu et al. An improved online dimensional measurement method of large hot cylindrical forging
CN105486289B (en) A kind of laser photography measuring system and camera calibration method
GB2554796A (en) Testing 3D imaging systems
CN107589069B (en) Non-contact type measuring method for object collision recovery coefficient
CN100376883C (en) Pixel frequency based star sensor high accuracy calibration method
CN107038753B (en) Stereoscopic vision three-dimensional reconstruction system and method
CN108154535B (en) Camera calibration method based on collimator
CN113483879B (en) Small satellite flutter high-speed video measurement method
CN103389072A (en) An image point positioning precision assessment method based on straight line fitting
Chen et al. Low-speed-camera-array-based high-speed three-dimensional deformation measurement method: Principle, validation, and application
CN112985258B (en) Calibration method and measurement method of three-dimensional measurement system
Song et al. Flexible line-scan camera calibration method using a coded eight trigrams pattern
CN109087341A (en) A kind of fusion method of short distance EO-1 hyperion camera and distance measuring sensor
CN114964316B (en) Position and attitude calibration method and device, and method and system for measuring target to be measured
Cai et al. Near-infrared camera calibration for optical surgical navigation
CN113724371B (en) Three-dimensional imaging method, system, electronic device and storage medium for coaxial illumination light field
CN108364325A (en) Regular sample X ray CT perspective view position translation separate-blas estimation and bearing calibration
CN113790710A (en) Three-dimensional positioning verification method and device for high-speed video underwater structure
CN110806572B (en) Device and method for testing distortion of long-focus laser three-dimensional imager based on angle measurement method
CN114062265A (en) Method for evaluating stability of supporting structure of visual system
CN114170321A (en) Camera self-calibration method and system based on distance measurement
Liu et al. Comparison study of three camera calibration methods considering the calibration board quality and 3D measurement accuracy

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant