CN108253964A - A kind of vision based on Time-Delay Filter/inertia combined navigation model building method - Google Patents

A kind of vision based on Time-Delay Filter/inertia combined navigation model building method Download PDF

Info

Publication number
CN108253964A
CN108253964A CN201711477073.XA CN201711477073A CN108253964A CN 108253964 A CN108253964 A CN 108253964A CN 201711477073 A CN201711477073 A CN 201711477073A CN 108253964 A CN108253964 A CN 108253964A
Authority
CN
China
Prior art keywords
navigation
error
speed
inertial navigation
carrier
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201711477073.XA
Other languages
Chinese (zh)
Inventor
李庆华
瞿敏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Qilu University of Technology
Original Assignee
Qilu University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Qilu University of Technology filed Critical Qilu University of Technology
Priority to CN201711477073.XA priority Critical patent/CN108253964A/en
Publication of CN108253964A publication Critical patent/CN108253964A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G01MEASURING; TESTING
    • G01CMEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/10Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration
    • G01C21/12Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning
    • G01C21/16Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation
    • G01C21/165Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01CMEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/20Instruments for performing navigational calculations

Abstract

The present invention relates to a kind of vision based on Time-Delay Filter/inertia combined navigation model building methods, it is when being corrected in the position using vision guided navigation and velocity information to inertial navigation parameter, the delay time error of integrated navigation is added in filter state equation, estimated by filtering, not only inertial navigation parameter error is estimated, while obtains the time synchronization error of integrated navigation.Solve the problems, such as that integrated navigation parameter calculation is nonsynchronous.The present invention has the functions such as inertial navigation resolving, vision guided navigation parameter calculation, structure Time-Delay Filter.Compared with prior art, present aspect has the advantages that theoretical tight, method novelty, performance efficiency.

Description

A kind of vision based on Time-Delay Filter/inertia combined navigation model building method
Technical field
The present invention relates to integrated navigation field, more particularly to a kind of vision/inertia combined navigation based on Time-Delay Filter Model building method.
Background technology
Inertial navigation system(Inertial Navigation System INS)Independence, good concealment can be carried continuously For the navigational parameter of a variety of degree of precision, antijamming capability is good, is a kind of common air navigation aid.But shortcoming be error at any time Accumulation, the navigation accuracy to work long hours is poor, in order to eliminate accumulative error, needs to introduce other outer navigation system to inertia Navigation is corrected, and forms the higher integrated navigation system of navigation accuracy.But during integrated navigation, vision guided navigation is adopted due to image Collection resolves reason with vision guided navigation, and navigational parameter output has delay, when inertial navigation algorithm carries out navigational parameter resolving, Output also has delay.Therefore it is necessary to solve the problems, such as time synchronization, accurate navigator fix can be only achieved.Do not have still at present The integrated navigation research of complete vision guided navigation/inertial navigation based on Time-Delay Filter, therefore the present invention has very strong theory Directive significance.
Invention content
The present invention is intended to provide a kind of vision based on Time-Delay Filter/inertia combined navigation model building method, with solution The deficiency that certainly navigation accuracy caused by integrated navigation time synchronization error lacks realizes the higher integrated navigation system of precision.
Technical scheme is as follows:
The method that a kind of vision guided navigation/inertial navigation solves navigational parameter, this method include the following steps:
S1, inertial navigation system parameter calculation:Local northeast day geographic coordinate system is chosen as navigational coordinate system, gyroscope is sensitive The angular speed of carrier, accelerometer carry out the non-gravitation i.e. specific force of sensitive carrier, and the posture of carrier is obtained by attitude algorithm formula Transition matrix, so as to find out the attitude information of carrier;Carrier speed over the ground and the current position of carrier is obtained in navigation calculation formula It puts;
S2, vision guided navigation parameter calculation method:Camera is demarcated first, camera imaging model parameter is obtained;Pass through image Processing Algorithm trains navigation marker, by connecting firmly the vehicle-mounted camera disposed downwards in mobile robot front vertical, obtains and moves The image sequence on mobile robot passed by road surface during the motion is then based on the feature point tracking matching algorithm of SURF, with And pixel displacement of the tracked characteristic point between two field pictures, speed is calculated in camera imaging model;By being connected in Camera in front of carrier, the filmed image during carrier movement, and stencil matching is carried out, successful match performs S3, otherwise holds Row S1;
S3, structure time delay Kalman filter:The state of time delay Kalman filter be 5 states, the velocity error of inertial navigation, position Error and the time synchronization error newly increased.The error equation of inertial navigation is taken to be resolved for filter state equation using vision guided navigation Position that obtained carrier positions, speed and inertial navigation resolves, speed are subtracted each other, and obtained difference is as Kalman filtering Measurement;
S4, estimated by the filtering of time delay Kalman filter, estimate inertial navigation parameter error and device error and regard The time delays margin of error of feel/inertia combined navigation so as to carry out output feedback compensation to inertial navigation, obtains more accurately Navigation accuracy.
The beneficial effects of the invention are as follows:The present invention is for the error accumulation and during integrated navigation at any time of inertial navigation precision Between the problems such as being delayed, be taken through building the Kalman filter based on time delay, by vision guided navigation to inertial navigation parameter While carrying out feedback compensation, the error accumulation problem of inertial navigation is not only solved, while it is same to solve the integrated navigation time The problem of step, substantially increases navigation accuracy.
Description of the drawings
The invention will be further described below in conjunction with the accompanying drawings.
Fig. 1 is system schema block diagram.
Fig. 2 is the fundamental diagram of integrated navigation delay calibration.
Site error estimations of the Fig. 3 without delay calibration.
Site error estimations of the Fig. 4 by delay calibration.
The site error estimation of Fig. 5 integrated navigations.
Fig. 6 integrated navigations velocity error is estimated.
Specific embodiment
The present invention is further described with reference to specific embodiment.
Embodiment 1
The step of building the integrated navigation model method based on Time-Delay Filter for the embodiment as shown in Figure 2:
S1, inertial navigation system parameter calculation:Local northeast day geographic coordinate system is chosen as navigational coordinate system, gyroscope is sensitive The angular speed of carrier, accelerometer carry out the non-gravitation i.e. specific force of sensitive carrier, and the posture of carrier is obtained by attitude algorithm formula Transition matrix, so as to find out the attitude information of carrier;Carrier speed over the ground and the current position of carrier is obtained in navigation calculation formula It puts;
S2, vision guided navigation parameter calculation method:Camera is demarcated first, camera imaging model parameter is obtained;Pass through image Processing Algorithm trains navigation marker, by connecting firmly the vehicle-mounted camera disposed downwards in mobile robot front vertical, obtains and moves The image sequence on mobile robot passed by road surface during the motion is then based on the feature point tracking matching algorithm of SURF, with And pixel displacement of the tracked characteristic point between two field pictures, speed is calculated in camera imaging model;By being connected in Camera in front of carrier, the filmed image during carrier movement, and stencil matching is carried out, successful match performs S3, otherwise holds Row S1;
S3, structure time delay Kalman filter:The state of time delay Kalman filter be 5 states, the velocity error of inertial navigation, position Error and the time synchronization error newly increased.The error equation of inertial navigation is taken to be resolved for filter state equation using vision guided navigation Position that obtained carrier positions, speed and inertial navigation resolves, speed are subtracted each other, and obtained difference is as Kalman filtering Measurement;The error equation of inertial navigation system is taken as the state equation of Kalman filter, filter state is velocity error, position Error and newly-increased integrated navigation time synchronization error Liang ⊿ (t), X=F, wherein, X is error state vector, Packet inertial navigation site error, velocity errorAnd time synchronization error Liang ⊿ (t);The position that vision guided navigation is calculated Put, the position that speed and inertial navigation are calculated, speed measuring value of the difference as Kalman filtering, measurement equation is,,For the east orientation and north orientation speed of inertial navigation,For The east orientation speed and north orientation speed of vision guided navigation.For the location parameter of inertial navigation,For vision guided navigation Location parameter, the measurement equation of the extended Kalman filter of time of fusion error are Z=HX+V, wherein Z=, amount Survey matrix H is 4*5 matrixes:Its nonzero element is:H (1 1)=1, H (2 2)=1, H (3 3)=1, H (4 4)=1, H (1 5)= , H (2 5)=, H (3 5)=/R, H (4 5)=/ R, whereinAcceleration for INS is flat in filtering cycle Mean value,For measuring values of the INS in filtering, R is earth radius;
S4, estimated by the filtering of time delay Kalman filter, estimate inertial navigation parameter error and device error and regard The time delays margin of error of feel/inertia combined navigation so as to carry out output feedback compensation to inertial navigation, obtains more accurately Navigation accuracy.
Finally illustrate, the above embodiments are merely illustrative of the technical solutions of the present invention and it is unrestricted, although with reference to compared with The present invention is described in detail in good embodiment, it will be understood by those of ordinary skill in the art that, it can be to the skill of the present invention Art scheme is modified or replaced equivalently, and without departing from the objective and range of technical solution of the present invention, should all be covered at this In the right of invention.

Claims (4)

1. a kind of vision based on Time-Delay Filter/inertia combined navigation model building method, it is characterized in that, there is following step Suddenly:
S1, inertial navigation system parameter calculation:Local northeast day geographic coordinate system is chosen as navigational coordinate system, gyroscope is sensitive The angular speed of carrier, accelerometer carry out the non-gravitation i.e. specific force of sensitive carrier, and the posture of carrier is obtained by attitude algorithm formula Transition matrix, so as to find out the attitude information of carrier;Carrier speed over the ground and the current position of carrier is obtained in navigation calculation formula It puts;
S2, vision guided navigation parameter calculation method:Camera is demarcated first, camera imaging model parameter is obtained;Pass through image Processing Algorithm trains navigation marker, by connecting firmly the vehicle-mounted camera disposed downwards in mobile robot front vertical, obtains and moves The image sequence on mobile robot passed by road surface during the motion is then based on the feature point tracking matching algorithm of SURF, with And pixel displacement of the tracked characteristic point between two field pictures, speed is calculated in camera imaging model;By being connected in Camera in front of carrier, the filmed image during carrier movement, and stencil matching is carried out, successful match performs S3, otherwise holds Row S1;
S3, structure time delay Kalman filter:The state of time delay Kalman filter be 5 states, the velocity error of inertial navigation, position Error and the time synchronization error newly increased take the error equation of inertial navigation to be resolved for filter state equation using vision guided navigation Position that obtained carrier positions, speed and inertial navigation resolves, speed are subtracted each other, and obtained difference is as Kalman filtering Measurement;
S4, estimated by the filtering of time delay Kalman filter, estimate inertial navigation parameter error and device error and regard The time delays margin of error of feel/inertia combined navigation so as to carry out output feedback compensation to inertial navigation, obtains more accurately Navigation accuracy.
2. inertial reference calculation method according to claim 1 is characterized in that:In S1, including:INS Platform is initially aligned, posture Transformation matrixUpdate, attitude angle extraction, navigational parameter calculate.
3. vision guided navigation parameter calculation method according to claim 1, it is characterized in that:In S2, camera calibration:Pass through mark Fixed board solves camera calibration the inside and outside parameter of video camera;By the tracking and matching method based on SURF characteristic points, according to figure As the detection, tracking and matching of the characteristic point between sequence, pixel displacement of the tracked characteristic point between two images is calculated, The speed and location information of mobile robot are calculated according to the imaging model of video camera.
4. time delay Kalman filter construction method according to claim 1, it is characterized in that:In S3, inertial navigation system is taken State equation of the error equation as Kalman filter, filter state are velocity error, site error and newly-increased combination Navigation time synchronous error Liang ⊿ (t), X=F, wherein, X is error state vector, packet inertial navigation site error, speed Spend errorAnd time synchronization error Liang ⊿ (t);Position, speed and the inertial navigation that vision guided navigation is calculated calculate Measuring value of the obtained position, the difference of speed as Kalman filtering, measurement equation are,,For the east orientation and north orientation speed of inertial navigation,For the east orientation speed and north orientation speed of vision guided navigation,For the location parameter of inertial navigation,For the location parameter of vision guided navigation, the extension karr of time of fusion error The measurement equation of graceful wave filter be Z=HX+V, wherein Z=, measurement matrix H is 4*5 matrixes:Its nonzero element For:H (1 1)=1, H (2 2)=1, H (3 3)=1, H (4 4)=1, H (1 5)=H(2 5)=, H (3 5)=/R, H (4 5)=/ R is whereinAverage value of the acceleration in filtering cycle for INS,For INS filtering when measuring value, R is earth radius.
CN201711477073.XA 2017-12-29 2017-12-29 A kind of vision based on Time-Delay Filter/inertia combined navigation model building method Pending CN108253964A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201711477073.XA CN108253964A (en) 2017-12-29 2017-12-29 A kind of vision based on Time-Delay Filter/inertia combined navigation model building method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201711477073.XA CN108253964A (en) 2017-12-29 2017-12-29 A kind of vision based on Time-Delay Filter/inertia combined navigation model building method

Publications (1)

Publication Number Publication Date
CN108253964A true CN108253964A (en) 2018-07-06

Family

ID=62724633

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201711477073.XA Pending CN108253964A (en) 2017-12-29 2017-12-29 A kind of vision based on Time-Delay Filter/inertia combined navigation model building method

Country Status (1)

Country Link
CN (1) CN108253964A (en)

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109443353A (en) * 2018-12-25 2019-03-08 中北大学 Vision based on fuzzy self-adaption ICKF-inertia close coupling Combinated navigation method
CN109443355A (en) * 2018-12-25 2019-03-08 中北大学 Vision based on adaptive Gauss PF-inertia close coupling Combinated navigation method
CN109724598A (en) * 2019-03-08 2019-05-07 哈尔滨工程大学 A kind of estimation and compensation method of the time delay error of GNSS/INS pine combination
CN110068325A (en) * 2019-04-11 2019-07-30 同济大学 A kind of lever arm error compensating method of vehicle-mounted INS/ visual combination navigation system
CN110375739A (en) * 2019-06-26 2019-10-25 中国科学院深圳先进技术研究院 A kind of mobile terminal vision fusion and positioning method, system and electronic equipment
CN110702139A (en) * 2019-09-29 2020-01-17 百度在线网络技术(北京)有限公司 Time delay calibration method and device, electronic equipment and medium
CN110880189A (en) * 2018-09-06 2020-03-13 舜宇光学(浙江)研究院有限公司 Combined calibration method and combined calibration device thereof and electronic equipment
CN111044069A (en) * 2019-12-16 2020-04-21 驭势科技(北京)有限公司 Vehicle positioning method, vehicle-mounted equipment and storage medium
CN111551191A (en) * 2020-04-28 2020-08-18 浙江商汤科技开发有限公司 Sensor external parameter calibration method and device, electronic equipment and storage medium
CN114136315A (en) * 2021-11-30 2022-03-04 山东天星北斗信息科技有限公司 Monocular vision-based auxiliary inertial integrated navigation method and system
CN116380148A (en) * 2023-04-06 2023-07-04 中国人民解放军93209部队 Two-stage space-time error calibration method and device for multi-sensor target tracking system

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104833352A (en) * 2015-01-29 2015-08-12 西北工业大学 Multi-medium complex-environment high-precision vision/inertia combination navigation method
CN105910602A (en) * 2016-05-30 2016-08-31 南京航空航天大学 Combined navigation method
US20170053538A1 (en) * 2014-03-18 2017-02-23 Sri International Real-time system for multi-modal 3d geospatial mapping, object recognition, scene annotation and analytics
CN107063246A (en) * 2017-04-24 2017-08-18 齐鲁工业大学 A kind of Loosely coupled air navigation aid of vision guided navigation/inertial navigation

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170053538A1 (en) * 2014-03-18 2017-02-23 Sri International Real-time system for multi-modal 3d geospatial mapping, object recognition, scene annotation and analytics
CN104833352A (en) * 2015-01-29 2015-08-12 西北工业大学 Multi-medium complex-environment high-precision vision/inertia combination navigation method
CN105910602A (en) * 2016-05-30 2016-08-31 南京航空航天大学 Combined navigation method
CN107063246A (en) * 2017-04-24 2017-08-18 齐鲁工业大学 A kind of Loosely coupled air navigation aid of vision guided navigation/inertial navigation

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
JE YOUNG LEE 等: "Adaptive GPS/INS integration for relative navigation", 《GPS SOLUTION》 *
张朝飞等: "一种复杂多介质环境下的视觉/惯性自适应组合导航方法 ", 《中国惯性技术学报》 *
秦玉鑫: "移动机器人组合导航系统研究与设计", 《中国优秀硕士学位论文全文数据库》 *
陈林 等: "INS/Vision组合导航中视觉系统动态定位方法研究", 《传感技术学报》 *

Cited By (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110880189A (en) * 2018-09-06 2020-03-13 舜宇光学(浙江)研究院有限公司 Combined calibration method and combined calibration device thereof and electronic equipment
CN110880189B (en) * 2018-09-06 2022-09-09 舜宇光学(浙江)研究院有限公司 Combined calibration method and combined calibration device thereof and electronic equipment
CN109443355B (en) * 2018-12-25 2020-10-27 中北大学 Visual-inertial tight coupling combined navigation method based on self-adaptive Gaussian PF
CN109443353A (en) * 2018-12-25 2019-03-08 中北大学 Vision based on fuzzy self-adaption ICKF-inertia close coupling Combinated navigation method
CN109443353B (en) * 2018-12-25 2020-11-06 中北大学 Visual-inertial tight coupling combined navigation method based on fuzzy self-adaptive ICKF
CN109443355A (en) * 2018-12-25 2019-03-08 中北大学 Vision based on adaptive Gauss PF-inertia close coupling Combinated navigation method
CN109724598A (en) * 2019-03-08 2019-05-07 哈尔滨工程大学 A kind of estimation and compensation method of the time delay error of GNSS/INS pine combination
CN110068325A (en) * 2019-04-11 2019-07-30 同济大学 A kind of lever arm error compensating method of vehicle-mounted INS/ visual combination navigation system
CN110375739A (en) * 2019-06-26 2019-10-25 中国科学院深圳先进技术研究院 A kind of mobile terminal vision fusion and positioning method, system and electronic equipment
CN110702139A (en) * 2019-09-29 2020-01-17 百度在线网络技术(北京)有限公司 Time delay calibration method and device, electronic equipment and medium
CN110702139B (en) * 2019-09-29 2021-08-27 百度在线网络技术(北京)有限公司 Time delay calibration method and device, electronic equipment and medium
CN111044069A (en) * 2019-12-16 2020-04-21 驭势科技(北京)有限公司 Vehicle positioning method, vehicle-mounted equipment and storage medium
CN111044069B (en) * 2019-12-16 2022-04-29 驭势科技(北京)有限公司 Vehicle positioning method, vehicle-mounted equipment and storage medium
CN111551191A (en) * 2020-04-28 2020-08-18 浙江商汤科技开发有限公司 Sensor external parameter calibration method and device, electronic equipment and storage medium
CN111551191B (en) * 2020-04-28 2022-08-09 浙江商汤科技开发有限公司 Sensor external parameter calibration method and device, electronic equipment and storage medium
CN114136315A (en) * 2021-11-30 2022-03-04 山东天星北斗信息科技有限公司 Monocular vision-based auxiliary inertial integrated navigation method and system
CN114136315B (en) * 2021-11-30 2024-04-16 山东天星北斗信息科技有限公司 Monocular vision-based auxiliary inertial integrated navigation method and system
CN116380148A (en) * 2023-04-06 2023-07-04 中国人民解放军93209部队 Two-stage space-time error calibration method and device for multi-sensor target tracking system
CN116380148B (en) * 2023-04-06 2023-11-10 中国人民解放军93209部队 Two-stage space-time error calibration method and device for multi-sensor target tracking system

Similar Documents

Publication Publication Date Title
CN108253964A (en) A kind of vision based on Time-Delay Filter/inertia combined navigation model building method
CN106225784B (en) Based on inexpensive Multi-sensor Fusion pedestrian dead reckoning method
WO2020087846A1 (en) Navigation method based on iteratively extended kalman filter fusion inertia and monocular vision
Schall et al. Global pose estimation using multi-sensor fusion for outdoor augmented reality
Sasani et al. Improving MEMS-IMU/GPS integrated systems for land vehicle navigation applications
CN110345937A (en) Appearance localization method and system are determined in a kind of navigation based on two dimensional code
CN106989744A (en) A kind of rotor wing unmanned aerial vehicle autonomic positioning method for merging onboard multi-sensor
KR101444685B1 (en) Method and Apparatus for Determining Position and Attitude of Vehicle by Image based Multi-sensor Data
CN110702143B (en) Rapid initial alignment method for SINS strapdown inertial navigation system moving base based on lie group description
TWI556198B (en) Positioning and directing data analysis system and method thereof
CN110361010A (en) It is a kind of based on occupy grating map and combine imu method for positioning mobile robot
CN104937377B (en) Method and apparatus for the vertical orientation for handling unconfined portable navigating device
CN110325822B (en) Cradle head pose correction method and cradle head pose correction device
CN108680942A (en) A kind of inertia/multiple antennas GNSS Combinated navigation methods and device
CN113432609B (en) Flexible attachment state collaborative estimation method
CN107702712A (en) Indoor pedestrian's combined positioning method based on inertia measurement bilayer WLAN fingerprint bases
CN109506660A (en) A kind of posture optimization calculation method for bionic navigation
CN109520476A (en) Resection dynamic pose measurement system and method based on Inertial Measurement Unit
CN114136315B (en) Monocular vision-based auxiliary inertial integrated navigation method and system
CN107270896A (en) A kind of pedestrian's positioning and trace tracking method and system
CN113551665A (en) High dynamic motion state sensing system and sensing method for motion carrier
CN110763238A (en) High-precision indoor three-dimensional positioning method based on UWB (ultra wide band), optical flow and inertial navigation
CN110986997A (en) Method and system for improving indoor inertial navigation precision
CN110986941B (en) Method for estimating installation angle of mobile phone
CN106643713B (en) Estimation method and device for zero-speed correction pedestrian locus for smooth and adaptive adjustment of threshold

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20180706

RJ01 Rejection of invention patent application after publication