CN112799413A - Target guidance approach control method and system for underwater autonomous vehicle - Google Patents
Target guidance approach control method and system for underwater autonomous vehicle Download PDFInfo
- Publication number
- CN112799413A CN112799413A CN202011539712.2A CN202011539712A CN112799413A CN 112799413 A CN112799413 A CN 112799413A CN 202011539712 A CN202011539712 A CN 202011539712A CN 112799413 A CN112799413 A CN 112799413A
- Authority
- CN
- China
- Prior art keywords
- underwater
- target
- autonomous vehicle
- sonar
- position information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013459 approach Methods 0.000 title claims abstract description 20
- 238000000034 method Methods 0.000 title claims abstract description 17
- 238000012545 processing Methods 0.000 claims abstract description 17
- 238000013527 convolutional neural network Methods 0.000 claims description 24
- 238000012549 training Methods 0.000 claims description 15
- 230000006870 function Effects 0.000 abstract description 4
- 238000004891 communication Methods 0.000 description 3
- 238000001514 detection method Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/04—Control of altitude or depth
- G05D1/06—Rate of change of altitude or depth
- G05D1/0692—Rate of change of altitude or depth specially adapted for under-water vehicles
Landscapes
- Engineering & Computer Science (AREA)
- Aviation & Aerospace Engineering (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
- Measurement Of Velocity Or Position Using Acoustic Or Ultrasonic Waves (AREA)
Abstract
The invention discloses an underwater autonomous vehicle target guiding and approaching control method and system, relates to the technical field of underwater autonomous vehicle control, and can improve the online identification capability of a vehicle, perform task re-planning after a suspected target is found, and ensure the functions of underwater autonomous path planning and target guiding and approaching of the underwater vehicle. The technical scheme of the invention is used for realizing the recognition, processing and guidance approaching of the underwater autonomous vehicle to the underwater target, and comprises the following steps: the underwater autonomous vehicle navigates autonomously underwater according to a pre-planned route. The forward-looking sonar detects the navigation area of the underwater autonomous vehicle to obtain a sonar image. The image processor carries out on-line processing on the sonar image and outputs the position information of the underwater target to the control unit in real time. And the control unit performs task re-planning according to the position information of the underwater target, and the underwater autonomous vehicle navigates according to the route obtained by re-planning and guides the underwater autonomous vehicle to approach the target.
Description
Technical Field
The invention relates to the technical field of control of an underwater autonomous vehicle, in particular to a target guidance approach control method and a target guidance approach control system of the underwater autonomous vehicle.
Background
When the underwater autonomous vehicle executes tasks underwater, the communication with an onshore console is usually realized through underwater acoustic communication, the underwater acoustic communication data volume is small, image information cannot be transmitted, most of autonomous vehicles do not have an online identification function, data are generally collected and stored through a sonar, after the vehicles are recovered, a suspected target is confirmed through manual intervention, and then the underwater autonomous vehicle executes corresponding tasks again, so that the operation is complex, and the time is long.
Therefore, the on-line identification capability of the underwater vehicle needs to be improved, task re-planning is carried out after a suspected target is found, the underwater vehicle autonomously approaches the target, and the intelligent level of the underwater vehicle is further improved.
Disclosure of Invention
In view of the above, the invention provides a target guidance and approach control method and system for an underwater autonomous vehicle, which can improve the online identification capability of the vehicle, perform task re-planning after finding a suspected target, and ensure the underwater autonomous path planning and target guidance and approach functions of the underwater vehicle.
In order to achieve the purpose, the technical scheme of the invention is used for realizing the recognition, processing and guidance approaching of an underwater target by an underwater autonomous vehicle, and comprises the following steps:
step 1, the underwater autonomous vehicle autonomously navigates underwater according to a pre-planned route.
And 2, detecting the navigation area of the underwater autonomous vehicle by the forward-looking sonar to obtain a sonar image.
And 3, carrying out online processing on the sonar image by the image processor, and outputting the position information of the underwater target to the control unit in real time.
And 4, performing task re-planning by the control unit according to the position information of the underwater target, and guiding the underwater autonomous vehicle to approach the target according to the route navigation obtained by re-planning.
Further, in step 1, the underwater autonomous vehicle autonomously navigates underwater according to a pre-planned route, specifically:
the propulsion device of the underwater autonomous vehicle is used for providing forward power for the underwater autonomous vehicle, and the steering engine is used for providing steering and depth control capabilities for the underwater autonomous vehicle.
And planning a plurality of path points in advance to form an airway, and autonomously navigating the underwater autonomous vehicle among the path points along the planned airway.
Further, in step 3, the image processor performs online processing on the sonar image, and outputs the position information of the underwater target to the control unit in real time, specifically:
and step 301, inputting sonar images collected by the forward-looking sonar into an image processor.
And step 302, the image processor performs target identification on the sonar image according to a preset identification model, and if an underwater target exists in the sonar image, the relative distance and the included angle between the sonar and the underwater target are obtained.
And step 303, the control unit obtains the position information and the attitude information of the underwater autonomous vehicle from inertial navigation of the underwater autonomous vehicle and sends the position information and the attitude information to the image processor.
And step 304, the image processor calculates the position information of the underwater target according to the position information and the attitude information of the underwater autonomous vehicle and the relative distance and the included angle between the sonar and the target, and sends the position information to the control unit.
Further, the image processor performs target recognition on the sonar image according to a preset recognition model, specifically:
and step 3021, constructing a convolutional neural network model, wherein the convolutional neural network model takes the sonar image as input, and takes whether the sonar image has an underwater target or not as output.
And step 3022, acquiring sonar images containing the underwater target as a training positive sample, and learning and training the constructed convolutional neural network model by using the sonar images not containing the underwater target as a training negative sample to obtain the trained convolutional neural network model.
And step 3023, collecting real-time sonar images by the forward-looking sonar, inputting the real-time sonar images into the trained convolutional neural network model, judging the false alarm rate of the trained convolutional neural network model, if the false alarm rate is higher than a set false alarm rate threshold, increasing the number of training negative samples, re-training the convolutional neural network model, and obtaining the trained convolutional neural network model again.
Further, step 4, the control unit performs task re-planning according to the position information of the underwater target, and the underwater autonomous vehicle navigates to the target to guide approach according to the route obtained by the re-planning, specifically:
step 401, the control unit obtains position information of underwater targets at a plurality of continuous moments, classifies the position information, judges whether the position information of the underwater targets obtained at each moment belongs to the same underwater target, merges the position information of the underwater targets belonging to the same underwater target, and performs smoothing processing on the position information of the underwater targets belonging to the same underwater target to finally obtain the position information of the underwater targets;
step 402, carrying out path re-planning according to the position information of the fused underwater targets, wherein the planned path passes through the positions of all the underwater targets;
and step 403, the underwater autonomous vehicle performs approaching observation on all underwater targets according to the re-planned route and the position information of the underwater targets.
The invention further provides an underwater autonomous vehicle target guiding and approaching control system which comprises a forward-looking sonar arranged on an underwater autonomous vehicle, an image processor and a control unit.
The underwater autonomous vehicle navigates autonomously underwater according to a pre-planned route.
And the forward-looking sonar is used for detecting the navigation area of the underwater autonomous vehicle to obtain a sonar image.
And the image processor is used for carrying out online processing on the sonar image and outputting the position information of the underwater target to the control unit in real time.
And the control unit is used for carrying out task re-planning according to the position information of the underwater target, and the underwater autonomous vehicle navigates to the target to guide approach according to the route obtained by the re-planning.
Has the advantages that:
according to the invention, a forward-looking sonar and an image processor are carried on the aircraft, wherein the forward-looking sonar is used for acquiring target detection information during the underwater navigation process of the aircraft. The image processor is used for analyzing the image and identifying the target and sending target information to the control unit. And the control unit performs data fusion on the received position information and replans the path according to the fused path points so as to realize target guidance approach control. The invention is suitable for the on-line identification and guidance approach of underwater targets, image processing and control systems consisting of forward-looking sonar, a control unit and an image processor are constructed, image data collected by the forward-looking sonar are stored, the data are processed in the processor according to a trained model, target information is output, and a central control unit carries out path re-planning according to fused information, so that the guidance approach control of the underwater targets is realized, manual intervention is not needed, the manual dependence degree of an aircraft can be greatly reduced, the labor cost is reduced, the intelligent level of the aircraft is improved, and the method has great popularization value.
Drawings
FIG. 1 is a component diagram of an underwater vehicle image processing system;
FIG. 2 is an underwater vehicle image processing system architecture layout.
Detailed Description
The invention is described in detail below by way of example with reference to the accompanying drawings.
The embodiment of the invention provides an underwater autonomous vehicle target guiding and approaching control method, which is used for realizing the recognition, processing and guiding and approaching of an underwater target by an underwater autonomous vehicle, and is characterized by comprising the following steps:
step 1, an underwater autonomous vehicle autonomously navigates underwater according to a pre-planned route; the method specifically comprises the following steps:
the underwater autonomous vehicle is provided with advancing power by using a propelling device such as a motor and the like, and steering and depth control capabilities are provided for the underwater autonomous vehicle by using a steering engine and the like; and planning a plurality of path points in advance to form an airway, and autonomously navigating the underwater autonomous vehicle among the path points along the planned airway.
Step 2, detecting a navigation area of the underwater autonomous vehicle by using a forward-looking sonar to obtain a sonar image; in the embodiment of the invention, a proper forward looking sonar is selected to collect data in real time. Selecting proper sonar according to the space position and the loading capacity of the platform, selecting proper measuring range, horizontal opening angle and vertical opening angle according to the detection distance and the size of the target, and selecting proper installation included angle according to the position information of the target.
And powering on or off the sonar equipment within a proper water depth range, and issuing a sonar working instruction according to an area to be detected so as to ensure the image acquisition of an effective area.
Step 3, the image processor carries out on-line processing on the sonar image and outputs the position information of the underwater target to the control unit in real time; the method specifically comprises the following steps:
301, inputting sonar images collected by a forward-looking sonar into an image processor;
step 302, the image processor identifies the target of the sonar image according to a preset identification model, and if an underwater target exists in the sonar image, the relative distance and the included angle between the sonar and the underwater target are obtained; the method specifically comprises the following steps:
and step 3021, constructing a convolutional neural network model, wherein the convolutional neural network model takes the sonar image as input, and takes whether the sonar image has an underwater target or not as output.
And step 3022, acquiring sonar images containing the underwater target as a training positive sample, and learning and training the constructed convolutional neural network model by using the sonar images not containing the underwater target as a training negative sample to obtain the trained convolutional neural network model.
And step 3023, collecting real-time sonar images by the forward-looking sonar, inputting the real-time sonar images into the trained convolutional neural network model, judging the false alarm rate of the trained convolutional neural network model, if the false alarm rate is higher than a set false alarm rate threshold (the threshold can be set according to experience, and the false alarm rate can be guaranteed to be lower than a certain value), increasing the number of training negative samples, re-training the convolutional neural network model, and obtaining the trained convolutional neural network model again.
And step 303, the control unit obtains the position information and the attitude information of the underwater autonomous vehicle from inertial navigation of the underwater autonomous vehicle and sends the position information and the attitude information to the image processor.
And step 304, the image processor calculates the position information of the underwater target according to the position information and the attitude information of the underwater autonomous vehicle and the relative distance and the included angle between the sonar and the target, and sends the position information to the control unit.
And 4, performing task re-planning by the control unit according to the position information of the underwater target, and guiding the underwater autonomous vehicle to approach the target according to the route navigation obtained by re-planning. The method specifically comprises the following steps:
step 401, the control unit obtains position information of the underwater targets at a plurality of continuous moments, classifies the position information, judges whether the position information of the underwater targets obtained at each moment belongs to the same underwater target, merges the position information of the underwater targets belonging to the same underwater target, and smoothes the position information of the underwater targets belonging to the same underwater target, so as to obtain the position information of the underwater targets.
Step 402, performing path re-planning according to the fused position information of the underwater targets, and re-planning the air route, wherein the planned path passes through the positions of all the underwater targets.
And step 403, the underwater autonomous vehicle performs approaching observation on all underwater targets according to the re-planned route and the position information of the underwater targets.
Another embodiment of the present invention, as shown in fig. 1, further provides an underwater autonomous vehicle target guidance approach control system, comprising a forward looking sonar, an image processor and a control unit disposed on an underwater autonomous vehicle, as shown in fig. 2.
The underwater autonomous vehicle navigates autonomously underwater according to a pre-planned route.
And the forward-looking sonar is used for detecting the navigation area of the underwater autonomous vehicle to obtain a sonar image.
And the image processor is used for carrying out online processing on the sonar image and outputting the position information of the underwater target to the control unit in real time.
And the control unit is used for carrying out task re-planning according to the position information of the underwater target, and the underwater autonomous vehicle navigates to the target to guide approach according to the route obtained by the re-planning.
Wherein the functions in the image processor and control unit can be arranged according to the relevant description in an autonomous underwater vehicle target guidance proximity control of the previous embodiment.
In summary, the above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (6)
1. An underwater autonomous vehicle target guiding and approaching control method is used for realizing the recognition, processing and guiding and approaching of an underwater target by an underwater autonomous vehicle, and is characterized by comprising the following steps:
step 1, an underwater autonomous vehicle autonomously navigates underwater according to a pre-planned route;
step 2, detecting a navigation area of the underwater autonomous vehicle by using a forward-looking sonar to obtain a sonar image;
step 3, the image processor carries out online processing on the sonar image and outputs the position information of the underwater target to the control unit in real time;
and 4, performing task re-planning by the control unit according to the position information of the underwater target, and guiding the underwater autonomous vehicle to approach the target according to the route navigation obtained by re-planning.
2. The method according to claim 1, wherein in step 1, the underwater autonomous vehicle autonomously navigates underwater according to a pre-planned route, specifically:
a propulsion device of the underwater autonomous vehicle is used for providing forward power for the underwater autonomous vehicle, and a steering engine is used for providing steering and depth control capabilities for the underwater autonomous vehicle;
and planning a plurality of path points in advance to form an airway, and the underwater autonomous vehicle autonomously navigates among the path points along the planned airway.
3. The method of claim 1, wherein in step 3, the image processor performs online processing on the sonar image and outputs the position information of the underwater target to the control unit in real time, specifically:
301, inputting sonar images collected by a forward-looking sonar into the image processor;
step 302, the image processor performs target identification on the sonar image according to a preset identification model, and if an underwater target exists in the sonar image, the relative distance and the included angle between the sonar and the underwater target are obtained;
303, obtaining position information and attitude information of the underwater autonomous vehicle from inertial navigation of the underwater autonomous vehicle by a control unit, and sending the position information and the attitude information to the image processor;
and step 304, the image processor calculates the position information of the underwater target according to the position information and the attitude information of the underwater autonomous vehicle and the relative distance and the included angle between the sonar and the target, and sends the position information to the control unit.
4. The method according to claim 3, wherein the image processor performs target recognition on the sonar image according to a preset recognition model, specifically:
step 3021, constructing a convolutional neural network model, wherein the convolutional neural network model takes a sonar image as input and takes whether the sonar image has an underwater target as output;
step 3022, acquiring sonar images containing underwater targets as training positive samples, acquiring sonar images not containing underwater targets as training negative samples, and learning and training the constructed convolutional neural network model by using the training samples to obtain a trained convolutional neural network model;
and step 3023, collecting real-time sonar images by using a forward-looking sonar, inputting the real-time sonar images into the trained convolutional neural network model, judging the false alarm rate of the trained convolutional neural network model, if the false alarm rate is higher than a set false alarm rate threshold, increasing the number of training negative samples, retraining the convolutional neural network model, and obtaining the trained convolutional neural network model again.
5. The method according to claim 1, wherein in step 4, the control unit performs task re-planning according to the position information of the underwater target, and the underwater autonomous vehicle guides approach to the target according to the route navigation obtained by the re-planning, specifically:
step 401, the control unit obtains position information of underwater targets at a plurality of continuous moments, classifies the position information, judges whether the position information of the underwater targets obtained at each moment belongs to the same underwater target, merges the position information belonging to the same underwater target, and smoothes the position information belonging to the same underwater target to finally obtain the position information of the underwater targets;
step 402, carrying out path re-planning according to the position information of the fused underwater targets, wherein the planned path passes through the positions of all the underwater targets;
and step 403, the underwater autonomous vehicle performs approaching observation on all underwater targets according to the re-planned route and the position information of the underwater targets.
6. An underwater autonomous vehicle target guiding and approaching control system is characterized by comprising a forward-looking sonar, an image processor and a control unit, wherein the forward-looking sonar is arranged on the underwater autonomous vehicle;
the underwater autonomous vehicle navigates autonomously underwater according to a pre-planned route;
the forward-looking sonar is used for detecting a navigation area of the underwater autonomous vehicle to obtain a sonar image;
the image processor is used for processing the sonar image on line and outputting the position information of the underwater target to the control unit in real time;
and the control unit is used for performing task re-planning according to the position information of the underwater target, and the underwater autonomous vehicle navigates to the target to guide approach according to the route obtained by re-planning.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011539712.2A CN112799413A (en) | 2020-12-23 | 2020-12-23 | Target guidance approach control method and system for underwater autonomous vehicle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011539712.2A CN112799413A (en) | 2020-12-23 | 2020-12-23 | Target guidance approach control method and system for underwater autonomous vehicle |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112799413A true CN112799413A (en) | 2021-05-14 |
Family
ID=75804103
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011539712.2A Pending CN112799413A (en) | 2020-12-23 | 2020-12-23 | Target guidance approach control method and system for underwater autonomous vehicle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112799413A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113815819A (en) * | 2021-05-21 | 2021-12-21 | 海南大学 | Multifunctional modularized deep sea environment monitoring type unmanned underwater vehicle |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106094819A (en) * | 2016-06-17 | 2016-11-09 | 江苏科技大学 | Underwater robot control system and course heading control method based on sonar image target recognition |
| CN108846323A (en) * | 2018-05-28 | 2018-11-20 | 哈尔滨工程大学 | A kind of convolutional neural networks optimization method towards Underwater Targets Recognition |
| CN110135239A (en) * | 2019-03-26 | 2019-08-16 | 北京理工雷科电子信息技术有限公司 | A kind of recognition methods of optical remote sensing image harbour Ship Target |
| CN110162042A (en) * | 2019-05-14 | 2019-08-23 | 中国舰船研究设计中心 | A kind of vision guide device and method for unmanned boat recycling |
-
2020
- 2020-12-23 CN CN202011539712.2A patent/CN112799413A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106094819A (en) * | 2016-06-17 | 2016-11-09 | 江苏科技大学 | Underwater robot control system and course heading control method based on sonar image target recognition |
| CN108846323A (en) * | 2018-05-28 | 2018-11-20 | 哈尔滨工程大学 | A kind of convolutional neural networks optimization method towards Underwater Targets Recognition |
| CN110135239A (en) * | 2019-03-26 | 2019-08-16 | 北京理工雷科电子信息技术有限公司 | A kind of recognition methods of optical remote sensing image harbour Ship Target |
| CN110162042A (en) * | 2019-05-14 | 2019-08-23 | 中国舰船研究设计中心 | A kind of vision guide device and method for unmanned boat recycling |
Non-Patent Citations (2)
| Title |
|---|
| 代长安: "基于强化学习的AUV任务规划技术研究", 《中国优秀硕士学位论文全文数据库 工程科技Ⅱ辑》 * |
| 朱玲羚: "基于前视声纳的水下无人航行器自导技术研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113815819A (en) * | 2021-05-21 | 2021-12-21 | 海南大学 | Multifunctional modularized deep sea environment monitoring type unmanned underwater vehicle |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102057532B1 (en) | Device for sharing and learning driving environment data for improving the intelligence judgments of autonomous vehicle and method thereof | |
| CN112650237B (en) | Ship path planning method and device based on clustering processing and artificial potential field | |
| CN108349492B (en) | For providing the method and system of the lane information in lane | |
| CN105629974A (en) | Robot path planning method and system based on improved artificial potential field method | |
| CN108593213A (en) | The inspection of unmanned plane sub-terrain mines and odoriferous homing device | |
| CN113708491A (en) | Intelligent inspection system for traction substation | |
| CN114489112A (en) | Cooperative sensing system and method for intelligent vehicle-unmanned aerial vehicle | |
| CN112799413A (en) | Target guidance approach control method and system for underwater autonomous vehicle | |
| CN113359796B (en) | Unmanned aerial vehicle searching method for underground multi-branch cave | |
| CN113125795A (en) | Obstacle speed detection method, device, equipment and storage medium | |
| US20220101723A1 (en) | Annotation and Mapping for Vehicle Operation in Low-Confidence Object Detection Conditions | |
| CN115657683B (en) | Unmanned cable-free submersible real-time obstacle avoidance method capable of being used for inspection operation task | |
| CN115373383A (en) | Autonomous obstacle avoidance method and device for garbage recovery unmanned boat and related equipment | |
| CN113885533B (en) | Unmanned driving method and system of unmanned boat | |
| CN113252861B (en) | Water quality detection method and device, electronic equipment and storage medium | |
| CN206862369U (en) | A kind of robot zone navigation system based on BIM | |
| CN113298044B (en) | Obstacle detection method, system, device and storage medium based on positioning compensation | |
| Acosta et al. | Some issues on the design of a low-cost autonomous underwater vehicle with an intelligent dynamic mission planner for pipeline and cable tracking | |
| CN110658827B (en) | Transport vehicle automatic guiding system and method based on Internet of things | |
| CN109901589B (en) | Mobile robot control method and device | |
| Alagić et al. | Design of mobile robot motion framework based on modified vector field histogram | |
| Murarka et al. | Vision-based frozen surface egress: A docking algorithm for the ENDURANCE AUV | |
| RU2735163C1 (en) | Method of preliminary ship routing generation | |
| Li et al. | LSDA-APF: A Local Obstacle Avoidance Algorithm for Unmanned Surface Vehicles Based on 5G Communication Environment. | |
| CN220743309U (en) | Water area patrol equipment and water area patrol system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210514 |