CN115880953B - Unmanned aerial vehicle management and control method and intelligent street lamp system - Google Patents
Unmanned aerial vehicle management and control method and intelligent street lamp system Download PDFInfo
- Publication number
- CN115880953B CN115880953B CN202310217332.4A CN202310217332A CN115880953B CN 115880953 B CN115880953 B CN 115880953B CN 202310217332 A CN202310217332 A CN 202310217332A CN 115880953 B CN115880953 B CN 115880953B
- Authority
- CN
- China
- Prior art keywords
- flight
- target
- street lamp
- coordinate
- unmanned aerial
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02B—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO BUILDINGS, e.g. HOUSING, HOUSE APPLIANCES OR RELATED END-USER APPLICATIONS
- Y02B20/00—Energy efficient lighting technologies, e.g. halogen lamps or gas discharge lamps
- Y02B20/40—Control techniques providing energy savings, e.g. smart controller or presence detection
Landscapes
- Traffic Control Systems (AREA)
Abstract
The invention relates to the technical field of the Internet of things, and discloses a control method of an unmanned aerial vehicle and an intelligent street lamp system, which are used for realizing position offset correction of an intelligent street lamp on the unmanned aerial vehicle flight process and improving the control accuracy of the unmanned aerial vehicle flight process. The method comprises the following steps: constructing a flight route coordinate set according to a preset flight route, and performing street lamp coordinate mapping on a plurality of street lamp identification information to obtain a street lamp coordinate set; collecting a plurality of groups of ground point cloud images of real-time position coordinates and identifying target identification information; performing course offset detection on the real-time position coordinates according to the flight route coordinate set to obtain a course offset detection result, and performing altitude error detection on the real-time position coordinates according to the target identification information to obtain an altitude error detection result; and generating a flight correction data set according to the course deviation detection result and the altitude error detection result, and correcting the course and the altitude of the target unmanned aerial vehicle according to the flight correction data set to generate a real-time flight data monitoring chart.
Description
Technical Field
The invention relates to the technical field of the Internet of things, in particular to a management and control method of an unmanned aerial vehicle and an intelligent street lamp system.
Background
With the development and popularization of the unmanned aerial vehicle industry, more and more unmanned aerial vehicle products have begun to walk into people's lives. When more unmanned aerial vehicles walk into the life of people, the flight safety and the control accuracy are particularly important.
At present, the unmanned aerial vehicle usually detects the altitude and the course by the manual work in the flight process, and then realizes unmanned aerial vehicle flight attitude's adjustment, and current scheme receives artificial experience's influence great, leads to the control rate of accuracy of current scheme low.
Disclosure of Invention
The invention provides a control method of an unmanned aerial vehicle and an intelligent street lamp system, which are used for realizing position deviation correction of an intelligent street lamp on the unmanned aerial vehicle in the flight process and improving the control accuracy of the unmanned aerial vehicle in the flight process.
The first aspect of the invention provides a control method of an unmanned aerial vehicle, which comprises the following steps:
acquiring a preset flight route of a target unmanned aerial vehicle in a target flight area, establishing network connection between the target unmanned aerial vehicle and a preset intelligent street lamp system, and inquiring a plurality of street lamp identification information preset in the target flight area;
constructing a corresponding flight route coordinate set according to the preset flight route, and mapping street lamp coordinates of the plurality of street lamp identification information to obtain a street lamp coordinate set;
Detecting the current flight track of the target unmanned aerial vehicle and returning real-time position coordinates, and calling an image acquisition terminal in the target unmanned aerial vehicle to acquire a plurality of groups of ground point cloud images of the real-time position coordinates;
identifying street lamp identification information on the plurality of groups of ground point cloud images to obtain target identification information corresponding to the real-time position coordinates;
performing course offset detection on the real-time position coordinates according to the flight route coordinate set to obtain a course offset detection result, and performing altitude error detection on the real-time position coordinates according to the target identification information to obtain an altitude error detection result;
and generating a flight correction data set of the target unmanned aerial vehicle according to the course deviation detection result and the altitude error detection result, and carrying out course and altitude correction on the target unmanned aerial vehicle according to the flight correction data set to generate a real-time flight data monitoring chart.
With reference to the first aspect, in a first implementation manner of the first aspect of the present invention, the obtaining a preset flight route of a target unmanned aerial vehicle in a target flight area, establishing a network connection between the target unmanned aerial vehicle and a preset intelligent street lamp system, and querying a plurality of street lamp identification information preset in the target flight area includes:
Receiving a flight task sent by a control terminal, and carrying out task analysis on the flight task to obtain a target flight area;
acquiring a preset flight route of a target unmanned aerial vehicle in a target flight area;
establishing network connection between the target unmanned aerial vehicle and a preset intelligent street lamp system, and acquiring a network connection state;
and inquiring a plurality of street lamp identification information preset in the target flight area according to the network connection state.
With reference to the first aspect, in a second implementation manner of the first aspect of the present invention, the constructing a corresponding flight route coordinate set according to the preset flight route, and performing street lamp coordinate mapping on the plurality of street lamp identification information to obtain a street lamp coordinate set includes:
selecting a coordinate origin according to the target flight area, and constructing a target coordinate system according to the coordinate origin;
according to the target coordinate system, carrying out coordinate transformation on the preset flight route to obtain a flight route coordinate set;
and carrying out street lamp coordinate mapping on the plurality of street lamp identification information based on the target coordinate system to obtain a street lamp coordinate set.
With reference to the first aspect, in a third implementation manner of the first aspect of the present invention, the detecting a current flight trajectory of the target unmanned aerial vehicle and returning a real-time position coordinate, invoking an image acquisition terminal in the target unmanned aerial vehicle to acquire a plurality of groups of ground point cloud images of the real-time position coordinate includes:
Controlling the target unmanned aerial vehicle to execute the flight task and generating a current flight track of the target unmanned aerial vehicle;
acquiring real-time position coordinates of the target unmanned aerial vehicle based on the target coordinate system, and transmitting the real-time position coordinates back to the control terminal;
and calling an image acquisition terminal in the target unmanned aerial vehicle based on a preset image acquisition range, and acquiring images of real-time position coordinates of the target unmanned aerial vehicle to obtain a plurality of groups of ground point cloud images corresponding to the real-time position coordinates.
With reference to the first aspect, in a fourth implementation manner of the first aspect of the present invention, the identifying street lamp identification information on the multiple sets of ground point cloud images to obtain target identification information corresponding to the real-time position coordinates includes:
image region segmentation is carried out on each group of ground point cloud images respectively, so that a plurality of region images corresponding to each group of ground point cloud images are obtained;
performing binarization processing on a plurality of area images corresponding to each group of ground point cloud images to obtain a binarized image of each area image;
and respectively carrying out street lamp identification information identification on the binarized image of each area image to obtain target identification information corresponding to the real-time position coordinates.
With reference to the first aspect, in a fifth implementation manner of the first aspect of the present invention, the performing, according to the set of flight route coordinates, heading offset detection on the real-time position coordinates to obtain a heading offset detection result, and performing, according to the target identification information, altitude error detection on the real-time position coordinates to obtain an altitude error detection result, includes:
calculating distance data between the real-time position coordinates and each coordinate in the flight path coordinate set;
according to the distance data, calculating a heading angle between the real-time position coordinate and each coordinate in the flight route coordinate set;
generating a heading offset detection result corresponding to the real-time position coordinate according to the heading angle;
determining at least three associated coordinates in the street lamp coordinate set according to the target identification information, and calculating average height difference data between the real-time position coordinates and the at least three associated coordinates in the street lamp coordinate set;
and carrying out height error check on the real-time position coordinates according to the average height difference data to generate a corresponding height error detection result.
With reference to the first aspect, in a sixth implementation manner of the first aspect of the present invention, the generating a flight correction data set of the target unmanned aerial vehicle according to the heading offset detection result and the altitude error detection result, and performing heading and altitude correction on the target unmanned aerial vehicle according to the flight correction data set, generating a real-time flight data monitoring chart includes:
Calculating heading angle correction data of the target unmanned aerial vehicle according to a heading offset detection result, and calculating altitude correction data of the target unmanned aerial vehicle according to an altitude error detection result;
the course angle correction data and the altitude correction data are put into a flight correction data set, and course and altitude correction is carried out on the target unmanned aerial vehicle according to the flight correction data set;
and acquiring real-time flight data of the target unmanned aerial vehicle and generating a real-time flight data monitoring chart.
A second aspect of the present invention provides an intelligent street lamp system, comprising:
the system comprises an acquisition module, a control module and a control module, wherein the acquisition module is used for acquiring a preset flight route of a target unmanned aerial vehicle in a target flight area, establishing network connection between the target unmanned aerial vehicle and a preset intelligent street lamp system, and inquiring a plurality of street lamp identification information preset in the target flight area;
the mapping module is used for constructing a corresponding flight route coordinate set according to the preset flight route, and carrying out street lamp coordinate mapping on the plurality of street lamp identification information to obtain a street lamp coordinate set;
the detection module is used for detecting the current flight track of the target unmanned aerial vehicle and returning real-time position coordinates, and calling an image acquisition terminal in the target unmanned aerial vehicle to acquire a plurality of groups of ground point cloud images of the real-time position coordinates;
The identification module is used for identifying street lamp identification information of the plurality of groups of ground point cloud images to obtain target identification information corresponding to the real-time position coordinates;
the processing module is used for carrying out course deviation detection on the real-time position coordinates according to the flight route coordinate set to obtain a course deviation detection result, and carrying out altitude error detection on the real-time position coordinates according to the target identification information to obtain an altitude error detection result;
and the correction module is used for generating a flight correction data set of the target unmanned aerial vehicle according to the course deviation detection result and the altitude error detection result, correcting the course and the altitude of the target unmanned aerial vehicle according to the flight correction data set, and generating a real-time flight data monitoring chart.
In the technical scheme provided by the invention, a flight route coordinate set is constructed according to a preset flight route, and street lamp coordinate mapping is carried out on a plurality of street lamp identification information to obtain the street lamp coordinate set; collecting a plurality of groups of ground point cloud images of real-time position coordinates and identifying target identification information; performing course offset detection on the real-time position coordinates according to the flight route coordinate set to obtain a course offset detection result, and performing altitude error detection on the real-time position coordinates according to the target identification information to obtain an altitude error detection result; according to the method, the system and the device, the flight correction data set is generated according to the course deviation detection result and the altitude error detection result, the course and the altitude of the target unmanned aerial vehicle are corrected according to the flight correction data set, and the real-time flight data monitoring chart is generated.
Drawings
FIG. 1 is a schematic view of an embodiment of a control method of a unmanned aerial vehicle according to an embodiment of the present invention;
FIG. 2 is a flow chart of constructing a flight path coordinate set and a street lamp coordinate set in an embodiment of the invention;
FIG. 3 is a flow chart of acquiring multiple sets of ground point cloud images in an embodiment of the present invention;
FIG. 4 is a flowchart of the identification information recognition of the street lamp according to the embodiment of the present invention;
fig. 5 is a schematic diagram of an embodiment of the intelligent street lamp system according to the present invention.
Detailed Description
The embodiment of the invention provides a control method and an intelligent street lamp system of an unmanned aerial vehicle, which are used for realizing position offset correction of the intelligent street lamp on the unmanned aerial vehicle in the flight process and improving the control accuracy of the unmanned aerial vehicle in the flight process. The terms "first," "second," "third," "fourth" and the like in the description and in the claims and in the above drawings, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments described herein may be implemented in other sequences than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed or inherent to such process, method, article, or apparatus.
For easy understanding, a specific flow of an embodiment of the present invention is described below, referring to fig. 1, and an embodiment of a method for controlling a unmanned aerial vehicle in an embodiment of the present invention includes:
s101, acquiring a preset flight route of a target unmanned aerial vehicle in a target flight area, establishing network connection between the target unmanned aerial vehicle and a preset intelligent street lamp system, and inquiring a plurality of street lamp identification information preset in the target flight area;
it can be understood that the execution subject of the present invention may be an intelligent street lamp system, and may also be a terminal or a server, which is not limited herein. The embodiment of the invention is described by taking a server as an execution main body as an example.
Specifically, the server obtains a preset flight path of the target unmanned aerial vehicle in the target flight area, further, the preset flight path can indicate a plurality of working areas of a plurality of operations to be performed by the unmanned aerial vehicle during the flight of the unmanned aerial vehicle, inter-area information is used for determining a plurality of inter-area flight paths, the unmanned aerial vehicle is allowed to fly through the plurality of working areas along the plurality of inter-area flight paths, further, the server establishes network connection with the target unmanned aerial vehicle and a preset intelligent street lamp system, and queries a plurality of street lamp identification information preset in the target flight area, and it is required to explain that, when the network connection is established, the server obtains network configuration parameters of the intelligent street lamp system, the server sends the network configuration parameters to the target unmanned aerial vehicle through a first communication link established between the server and the target unmanned aerial vehicle, and the target unmanned aerial vehicle is used for establishing network connection with the intelligent street lamp system by adopting the network configuration parameters.
S102, constructing a corresponding flight route coordinate set according to a preset flight route, and mapping street lamp coordinates of a plurality of street lamp identification information to obtain the street lamp coordinate set;
specifically, the server builds a three-dimensional coordinate system in advance, meanwhile, builds a corresponding flight route coordinate set according to a preset flight route, and then the server performs street lamp coordinate mapping on the street lamp identification information, wherein when the server performs street lamp coordinate mapping on the street lamp identification information, the server performs coordinate value matching on the street lamp identification information through the three-dimensional coordinate system, corresponding coordinate value information is determined, and finally, the server performs coordinate mapping on the street lamp identification information according to the coordinate value information to obtain a street lamp coordinate set.
S103, detecting the current flight track of the target unmanned aerial vehicle, returning the real-time position coordinates, and calling an image acquisition terminal in the target unmanned aerial vehicle to acquire a plurality of groups of ground point cloud images of the real-time position coordinates;
specifically, the server controls the target unmanned aerial vehicle to execute a flight task, generates a current flight track of the target unmanned aerial vehicle, acquires real-time position coordinates of the target unmanned aerial vehicle based on a target coordinate system, transmits the real-time position coordinates back to the control terminal, invokes an image acquisition terminal in the target unmanned aerial vehicle based on a preset image acquisition range, and performs image acquisition on the real-time position coordinates of the target unmanned aerial vehicle to obtain a plurality of groups of ground point cloud images corresponding to the real-time position coordinates.
S104, identifying street lamp identification information of a plurality of groups of ground point cloud images to obtain target identification information corresponding to real-time position coordinates;
specifically, the server identifies street lamp identification information of a plurality of groups of ground point cloud images, wherein the server acquires identification information corresponding to at least one target street lamp according to the plurality of groups of ground point cloud images, further the server generates an identification instruction according to the identification information, and carries out street lamp identification information identification based on the identification information to obtain the target identification information corresponding to the real-time position coordinates.
S105, performing course offset detection on the real-time position coordinates according to the flight route coordinate set to obtain a course offset detection result, and performing altitude error detection on the real-time position coordinates according to the target identification information to obtain an altitude error detection result;
specifically, the server calculates distance data between the real-time position coordinates and each coordinate in the flight route coordinate set, the server calculates a heading angle between the real-time position coordinates and each coordinate in the flight route coordinate set according to the distance data, the server generates a heading offset detection result corresponding to the real-time position coordinates according to the heading angle, the server determines at least three associated coordinates in the street lamp coordinate set according to the target identification information, calculates average height difference data between the real-time position coordinates and the at least three associated coordinates in the street lamp coordinate set, and performs height error check on the real-time position coordinates according to the average height difference data to generate a corresponding height error detection result.
Performing heading angle classification on a target vehicle detection frame by using a preset heading angle prediction model to obtain a heading angle prediction result, wherein the heading angle prediction result specifically comprises a heading angle interval and a heading angle offset of a target vehicle; determining a course angle priori value of the target vehicle according to the detection result of the target vehicle; and determining the course angle of the target vehicle according to the course angle prediction result and the course angle priori value. According to the method and the device, the course angle is classified through the preset course angle prediction model, the course angle prediction range is reduced, the prediction error is reduced, finer adjustment is performed on the basis of the course angle prior value, and the prediction accuracy is improved.
S106, generating a flight correction data set of the target unmanned aerial vehicle according to the course deviation detection result and the altitude error detection result, and carrying out course and altitude correction on the target unmanned aerial vehicle according to the flight correction data set to generate a real-time flight data monitoring chart.
Specifically, the server generates a flight correction data set of the target unmanned aerial vehicle according to the course deviation detection result and the altitude error detection result, carries out course and altitude correction on the target unmanned aerial vehicle according to the flight correction data set, and generates a real-time flight data monitoring chart, wherein the server generates corresponding course angle correction data according to the course deviation detection result and the altitude error detection result, further generates a flight correction data set of the target unmanned aerial vehicle according to the course angle correction data, finally, carries out course and altitude correction on the mother unmanned aerial vehicle according to the flight correction data set, and finally, the server generates the flight data detection chart.
In the embodiment of the invention, a flight route coordinate set is constructed according to a preset flight route, and street lamp coordinate mapping is carried out on a plurality of street lamp identification information to obtain the street lamp coordinate set; collecting a plurality of groups of ground point cloud images of real-time position coordinates and identifying target identification information; performing course offset detection on the real-time position coordinates according to the flight route coordinate set to obtain a course offset detection result, and performing altitude error detection on the real-time position coordinates according to the target identification information to obtain an altitude error detection result; according to the method, the system and the device, the flight correction data set is generated according to the course deviation detection result and the altitude error detection result, the course and the altitude of the target unmanned aerial vehicle are corrected according to the flight correction data set, and the real-time flight data monitoring chart is generated.
In a specific embodiment, the process of executing step S101 may specifically include the following steps:
(1) Receiving a flight task sent by a control terminal, and carrying out task analysis on the flight task to obtain a target flight area;
(2) Acquiring a preset flight route of a target unmanned aerial vehicle in a target flight area;
(3) Establishing network connection between a target unmanned aerial vehicle and a preset intelligent street lamp system, and acquiring a network connection state;
(4) And inquiring a plurality of street lamp identification information preset in the target flight area according to the network connection state.
Specifically, the server receives a flight task sent by the control terminal, performs task analysis on the flight task to obtain a target flight area, performs task analysis on flight task and original data of target area parameters according to an XML format when performing task analysis on the flight task to obtain the target flight area, acquires a preset flight route of a target unmanned aerial vehicle in the target flight area, establishes network connection with a preset intelligent street lamp system, and acquires a network connection state, wherein when the server establishes the network connection, the server acquires network configuration parameters of the intelligent street lamp system, the server sends the network configuration parameters to the target unmanned aerial vehicle through a first communication link established between the server and the target unmanned aerial vehicle, the target unmanned aerial vehicle is used for establishing network connection with the intelligent street lamp system by adopting the network configuration parameters, and finally, the server inquires a plurality of street lamp identification information preset in the target flight area according to the network connection state.
In a specific embodiment, as shown in fig. 2, the process of executing step S102 may specifically include the following steps:
s201, selecting a coordinate origin according to a target flight area, and constructing a target coordinate system according to the coordinate origin;
s202, carrying out coordinate transformation on a preset flight route according to a target coordinate system to obtain a flight route coordinate set;
s203, based on the target coordinate system, performing street lamp coordinate mapping on the plurality of street lamp identification information to obtain a street lamp coordinate set.
Specifically, a server selects a coordinate origin according to a target flight area, constructs a target coordinate system according to the coordinate origin, performs coordinate conversion on a preset flight route according to the target coordinate system to obtain a flight route coordinate set, wherein the server acquires coordinate information of flight route points in the target coordinate system from a designated area, determines the coordinate information of the flight route in the target coordinate system based on a distance relation between the flight route and a plurality of flight route points in the target coordinate system, finally obtains the flight route coordinate set, and further performs street lamp coordinate mapping on a plurality of street lamp identification information based on the target coordinate system to obtain the street lamp coordinate set.
In a specific embodiment, as shown in fig. 3, the process of executing step S103 may specifically include the following steps:
s301, controlling the target unmanned aerial vehicle to execute a flight task and generating a current flight track of the target unmanned aerial vehicle;
s302, acquiring real-time position coordinates of the target unmanned aerial vehicle based on a target coordinate system, and transmitting the real-time position coordinates back to the control terminal;
s303, based on a preset image acquisition range, calling an image acquisition terminal in the target unmanned aerial vehicle, and performing image acquisition on real-time position coordinates of the target unmanned aerial vehicle to obtain a plurality of groups of ground point cloud images corresponding to the real-time position coordinates.
Specifically, the server controls the target unmanned aerial vehicle to execute a flight task, generates a current flight track of the target unmanned aerial vehicle, acquires real-time position coordinates of the target unmanned aerial vehicle based on a target coordinate system, transmits the real-time position coordinates back to the control terminal, invokes an image acquisition terminal in the target unmanned aerial vehicle based on a preset image acquisition range, and acquires a plurality of groups of ground point cloud images corresponding to the real-time position coordinates, wherein the server acquires at least two groups of 2D images of a detected object acquired by acquisition equipment under different preset brightness control parameters, creates code value states in a predefined image, acquires a marked image, performs point cloud reconstruction on a code value state marked area of the marked image, acquires a point cloud image corresponding to the marked image, and finally determines a plurality of groups of ground point cloud images corresponding to the real-time position coordinates.
In a specific embodiment, as shown in fig. 4, the process of executing step S104 may specifically include the following steps:
s401, respectively carrying out image region segmentation on each group of ground point cloud images to obtain a plurality of region images corresponding to each group of ground point cloud images;
s402, performing binarization processing on a plurality of area images corresponding to each group of ground point cloud images to obtain a binarized image of each area image;
s403, respectively identifying street lamp identification information of the binarized image of each area image to obtain target identification information corresponding to the real-time position coordinates.
Specifically, the server performs image region segmentation on each group of ground point cloud images to obtain a plurality of region images corresponding to each group of ground point cloud images, wherein the server scans each group of ground point cloud images to establish an adjacent relation between segmentation blocks, an initial bottom small-scale region structure is generated, then features such as gray scale, texture and shape are added successively on the basis to perform merging adjustment to obtain a plurality of region images corresponding to each group of ground point cloud images, and further, the server performs binarization processing on the plurality of region images corresponding to each group of ground point cloud images to obtain a binarized image of each region image, wherein the server performs binarization processing on each group of obtained ground point cloud images by using at least two binarization algorithms to obtain a corresponding binarized image and confidence of each pixel point in the binarized image; determining the weight of each pixel point in the obtained binarized image based on the confidence coefficient of the pixel point; and carrying out fusion processing on the obtained binarized images based on the weights of the obtained pixel points to obtain binarized images of each area image, and finally, respectively carrying out street lamp identification information identification on the binarized images of each area image by a server to obtain target identification information corresponding to real-time position coordinates.
In a specific embodiment, the process of executing step S105 may specifically include the following steps:
(1) Calculating distance data between the real-time position coordinates and each coordinate in the flight route coordinate set;
(2) According to the distance data, calculating a heading angle between the real-time position coordinate and each coordinate in the flight route coordinate set;
(3) Generating a heading offset detection result corresponding to the real-time position coordinate according to the heading angle;
(4) Determining at least three associated coordinates in the street lamp coordinate set according to the target identification information, and calculating average height difference data between the real-time position coordinates and the at least three associated coordinates in the street lamp coordinate set;
(5) And carrying out height error check on the real-time position coordinates according to the average height difference data to generate a corresponding height error detection result.
Specifically, the server calculates distance data between the real-time position coordinates and each coordinate in the flight route coordinate set, calculates a heading angle between the real-time position coordinates and each coordinate in the flight route coordinate set according to the distance data, generates a heading offset detection result corresponding to the real-time position coordinates according to the heading angle, determines at least three associated coordinates in the street lamp coordinate set according to the target identification information, calculates average height difference data between the real-time position coordinates and the at least three associated coordinates in the street lamp coordinate set, performs height error check on the real-time position coordinates according to the average height difference data, and generates a corresponding height error detection result.
In a specific embodiment, the process of executing step S106 may specifically include the following steps:
(1) Calculating heading angle correction data of the target unmanned aerial vehicle according to the heading offset detection result, and calculating altitude correction data of the target unmanned aerial vehicle according to the altitude error detection result;
(2) The course angle correction data and the altitude correction data are put into a flight correction data set, and course and altitude correction is carried out on the target unmanned aerial vehicle according to the flight correction data set;
(3) And acquiring real-time flight data of the target unmanned aerial vehicle and generating a real-time flight data monitoring chart.
Specifically, the server calculates heading angle correction data of the target unmanned aerial vehicle according to the heading offset detection result, and calculates height correction data of the target unmanned aerial vehicle according to the height error detection result; the method comprises the steps that course angle correction data and altitude correction data are put into a flight correction data set by a server, course and altitude correction are carried out on a target unmanned aerial vehicle according to the flight correction data set, the flight correction data set of the target unmanned aerial vehicle is generated by the server according to a course deviation detection result and an altitude error detection result, a real-time flight data monitoring chart is generated by carrying out course and altitude correction on the target unmanned aerial vehicle according to the flight correction data set, corresponding course angle correction data are generated by the server according to the course deviation detection result and the altitude error detection result, further, the flight correction data set of the target unmanned aerial vehicle is generated by the server according to the course angle correction data, finally, course and altitude correction are carried out on the target unmanned aerial vehicle according to the flight correction data set by the server, the flight data detection chart is generated by the server, and the real-time flight data of the target unmanned aerial vehicle and the real-time flight data monitoring chart are generated by the server.
The unmanned aerial vehicle control method in the embodiment of the present invention is described above, and the intelligent street lamp system in the embodiment of the present invention is described below, referring to fig. 5, an embodiment of the intelligent street lamp system in the embodiment of the present invention includes:
the acquiring module 501 is configured to acquire a preset flight route of a target unmanned aerial vehicle in a target flight area, establish network connection between the target unmanned aerial vehicle and a preset intelligent street lamp system, and query a plurality of street lamp identification information preset in the target flight area;
the mapping module 502 is configured to construct a corresponding flight route coordinate set according to the preset flight route, and map street lamp coordinates of the plurality of street lamp identification information to obtain a street lamp coordinate set;
the detection module 503 is configured to detect a current flight trajectory of the target unmanned aerial vehicle and return a real-time position coordinate, and call an image acquisition terminal in the target unmanned aerial vehicle to acquire a plurality of groups of ground point cloud images of the real-time position coordinate;
the identifying module 504 is configured to identify street lamp identification information on the multiple groups of ground point cloud images, so as to obtain target identification information corresponding to the real-time position coordinates;
the processing module 505 is configured to perform course offset detection on the real-time position coordinates according to the flight route coordinate set to obtain a course offset detection result, and perform altitude error detection on the real-time position coordinates according to the target identification information to obtain an altitude error detection result;
And the correction module 506 is configured to generate a flight correction data set of the target unmanned aerial vehicle according to the heading deviation detection result and the altitude error detection result, and perform heading and altitude correction on the target unmanned aerial vehicle according to the flight correction data set, so as to generate a real-time flight data monitoring chart.
Through the cooperative cooperation of the components, a flight route coordinate set is constructed according to a preset flight route, and street lamp coordinate mapping is carried out on a plurality of street lamp identification information to obtain a street lamp coordinate set; collecting a plurality of groups of ground point cloud images of real-time position coordinates and identifying target identification information; performing course offset detection on the real-time position coordinates according to the flight route coordinate set to obtain a course offset detection result, and performing altitude error detection on the real-time position coordinates according to the target identification information to obtain an altitude error detection result; according to the method, the system and the device, the flight correction data set is generated according to the course deviation detection result and the altitude error detection result, the course and the altitude of the target unmanned aerial vehicle are corrected according to the flight correction data set, and the real-time flight data monitoring chart is generated.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, which are not repeated herein.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied essentially or in part or all of the technical solution or in part in the form of a software product stored in a storage medium, including instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a read-only memory (ROM), a random access memory (random acceS memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (2)
1. The unmanned aerial vehicle control method is characterized by comprising the following steps of:
acquiring a preset flight route of a target unmanned aerial vehicle in a target flight area, establishing network connection between the target unmanned aerial vehicle and a preset intelligent street lamp system, and inquiring a plurality of street lamp identification information preset in the target flight area; receiving a flight task sent by a control terminal, and carrying out task analysis on the flight task to obtain a target flight area; acquiring a preset flight route of a target unmanned aerial vehicle in a target flight area; establishing network connection between the target unmanned aerial vehicle and a preset intelligent street lamp system, and acquiring a network connection state; inquiring a plurality of street lamp identification information preset in the target flight area according to the network connection state; specifically, the preset flight route indicates a plurality of working areas corresponding to a plurality of operations to be executed by the unmanned aerial vehicle during the flight, the inter-area information is used for determining a plurality of inter-area flight routes, the unmanned aerial vehicle flies through the plurality of working areas along the plurality of inter-area flight routes, network connection is established between the target unmanned aerial vehicle and a preset intelligent street lamp system, a plurality of street lamp identification information preset in the target flight area is queried, when the network connection is established, network configuration parameters of the intelligent street lamp system are obtained, the network configuration parameters are sent to the target unmanned aerial vehicle through a communication link established between the unmanned aerial vehicle and the target unmanned aerial vehicle, and the target unmanned aerial vehicle adopts the network configuration parameters to establish network connection with the intelligent street lamp system; specifically, when the task analysis is performed on the flight task, the task analysis is performed on the flight task and the original data of the target area to obtain the target flight area, and a plurality of street lamp identification information preset in the target flight area is inquired according to the network connection state;
Constructing a corresponding flight route coordinate set according to the preset flight route, and mapping street lamp coordinates of the plurality of street lamp identification information to obtain a street lamp coordinate set; selecting a coordinate origin according to the target flight area, and constructing a target coordinate system according to the coordinate origin; according to the target coordinate system, carrying out coordinate transformation on the preset flight route to obtain a flight route coordinate set; based on the target coordinate system, performing street lamp coordinate mapping on the plurality of street lamp identification information to obtain a street lamp coordinate set; specifically, a three-dimensional coordinate system is built in advance, a corresponding flight path coordinate set is built according to a preset flight path, and street lamp coordinate mapping is conducted on a plurality of street lamp identification information, wherein when the street lamp coordinate mapping is conducted on the street lamp identification information, coordinate value matching is conducted on the street lamp identification information through the three-dimensional coordinate system, corresponding coordinate value information is determined, coordinate mapping is conducted on the street lamp identification information according to the coordinate value information, and a street lamp coordinate set is obtained; specifically, selecting a coordinate origin according to a target flight area, constructing a target coordinate system according to the coordinate origin, performing coordinate conversion on a preset flight route according to the target coordinate system to obtain a flight route coordinate set, wherein coordinate information of flight route points in the target coordinate system is obtained from a designated area, the coordinate information of the flight route in the target coordinate system is determined based on the distance relation between the flight route and a plurality of flight route points in the target coordinate system, the flight route coordinate set is finally obtained, and street lamp coordinate mapping is performed on a plurality of street lamp identification information based on the target coordinate system to obtain the street lamp coordinate set;
Detecting the current flight track of the target unmanned aerial vehicle and returning real-time position coordinates, and calling an image acquisition terminal in the target unmanned aerial vehicle to acquire a plurality of groups of ground point cloud images of the real-time position coordinates; the target unmanned aerial vehicle is controlled to execute the flight task, and a current flight track of the target unmanned aerial vehicle is generated; acquiring real-time position coordinates of the target unmanned aerial vehicle based on the target coordinate system, and transmitting the real-time position coordinates back to the control terminal; calling an image acquisition terminal in the target unmanned aerial vehicle based on a preset image acquisition range, and acquiring images of real-time position coordinates of the target unmanned aerial vehicle to obtain a plurality of groups of ground point cloud images corresponding to the real-time position coordinates; specifically, at least two groups of 2D images of a detected object, which are acquired by acquisition equipment under different preset brightness control parameters, are acquired, code value states in the predefined images are marked to obtain marked images, point cloud reconstruction is carried out on code value state marked areas of the marked images to obtain point cloud images corresponding to the marked images, and multiple groups of ground point cloud images corresponding to real-time position coordinates are determined;
Identifying street lamp identification information on the plurality of groups of ground point cloud images to obtain target identification information corresponding to the real-time position coordinates; image region segmentation is carried out on each group of ground point cloud images respectively to obtain a plurality of region images corresponding to each group of ground point cloud images; performing binarization processing on a plurality of area images corresponding to each group of ground point cloud images to obtain a binarized image of each area image; respectively identifying street lamp identification information of the binarized image of each area image to obtain target identification information corresponding to the real-time position coordinates; specifically, scanning each group of ground point cloud images to establish an adjacent relation between the segmented blocks, generating an initial bottom small-scale area structure, adding gray scales, textures and shape features to perform merging adjustment to obtain a plurality of area images corresponding to each group of ground point cloud images, performing binarization processing on the plurality of area images corresponding to each group of ground point cloud images to obtain a binarized image of each area image, and performing binarization processing on each acquired group of ground point cloud images to obtain a corresponding binarized image and confidence of each pixel point in the binarized image; determining the weight of each pixel point based on the confidence coefficient of each pixel point in the obtained binarized image; the obtained binarized images are fused based on the weight of each pixel point to obtain binarized images of each area image, and street lamp identification information identification is carried out on the binarized images of each area image to obtain target identification information corresponding to real-time position coordinates;
Performing course offset detection on the real-time position coordinates according to the flight route coordinate set to obtain a course offset detection result, and performing altitude error detection on the real-time position coordinates according to the target identification information to obtain an altitude error detection result; calculating distance data between the real-time position coordinates and each coordinate in the flight route coordinate set; according to the distance data, calculating a heading angle between the real-time position coordinate and each coordinate in the flight route coordinate set; generating a heading offset detection result corresponding to the real-time position coordinate according to the heading angle; determining at least three associated coordinates in the street lamp coordinate set according to the target identification information, and calculating average height difference data between the real-time position coordinates and the at least three associated coordinates in the street lamp coordinate set; performing height error verification on the real-time position coordinates according to the average height difference data to generate corresponding height error detection results;
generating a flight correction data set of the target unmanned aerial vehicle according to the course deviation detection result and the altitude error detection result, and carrying out course and altitude correction on the target unmanned aerial vehicle according to the flight correction data set to generate a real-time flight data monitoring chart; the method comprises the steps of calculating heading angle correction data of the target unmanned aerial vehicle according to a heading deviation detection result, and calculating altitude correction data of the target unmanned aerial vehicle according to an altitude error detection result; the course angle correction data and the altitude correction data are put into a flight correction data set, and course and altitude correction is carried out on the target unmanned aerial vehicle according to the flight correction data set; and acquiring real-time flight data of the target unmanned aerial vehicle and generating a real-time flight data monitoring chart.
2. An intelligent street lamp system, characterized in that the intelligent street lamp system comprises:
the system comprises an acquisition module, a control module and a control module, wherein the acquisition module is used for acquiring a preset flight route of a target unmanned aerial vehicle in a target flight area, establishing network connection between the target unmanned aerial vehicle and a preset intelligent street lamp system, and inquiring a plurality of street lamp identification information preset in the target flight area; receiving a flight task sent by a control terminal, and carrying out task analysis on the flight task to obtain a target flight area; acquiring a preset flight route of a target unmanned aerial vehicle in a target flight area; establishing network connection between the target unmanned aerial vehicle and a preset intelligent street lamp system, and acquiring a network connection state; inquiring a plurality of street lamp identification information preset in the target flight area according to the network connection state; specifically, the preset flight route indicates a plurality of working areas corresponding to a plurality of operations to be executed by the unmanned aerial vehicle during the flight, the inter-area information is used for determining a plurality of inter-area flight routes, the unmanned aerial vehicle flies through the plurality of working areas along the plurality of inter-area flight routes, network connection is established between the target unmanned aerial vehicle and a preset intelligent street lamp system, a plurality of street lamp identification information preset in the target flight area is queried, when the network connection is established, network configuration parameters of the intelligent street lamp system are obtained, the network configuration parameters are sent to the target unmanned aerial vehicle through a communication link established between the unmanned aerial vehicle and the target unmanned aerial vehicle, and the target unmanned aerial vehicle adopts the network configuration parameters to establish network connection with the intelligent street lamp system; specifically, when the task analysis is performed on the flight task, the task analysis is performed on the flight task and the original data of the target area to obtain the target flight area, and a plurality of street lamp identification information preset in the target flight area is inquired according to the network connection state;
The mapping module is used for constructing a corresponding flight route coordinate set according to the preset flight route, and carrying out street lamp coordinate mapping on the plurality of street lamp identification information to obtain a street lamp coordinate set; selecting a coordinate origin according to the target flight area, and constructing a target coordinate system according to the coordinate origin; according to the target coordinate system, carrying out coordinate transformation on the preset flight route to obtain a flight route coordinate set; based on the target coordinate system, performing street lamp coordinate mapping on the plurality of street lamp identification information to obtain a street lamp coordinate set; specifically, a three-dimensional coordinate system is built in advance, a corresponding flight path coordinate set is built according to a preset flight path, and street lamp coordinate mapping is conducted on a plurality of street lamp identification information, wherein when the street lamp coordinate mapping is conducted on the street lamp identification information, coordinate value matching is conducted on the street lamp identification information through the three-dimensional coordinate system, corresponding coordinate value information is determined, coordinate mapping is conducted on the street lamp identification information according to the coordinate value information, and a street lamp coordinate set is obtained; specifically, selecting a coordinate origin according to a target flight area, constructing a target coordinate system according to the coordinate origin, performing coordinate conversion on a preset flight route according to the target coordinate system to obtain a flight route coordinate set, wherein coordinate information of flight route points in the target coordinate system is obtained from a designated area, the coordinate information of the flight route in the target coordinate system is determined based on the distance relation between the flight route and a plurality of flight route points in the target coordinate system, the flight route coordinate set is finally obtained, and street lamp coordinate mapping is performed on a plurality of street lamp identification information based on the target coordinate system to obtain the street lamp coordinate set;
The detection module is used for detecting the current flight track of the target unmanned aerial vehicle and returning real-time position coordinates, and calling an image acquisition terminal in the target unmanned aerial vehicle to acquire a plurality of groups of ground point cloud images of the real-time position coordinates; the target unmanned aerial vehicle is controlled to execute the flight task, and a current flight track of the target unmanned aerial vehicle is generated; acquiring real-time position coordinates of the target unmanned aerial vehicle based on the target coordinate system, and transmitting the real-time position coordinates back to the control terminal; calling an image acquisition terminal in the target unmanned aerial vehicle based on a preset image acquisition range, and acquiring images of real-time position coordinates of the target unmanned aerial vehicle to obtain a plurality of groups of ground point cloud images corresponding to the real-time position coordinates; specifically, at least two groups of 2D images of a detected object, which are acquired by acquisition equipment under different preset brightness control parameters, are acquired, code value states in the predefined images are marked to obtain marked images, point cloud reconstruction is carried out on code value state marked areas of the marked images to obtain point cloud images corresponding to the marked images, and multiple groups of ground point cloud images corresponding to real-time position coordinates are determined;
The identification module is used for identifying street lamp identification information of the plurality of groups of ground point cloud images to obtain target identification information corresponding to the real-time position coordinates; image region segmentation is carried out on each group of ground point cloud images respectively to obtain a plurality of region images corresponding to each group of ground point cloud images; performing binarization processing on a plurality of area images corresponding to each group of ground point cloud images to obtain a binarized image of each area image; respectively identifying street lamp identification information of the binarized image of each area image to obtain target identification information corresponding to the real-time position coordinates; specifically, scanning each group of ground point cloud images to establish an adjacent relation between the segmented blocks, generating an initial bottom small-scale area structure, adding gray scales, textures and shape features to perform merging adjustment to obtain a plurality of area images corresponding to each group of ground point cloud images, performing binarization processing on the plurality of area images corresponding to each group of ground point cloud images to obtain a binarized image of each area image, and performing binarization processing on each acquired group of ground point cloud images to obtain a corresponding binarized image and confidence of each pixel point in the binarized image; determining the weight of each pixel point based on the confidence coefficient of each pixel point in the obtained binarized image; the obtained binarized images are fused based on the weight of each pixel point to obtain binarized images of each area image, and street lamp identification information identification is carried out on the binarized images of each area image to obtain target identification information corresponding to real-time position coordinates;
The processing module is used for carrying out course deviation detection on the real-time position coordinates according to the flight route coordinate set to obtain a course deviation detection result, and carrying out altitude error detection on the real-time position coordinates according to the target identification information to obtain an altitude error detection result; calculating distance data between the real-time position coordinates and each coordinate in the flight route coordinate set; according to the distance data, calculating a heading angle between the real-time position coordinate and each coordinate in the flight route coordinate set; generating a heading offset detection result corresponding to the real-time position coordinate according to the heading angle; determining at least three associated coordinates in the street lamp coordinate set according to the target identification information, and calculating average height difference data between the real-time position coordinates and the at least three associated coordinates in the street lamp coordinate set; performing height error verification on the real-time position coordinates according to the average height difference data to generate corresponding height error detection results;
the correction module is used for generating a flight correction data set of the target unmanned aerial vehicle according to the course deviation detection result and the altitude error detection result, correcting the course and the altitude of the target unmanned aerial vehicle according to the flight correction data set, and generating a real-time flight data monitoring chart; the method comprises the steps of calculating heading angle correction data of the target unmanned aerial vehicle according to a heading deviation detection result, and calculating altitude correction data of the target unmanned aerial vehicle according to an altitude error detection result; the course angle correction data and the altitude correction data are put into a flight correction data set, and course and altitude correction is carried out on the target unmanned aerial vehicle according to the flight correction data set; and acquiring real-time flight data of the target unmanned aerial vehicle and generating a real-time flight data monitoring chart.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310217332.4A CN115880953B (en) | 2023-03-08 | 2023-03-08 | Unmanned aerial vehicle management and control method and intelligent street lamp system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310217332.4A CN115880953B (en) | 2023-03-08 | 2023-03-08 | Unmanned aerial vehicle management and control method and intelligent street lamp system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115880953A CN115880953A (en) | 2023-03-31 |
| CN115880953B true CN115880953B (en) | 2023-05-23 |
Family
ID=85762075
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310217332.4A Active CN115880953B (en) | 2023-03-08 | 2023-03-08 | Unmanned aerial vehicle management and control method and intelligent street lamp system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115880953B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116343532B (en) * | 2023-05-26 | 2023-08-08 | 优选空天装备技术(北京)有限公司 | Intelligent combined unmanned aerial vehicle management and control system based on data analysis |
| CN117308938B (en) * | 2023-11-29 | 2024-02-20 | 长春通视光电技术股份有限公司 | Inertial navigation north-seeking convergence error rapid compensation method based on multiple laser ranging |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH02279997A (en) * | 1989-04-19 | 1990-11-15 | Nec Corp | Control device for target drone |
| CN107702714B (en) * | 2017-07-31 | 2020-01-07 | 广州维绅科技有限公司 | Positioning method, device and system |
| CN110413007B (en) * | 2019-08-30 | 2022-04-19 | 中国科学技术大学 | Control method and system for flight path of unmanned aerial vehicle, electronic device and medium |
| CN110825111A (en) * | 2019-11-15 | 2020-02-21 | 天津光电通信技术有限公司 | Unmanned aerial vehicle control method suitable for overhead warehouse goods inventory, goods inventory method, device, server and storage medium |
| CN111537954A (en) * | 2020-04-20 | 2020-08-14 | 孙剑 | Real-time high-dynamic fusion positioning method and device |
| CN111654814B (en) * | 2020-05-28 | 2022-03-25 | 联想(北京)有限公司 | Positioning method, unmanned aerial vehicle and computer readable storage medium |
| CN113516708B (en) * | 2021-05-25 | 2024-03-08 | 中国矿业大学 | Power transmission line inspection unmanned aerial vehicle accurate positioning system and method based on image recognition and UWB positioning fusion |
| CN113358135B (en) * | 2021-08-09 | 2021-11-26 | 青州耐威航电科技有限公司 | Method for correcting aircraft position by photoelectric measurement data |
-
2023
- 2023-03-08 CN CN202310217332.4A patent/CN115880953B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN115880953A (en) | 2023-03-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115880953B (en) | Unmanned aerial vehicle management and control method and intelligent street lamp system | |
| CN108256446B (en) | Method, device and equipment for determining lane line in road | |
| US10102433B2 (en) | Traveling road surface detection apparatus and traveling road surface detection method | |
| JP6682833B2 (en) | Database construction system for machine learning of object recognition algorithm | |
| CN111160302A (en) | Obstacle information identification method and device based on automatic driving environment | |
| CN101509781B (en) | Walking robot positioning system based on monocular cam | |
| EP3605390A1 (en) | Information processing method, information processing apparatus, and program | |
| CN110988912A (en) | Road target and distance detection method, system and device for automatic driving vehicle | |
| CN111563450B (en) | Data processing method, device, equipment and storage medium | |
| US20220343758A1 (en) | Data Transmission Method and Apparatus | |
| EP3843036B1 (en) | Sample labeling method and device, and damage category identification method and device | |
| EP4086846A1 (en) | Automatic detection of a calibration standard in unstructured lidar point clouds | |
| CN115049700A (en) | Target detection method and device | |
| CN112166458B (en) | Target detection and tracking method, system, equipment and storage medium | |
| CN115586749B (en) | Workpiece machining track control method based on machine vision and related device | |
| CN112883820A (en) | Road target 3D detection method and system based on laser radar point cloud | |
| US11941892B2 (en) | Method and device for providing data for creating a digital map | |
| CN105307116A (en) | Distributed vision positioning system and method based on mobile robot | |
| CN111709988A (en) | Method and device for determining characteristic information of object, electronic equipment and storage medium | |
| CN114528941A (en) | Sensor data fusion method and device, electronic equipment and storage medium | |
| WO2021063756A1 (en) | Improved trajectory estimation based on ground truth | |
| CN115729269B (en) | Unmanned aerial vehicle intelligent recognition system based on machine vision | |
| CN111899277A (en) | Moving object detection method and device, storage medium and electronic device | |
| CN114425774B (en) | Robot walking road recognition method, robot walking road recognition device, and storage medium | |
| CN112946603B (en) | Road maintenance detection system based on laser radar and detection method thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |