CN116310913A - Natural resource investigation monitoring method and device based on unmanned aerial vehicle measurement technology - Google Patents

Natural resource investigation monitoring method and device based on unmanned aerial vehicle measurement technology Download PDF

Info

Publication number
CN116310913A
CN116310913A CN202310533299.6A CN202310533299A CN116310913A CN 116310913 A CN116310913 A CN 116310913A CN 202310533299 A CN202310533299 A CN 202310533299A CN 116310913 A CN116310913 A CN 116310913A
Authority
CN
China
Prior art keywords
tree
image
symptom
standard
aerial vehicle
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202310533299.6A
Other languages
Chinese (zh)
Other versions
CN116310913B (en
Inventor
丁玉平
钱冬冬
赵建南
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jiangsu Suhai Information Technology Group Co ltd
Original Assignee
Jiangsu Suhai Information Technology Group Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jiangsu Suhai Information Technology Group Co ltd filed Critical Jiangsu Suhai Information Technology Group Co ltd
Priority to CN202310533299.6A priority Critical patent/CN116310913B/en
Publication of CN116310913A publication Critical patent/CN116310913A/en
Application granted granted Critical
Publication of CN116310913B publication Critical patent/CN116310913B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/10Terrestrial scenes
    • G06V20/17Terrestrial scenes taken from planes or by drones
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/762Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
    • G06V10/763Non-hierarchical techniques, e.g. based on statistics of modelling distributions
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/10Terrestrial scenes
    • G06V20/188Vegetation
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02ATECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A90/00Technologies having an indirect contribution to adaptation to climate change
    • Y02A90/10Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Remote Sensing (AREA)
  • Probability & Statistics with Applications (AREA)
  • Investigating Or Analysing Biological Materials (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a natural resource investigation and monitoring method and device based on a small unmanned aerial vehicle measurement technology, which relate to the technical field of natural resource investigation and monitoring and comprise the following steps: k second tree images acquired by the unmanned aerial vehicle are acquired; extracting K pieces of second tree image basic information, acquiring preset image setting standard information, performing image screening on the K pieces of second tree images, and performing standardized processing on the screened second tree images to acquire standard second tree images; determining tree symptom analysis information corresponding to a standard second tree image based on a pre-stored corresponding relation between the preset historical standard second tree image and the tree symptom analysis information; acquiring a third tree image acquired by the unmanned aerial vehicle, wherein the third tree image is an actual branch symptom image; and comparing the third tree image with the corresponding standard tree symptom image, and identifying and determining the identified tree symptom cause of the pathological tree object according to the comparison result.

Description

Natural resource investigation monitoring method and device based on unmanned aerial vehicle measurement technology
Technical Field
The invention relates to the technical field of natural resource investigation and monitoring, in particular to a natural resource investigation and monitoring method and device based on a small unmanned aerial vehicle measurement technology.
Background
Natural resources refer to various substances, energy sources and information resources with utilization value in nature, including but not limited to land, water resources, forest resources, mineral resources, marine resources, atmospheric resources, wild animal and plant resources and the like; the forests are extremely important components in the earth ecological system, play important roles in wind prevention, sand fixation, air purification and the like, and at present, some forests are influenced by natural disasters (such as fungi, drought, nematodes and the like) and factors such as human environment destruction (such as water source pollution and the like), so that a series of different symptoms (such as leaf discoloration, bark blackening, trunk dryness and the like) of some forests can occur; therefore, how to effectively monitor and manage forest natural resources has become an important point of current research.
At present, the existing method for investigation and monitoring of forest resources is mostly realized by manual inspection, technicians usually judge possible factors causing 'illness' of trees according to experience in the manual inspection process, however, forest trees are various in variety, and different tree types are hundreds of symptoms, and misjudgment is easy to occur only by experience of the technicians, and the method is time-consuming, labor-consuming and low in efficiency; some forest resource monitoring methods using satellite remote sensing and aerial remote sensing technologies, such as authorized bulletin numbers, also appear: CN114399685B discloses a remote sensing monitoring and evaluating method and device for forest pest and disease damage, and although the invention realizes the monitoring of forest pest and disease damage through satellite remote sensing, the inventor experiment finds that the method has the following defects:
(1) The method is only suitable for detecting large-area disease and insect pest phenomena of the forest, one-to-one type of disease and insect pest phenomena of the forest cannot be detected for each tree, and due to the reasons of forest types and pathology body quantity, training parameters for diagnosing the disease and insect pest phenomena of the forest through a model are too large, response time is slow, the disease and insect pest phenomena of the forest are caused to be overlong, so that investigation and monitoring efficiency of forest natural resources is low, and the method is difficult to be effectively applied to actual forest tree monitoring and diagnosis;
(2) The method has the advantages that the effective processing of the images of the pathological trees is lacking, and the basis data of the disease-seeing of the forest trees is single, so that the accuracy of the disease-seeing of the forest trees is low, and further, effective data support is difficult to be provided for the related management departments to manage forest resources;
(3) The feature relationship and the position of the pathological tree object cannot be updated in real time.
In view of this, we propose a method and apparatus for natural resource survey monitoring based on unmanned aerial vehicle measurement technology to solve the above-mentioned problems.
Disclosure of Invention
The invention aims to solve the defects in the prior art, and provides a natural resource investigation and monitoring method and device based on a small unmanned aerial vehicle measurement technology.
In order to achieve the above purpose, the present invention adopts the following technical scheme:
the method is applied to natural resource diagnosis and analysis equipment and comprises the following steps of:
s101: acquiring K second tree images of pathological tree objects in n target subareas in a monitored target forest, wherein n is more than or equal to 1, and K is more than or equal to 1; the second tree image is a leaf feature image of a corresponding pathology tree object;
s102: extracting K pieces of second tree image basic information, acquiring preset image setting standard information, performing image screening on the K pieces of second tree images, and performing standardized processing on the screened second tree images to acquire standard second tree images;
s103: determining tree symptom analysis information corresponding to a standard second tree image based on a pre-stored corresponding relation between the preset historical standard second tree image and the tree symptom analysis information; the tree symptom analysis information comprises m tree symptom causes and standard tree symptom images corresponding to each tree symptom cause, wherein m is more than or equal to 1;
s104: acquiring a third tree image acquired by the unmanned aerial vehicle, wherein the third tree image is an actual branch symptom image;
s105: and comparing the third tree image with the corresponding standard tree symptom image, and identifying and determining the identified tree symptom cause of the pathological tree object according to the comparison result.
Further, the method further comprises the following steps:
acquiring first tree images of each tree object in n target subregions in a monitored target forest acquired by an unmanned aerial vehicle; the first tree image is an overall crown image of each tree object;
performing primary identification on the first tree image based on a pre-constructed first machine learning model to obtain a corresponding tree type of the tree object;
selecting a corresponding second machine learning model to secondarily identify the first tree image according to the corresponding tree type to determine whether a tree pathology exists in the tree object;
if the disease state of the tree exists, determining that the tree object is the disease state tree object, and adjusting the gesture of the unmanned aerial vehicle to acquire K second tree images of the disease state tree object.
Further, the basic information comprises an actual image contrast value, an actual image brightness value and an actual image noise value of the second tree image; the preset image setting standard information comprises a set image contrast value, a set image brightness value and a set image noise value.
Further, extracting the basic information of the K second tree images, and obtaining preset image setting standard information to perform image screening on the K second tree images, including:
calculating a clear difference value between an actual image contrast value and a set actual image contrast value of each second tree image, and taking the clear difference value as a definition evaluation coefficient;
calculating the brightness difference value between the actual image brightness value and the set image brightness value of each second tree image, and taking the brightness difference value as a brightness evaluation coefficient;
calculating a noise difference value between the actual image noise value and the set image noise value of each second tree image, and taking the noise difference value as a noise evaluation coefficient;
formulating the sharpness evaluation coefficient, brightness evaluation coefficient and noise evaluation coefficient to obtain a screening evaluation coefficient
Figure SMS_3
,/>
Figure SMS_5
Wherein: />
Figure SMS_8
,/>
Figure SMS_2
And->
Figure SMS_4
As a weight factor, ++>
Figure SMS_6
For the clarity evaluation coefficient, +.>
Figure SMS_7
For the brightness evaluation coefficient, +.>
Figure SMS_1
Is a noise evaluation coefficient;
setting a screening evaluation threshold TK and setting a screening evaluation coefficient
Figure SMS_9
Comparing with the screening evaluation threshold TK, if the screening evaluation coefficient is +.>
Figure SMS_10
If the screening evaluation threshold TK is larger than or equal to the screening evaluation threshold TK, extracting a corresponding second tree image, otherwise, if the screening evaluation coefficient is +.>
Figure SMS_11
Less than screening evaluationA threshold TK, eliminating the corresponding second tree image;
screening and evaluating coefficients corresponding to the second tree image
Figure SMS_12
Marking the difference value with the screening evaluation threshold TK as a threshold difference value, and marking the screening evaluation coefficient +.>
Figure SMS_13
And sorting the threshold difference values which are larger than or equal to the screening evaluation threshold TK according to the size.
Further, the standardization processing is carried out on the second tree image after screening, which comprises the following steps:
extracting a second tree image with the largest threshold value difference;
performing image preprocessing on the second tree image with the largest threshold value difference, and performing graying processing on the second tree image with the largest threshold value difference after preprocessing to convert the second tree image into a gray image;
carrying out pixel point distinction on the gray image by using a K-means clustering algorithm, taking a pixel point clustering forming area as a target area and extracting, wherein the target area comprises a block or Q block leaf characteristic area, and Q is a positive integer less than or equal to 5;
calculating the area difference between the area D% of each target area and a preset standard area; the preset standard area is the template area P% + area optional error S%;
and taking the corresponding target area with the area difference smaller than a preset area difference threshold as a standard second tree image.
Further, comparing the third tree image with the corresponding standard tree symptom image to identify a corresponding tree symptom of the pathological tree object according to the comparison result, including:
vectorizing the third tree image and the corresponding standard tree symptom image, and calculating the image similarity of the vectorized third tree image and the corresponding vectorized standard tree symptom image according to the cosine metric model;
judging whether the image similarity is larger than a preset similarity threshold, if so, generating a comparison result, and extracting a tree symptom cause corresponding to the corresponding standard tree symptom image according to the comparison result to serve as the identified tree symptom cause of the pathological tree object.
Further, the method further comprises the following steps:
judging whether the comparison result has a null value or not, if so, recording that the corresponding third tree image is an unconventional image;
updating the corresponding relation between the preset standard second tree image and the tree symptom analysis information according to the unconventional image;
and acquiring corresponding coordinate data of the unmanned aerial vehicle according to the comparison result or the non-conventional image.
Natural resource investigation monitoring devices based on unmanned aerial vehicle measurement technique includes:
the first data acquisition module is used for acquiring K second tree images of pathological tree objects in n target subareas in the monitored target forest, wherein n is more than or equal to 1, and K is more than or equal to 1; the second tree image is a leaf feature image of a corresponding pathology tree object;
the image processing module is used for extracting K pieces of basic information of the second tree images, carrying out image screening on the K pieces of second tree images according to the basic information, and carrying out standardized processing on the screened second tree images so as to obtain standard second tree images;
the information determining module is used for determining tree symptom analysis information corresponding to the standard second tree image based on the corresponding relation between the pre-stored preset history standard second tree image and the tree symptom analysis information; the tree symptom analysis information comprises m tree symptom causes and standard tree symptom images corresponding to each tree symptom cause, wherein m is more than or equal to 1;
the second data acquisition module is used for acquiring a third tree image acquired by the unmanned aerial vehicle, wherein the third tree image is an actual branch symptom image;
and the symptom cause identification module is used for comparing the third tree image with the corresponding standard tree symptom image, and identifying and determining the identification tree symptom cause of the symptom tree object according to the comparison result.
A natural resource diagnostic analysis device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the unmanned aerial vehicle measurement technique-based natural resource survey monitoring method of any of the above when executing the computer program.
A computer readable storage medium having stored thereon a computer program which when executed by a processor implements the method for natural resource survey monitoring based on unmanned aerial vehicle measurement technique of any of the above.
Compared with the prior art, the invention has the beneficial effects that:
the application discloses a natural resource investigation monitoring method and device based on a small unmanned aerial vehicle measurement technology, wherein K second tree images of a symptom tree object are acquired through an unmanned aerial vehicle, screening and standardization processing are carried out on the K second tree images to acquire standard second tree images, then tree symptom analysis information corresponding to the standard second tree images is determined based on the corresponding relation between the pre-stored preset historical standard second tree images and tree symptom analysis information, then an actual branch symptom image is acquired, and finally the actual branch symptom image is compared with the standard tree symptom image, so that the corresponding tree symptom cause of the symptom tree object is determined; according to the tree disease screening method, the corresponding relation between the leaf characteristic images and the tree symptom analysis information is pre-stored, and tree disease factor range reduction screening is carried out according to the corresponding relation, so that the disease-seeing time of a disease-state tree object is shortened to a great extent; meanwhile, feature combination is carried out by combining leaf feature images and branch features, and etiology determination is carried out, so that the accuracy of disease-seeing of the disease-state tree object is greatly improved; in addition, one-to-one type 'seeing' and disease state tree object positioning are carried out through the unmanned aerial vehicle, so that the investigation and monitoring efficiency of forest natural resources is greatly improved; the method is beneficial to being applied to the aspect of actual forest tree monitoring and diagnosis, and can provide effective data support for the management of forest resources by related management departments.
Drawings
The accompanying drawings are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate the invention and together with the embodiments of the invention, serve to explain the invention.
Fig. 1 is an overall flowchart of a natural resource investigation monitoring method based on a unmanned aerial vehicle measurement technology provided by the invention;
fig. 2 is a schematic diagram of the overall structure of the natural resource investigation monitoring device based on the unmanned aerial vehicle measurement technology.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments.
In the description of the present invention, it should be understood that the terms "upper," "lower," "front," "rear," "left," "right," "top," "bottom," "inner," "outer," and the like indicate or are based on the orientation or positional relationship shown in the drawings, merely to facilitate description of the present invention and to simplify the description, and do not indicate or imply that the devices or elements referred to must have a specific orientation, be configured and operated in a specific orientation, and thus should not be construed as limiting the present invention.
Example 1
Referring to fig. 1, the disclosure of the present embodiment provides a method for monitoring natural resource survey based on a measurement technique of a small unmanned aerial vehicle, where the method is applied to a natural resource diagnostic and analytical device, and the method includes:
s101: acquiring K second tree images of pathological tree objects in n target subareas in a monitored target forest, wherein n is more than or equal to 1, and K is more than or equal to 1; the second tree image is a leaf feature image of a corresponding pathology tree object;
it should be noted that: the n target subareas are obtained by dividing the area of a pre-acquired monitored target forest into equal proportion rectangles; further to be described is: after n target subareas are obtained, serial number marking naming and sorting are carried out on each target subarea, and a flight route is set for the unmanned aerial vehicle according to each sorted target subarea, so that the unmanned aerial vehicle can carry out data acquisition according to a preset route, and the data acquisition efficiency of the unmanned aerial vehicle is improved;
specifically, before acquiring K second tree images, the method further includes:
acquiring first tree images of each tree object in n target subregions in a monitored target forest acquired by an unmanned aerial vehicle; the first tree image is an overall crown image of each tree object;
performing primary identification on the first tree image based on a pre-constructed first machine learning model to obtain a corresponding tree type of the tree object;
it should be noted that: the first machine learning model is specifically a neural network algorithm model and is used for solving the problem of multi-classification of the tree objects, namely identifying corresponding tree types, and is obtained by inputting a large number of first tree images of the tree objects which are pre-acquired and subjected to type labeling into a pre-built neural network algorithm model as sample data for training;
selecting a corresponding second machine learning model to secondarily identify the first tree image according to the corresponding tree type to determine whether a tree pathology exists in the tree object;
it should be noted that: the second machine learning model is specifically one of logistic regression, a support vector machine or a random forest algorithm, and is used for realizing the classification of the tree pathology of the tree object (namely, the tree pathology exists or the tree pathology does not exist); the second machine learning model is obtained by taking a plurality of first tree images of the pre-acquired and pathology-marked tree objects as sample data, and inputting the sample data into one model of a pre-constructed logistic regression, a support vector machine or a random forest algorithm for training;
if the disease state of the tree exists, determining that the tree object is a disease state tree object, and adjusting the gesture of the unmanned aerial vehicle to acquire K second tree images of the disease state tree object;
extracting K pieces of second tree image basic information, acquiring preset image setting standard information, performing image screening on the K pieces of second tree images, and performing standardized processing on the screened second tree images to acquire standard second tree images;
specifically, the basic information includes an actual image contrast value, an actual image brightness value and an actual image noise value of the second tree image; the preset image setting standard information comprises a set image contrast value, a set image brightness value and a set image noise value;
specifically, extracting the basic information of the K second tree images, and obtaining preset image setting standard information to perform image screening on the K second tree images, including:
calculating a clear difference value between an actual image contrast value and a set actual image contrast value of each second tree image, and taking the clear difference value as a definition evaluation coefficient;
calculating the brightness difference value between the actual image brightness value and the set image brightness value of each second tree image, and taking the brightness difference value as a brightness evaluation coefficient;
calculating a noise difference value between the actual image noise value and the set image noise value of each second tree image, and taking the noise difference value as a noise evaluation coefficient;
formulating the sharpness evaluation coefficient, brightness evaluation coefficient and noise evaluation coefficient to obtain a screening evaluation coefficient
Figure SMS_14
The calculation formula is ∈>
Figure SMS_18
Wherein: />
Figure SMS_20
,/>
Figure SMS_15
And->
Figure SMS_17
As a weight factor, ++>
Figure SMS_19
For the clarity evaluation coefficient, +.>
Figure SMS_21
For the brightness evaluation coefficient, +.>
Figure SMS_16
Is a noise evaluation coefficient;
setting a screening evaluation threshold TK and setting a screening evaluation coefficient
Figure SMS_22
Comparing with the screening evaluation threshold TK, if the screening evaluation coefficient is +.>
Figure SMS_23
If the screening evaluation threshold TK is larger than or equal to the screening evaluation threshold TK, extracting a corresponding second tree image, otherwise, if the screening evaluation coefficient is +.>
Figure SMS_24
If the image is smaller than the screening evaluation threshold TK, the corresponding second tree image is removed;
screening and evaluating coefficients corresponding to the second tree image
Figure SMS_25
Marking the difference value with the screening evaluation threshold TK as a threshold difference value, and marking the screening evaluation coefficient +.>
Figure SMS_26
Sorting the threshold difference values which are larger than or equal to the screening evaluation threshold TK according to the size;
optionally, the normalizing process is performed on the screened second tree image, including:
extracting a second tree image with the largest threshold value difference;
performing image preprocessing on the second tree image with the largest threshold value difference, and performing graying processing on the second tree image with the largest threshold value difference after preprocessing to convert the second tree image into a gray image;
carrying out pixel point distinction on the gray image by using a K-means clustering algorithm, taking a pixel point clustering forming area as a target area and extracting, wherein the target area comprises a block or Q block leaf characteristic area, and Q is a positive integer which is more than 1 and less than or equal to 5;
calculating the area difference between the area D% of each target area and a preset standard area; the preset standard area is the template area P% + area optional error S%;
it should be noted that: the S% represents the size of the area selectable error, namely the area error range of the area D% of the target area and the area P% of the template area; the further explanation is as follows: the area D of the target area, the template area P% and the area selectable error S% are all fixed values, and are determined according to the specific leaf type, and the invention does not limit the specific leaf type;
it will be appreciated that: by calculating the area difference between the area D% of each target area and the template area P% + and the area selectable error S%, the background of some non-leaf features can be removed, so that the accuracy of the tree symptom analysis information corresponding to the standard second tree image is ensured to be determined subsequently;
taking the corresponding target area with the area difference smaller than a preset area difference threshold value as a standard second tree image;
s103: determining tree symptom analysis information corresponding to a standard second tree image based on a pre-stored corresponding relation between the preset historical standard second tree image and the tree symptom analysis information; the tree symptom analysis information comprises m tree symptom causes and standard tree symptom images corresponding to each tree symptom cause, wherein m is more than or equal to 1;
it should be noted that: the standard tree symptom image is specifically a branch characteristic image of a symptom tree object;
it can be understood that: a plurality of tree symptoms and tree symptom analysis information corresponding to each tree symptom are stored in advance in natural resource diagnosis and analysis equipment, and each tree symptom analysis information comprises a plurality of tree symptoms; the further explanation is that a plurality of tree symptoms corresponding to each tree symptom are determined through the leaf characteristic image, so that the tree symptom scope of the symptom tree object is reduced, the reduction of the data processing dimension is realized on the data processing level, the disease-seeing time of the symptom tree object is shortened to a great extent, and the method is beneficial to the application in the investigation and detection of a large amount of forest resources;
s104: acquiring a third tree image acquired by the unmanned aerial vehicle, wherein the third tree image is an actual branch symptom image;
the third tree image is an actual branch symptom image; the third tree image is acquired after determining tree symptom analysis information corresponding to the standard second tree image; also to be described is: the third tree image also comprises an image processing link which is approximately consistent with the image processing process of the second tree image, and details can refer to the image processing process of the second tree image, so that the invention does not redundant description;
s105: comparing the third tree image with the corresponding standard tree symptom image, and identifying and determining the identified tree symptom cause of the pathological tree object according to the comparison result;
specifically, the image comparison is performed on the third tree image and the corresponding standard tree symptom image, and the corresponding tree symptom cause of the pathological tree object is identified according to the comparison result, which comprises the following steps:
vectorizing the third tree image and the corresponding standard tree symptom image, and calculating the image similarity of the vectorized third tree image and the corresponding vectorized standard tree symptom image according to the cosine metric model;
judging whether the image similarity is larger than a preset similarity threshold, if so, generating a comparison result, and extracting a tree symptom cause corresponding to a corresponding standard tree symptom image according to the comparison result to serve as a recognition tree symptom cause of the pathological tree object;
it should be noted that: if the comparison result which is larger than the preset similarity threshold value does not exist, displaying a null value;
optionally, the method may further include:
judging whether the comparison result has a null value or not, if so, recording that the corresponding third tree image is an unconventional image;
updating the corresponding relation between the preset standard second tree image and the tree symptom analysis information according to the unconventional image;
acquiring corresponding coordinate data of the unmanned aerial vehicle according to the comparison result or the unconventional image;
it should be noted that: the natural resource diagnosis and analysis equipment is in remote communication connection with the unmanned aerial vehicle;
as can be appreciated in summary: when determining the symptom of each tree object of a detected target forest, firstly acquiring an integral crown image of each tree object through an unmanned aerial vehicle, determining whether a tree symptom exists in a corresponding tree object according to the integral crown image, acquiring K second tree images of the symptom tree object through the unmanned aerial vehicle if the tree symptom exists, screening and standardizing the K second tree images to acquire standard second tree images, determining tree symptom analysis information corresponding to the standard second tree images based on the corresponding relation between the pre-stored preset historical standard second tree images and the tree symptom analysis information, reducing the data processing dimension, shortening the disease seeing time of the symptom tree object, acquiring actual branch symptom images, and finally comparing the actual branch symptom images with the standard tree symptom images, thereby determining the corresponding tree symptom of the symptom tree object; compared with the existing investigation and monitoring mode of forest natural resources, the method has the advantages that feature combination is carried out by combining leaf feature images and branch features, and etiology is determined, so that the accuracy of disease seeing of disease-state tree objects is greatly improved, and accurate diagnosis of the disease-state tree objects can be effectively realized; in addition, the corresponding relation between the leaf characteristic image and the tree symptom analysis information is pre-stored, and tree symptom is reduced and screened according to the corresponding relation, so that the disease-checking time of a disease-state tree object is greatly shortened, the investigation and monitoring efficiency of forest natural resources is greatly improved, each tree is checked and monitored in a one-to-one mode, the position of each tree is determined by no person, the corresponding relation between the second tree image and the tree symptom analysis information is updated, and therefore efficient management of forest natural resources is facilitated, and the relevant management departments are guided to conduct efficient forest resource management in a large number of forest systems.
Example two
Referring to fig. 2, the disclosure of the present embodiment provides a natural resource investigation and monitoring device based on unmanned aerial vehicle measurement technology, including:
the first data acquisition module 201 is used for acquiring K second tree images of pathological tree objects in n target subareas in the monitored target forest, wherein n is more than or equal to 1, and K is more than or equal to 1; the second tree image is a leaf feature image of a corresponding pathology tree object;
the image processing module 202 is configured to extract K pieces of basic information of the second tree image, perform image screening on the K pieces of second tree image according to the basic information, and perform standardization processing on the screened second tree image to obtain a standard second tree image;
the information determining module 203 determines tree symptom analysis information corresponding to the standard second tree image based on a corresponding relation between the pre-stored preset history standard second tree image and the tree symptom analysis information; the tree symptom analysis information comprises m tree symptom causes and standard tree symptom images corresponding to each tree symptom cause, wherein m is more than or equal to 1;
the second data obtaining module 204 is configured to obtain a third tree image collected by the unmanned aerial vehicle, where the third tree image is an actual branch symptom image;
and the symptom cause identification module 205 is configured to compare the third tree image with a corresponding standard tree symptom image, and identify and determine an identified tree symptom cause of the symptom tree object according to the comparison result.
Optionally, the natural resource investigation monitoring device based on unmanned aerial vehicle measurement technology may further include:
the threshold value judging module 206 is used for judging whether the comparison result has a null value or not, and if so, recording that the corresponding third tree image is an unconventional image;
the information updating module 207 updates the corresponding relation between the second tree image with the preset standard and the tree symptom analysis information according to the non-conventional image;
it should be noted that: if the unconventional image is identified, indicating that the pathology of the corresponding pathology tree object is not found before or belongs to a novel pathology phenomenon, carrying out corresponding targeted analysis by a technician, giving out a corresponding tree symptom cause, collecting a corresponding standard tree symptom image of the corresponding tree symptom cause under the unconventional image, and updating the corresponding relation between a preset standard second tree image and tree symptom analysis information according to the corresponding tree symptom cause under the unconventional image and the corresponding standard tree symptom image of the corresponding tree symptom cause under the unconventional image;
the object position obtaining module 208 obtains corresponding coordinate data of the unmanned aerial vehicle according to the comparison result or the unconventional image.
In one embodiment, the present invention further provides a natural resource diagnosis and analysis device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes the computer program to implement the method for natural resource investigation and monitoring based on the unmanned aerial vehicle measurement technology according to any one of the above embodiments.
In one embodiment, the present invention further provides a computer readable storage medium including a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the method for natural resource survey monitoring based on unmanned aerial vehicle measurement technology according to any one of the above embodiments when the computer program is executed.
The above embodiments may be implemented in whole or in part by software, hardware, firmware, or any other combination. When implemented in software, the above-described embodiments may be implemented in whole or in part in the form of a computer program product. The computer program product comprises one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, the processes or functions described in accordance with the embodiments of the present application are all or partially produced. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website site, computer, server, or data center to another website site, computer, server, or data center by wired (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains one or more sets of available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. The semiconductor medium may be a solid state disk.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, and are not repeated herein.
In the several embodiments provided in the present application, it should be understood that the disclosed systems, apparatuses, and methods may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely one, and there may be additional divisions in actual implementation, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other forms.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in each embodiment of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a read-only memory (ROM), a random access memory (random access memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The foregoing is only a preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art, who is within the scope of the present invention, should make equivalent substitutions or modifications according to the technical scheme of the present invention and the inventive concept thereof, and should be covered by the scope of the present invention.

Claims (10)

1. The natural resource investigation monitoring method based on the unmanned aerial vehicle measurement technology is characterized by being applied to natural resource diagnosis analysis equipment, and comprises the following steps:
s101: acquiring K second tree images of pathological tree objects in n target subareas in a monitored target forest, wherein n is more than or equal to 1, and K is more than or equal to 1; the second tree image is a leaf feature image of a corresponding pathology tree object;
s102: extracting K pieces of second tree image basic information, acquiring preset image setting standard information, performing image screening on the K pieces of second tree images, and performing standardized processing on the screened second tree images to acquire standard second tree images;
s103: determining tree symptom analysis information corresponding to a standard second tree image based on a pre-stored corresponding relation between the preset historical standard second tree image and the tree symptom analysis information; the tree symptom analysis information comprises m tree symptom causes and standard tree symptom images corresponding to each tree symptom cause, wherein m is more than or equal to 1;
s104: acquiring a third tree image acquired by the unmanned aerial vehicle, wherein the third tree image is an actual branch symptom image;
s105: and comparing the third tree image with the corresponding standard tree symptom image, and identifying and determining the identified tree symptom cause of the pathological tree object according to the comparison result.
2. The method for monitoring natural resource survey based on unmanned aerial vehicle measurement technology of claim 1, further comprising:
acquiring first tree images of each tree object in n target subregions in a monitored target forest acquired by an unmanned aerial vehicle; the first tree image is an overall crown image of each tree object;
performing primary identification on the first tree image based on a pre-constructed first machine learning model to obtain a corresponding tree type of the tree object;
selecting a corresponding second machine learning model to secondarily identify the first tree image according to the corresponding tree type to determine whether a tree pathology exists in the tree object;
if the disease state of the tree exists, determining that the tree object is the disease state tree object, and adjusting the gesture of the unmanned aerial vehicle to acquire K second tree images of the disease state tree object.
3. The unmanned aerial vehicle measurement technology-based natural resource investigation and monitoring method according to claim 2, wherein the second tree image basic information comprises an actual image contrast value, an actual image brightness value and an actual image noise value of a second tree image; the preset image setting standard information comprises a set image contrast value, a set image brightness value and a set image noise value.
4. The method for monitoring natural resource survey based on unmanned aerial vehicle measurement technology according to claim 3, wherein extracting the K pieces of second tree image basic information and obtaining preset image setting standard information to perform image screening on the K pieces of second tree images comprises:
calculating a clear difference value between an actual image contrast value and a set actual image contrast value of each second tree image, and taking the clear difference value as a definition evaluation coefficient;
formulating the sharpness evaluation coefficient, brightness evaluation coefficient and noise evaluation coefficient to obtain a screening evaluation coefficient
Figure QLYQS_2
,/>
Figure QLYQS_4
Wherein: />
Figure QLYQS_6
,/>
Figure QLYQS_3
And->
Figure QLYQS_5
As a weight factor, ++>
Figure QLYQS_7
For the clarity evaluation coefficient, +.>
Figure QLYQS_8
For the brightness evaluation coefficient, +.>
Figure QLYQS_1
Is a noise evaluation coefficient;
setting a screening evaluation threshold TK and setting a screening evaluation coefficient
Figure QLYQS_9
Comparing with the screening evaluation threshold TK, if the screening evaluation coefficient is
Figure QLYQS_10
If the screening evaluation threshold TK is larger than or equal to the screening evaluation threshold TK, extracting a corresponding second tree image, and if the screening evaluation coefficient is +.>
Figure QLYQS_11
If the image is smaller than the screening evaluation threshold TK, the corresponding second tree image is removed;
screening and evaluating coefficients corresponding to the second tree image
Figure QLYQS_12
Marking the difference value with the screening evaluation threshold TK as a threshold difference value, and marking the screening evaluation coefficient +.>
Figure QLYQS_13
And sorting the threshold difference values which are larger than or equal to the screening evaluation threshold TK according to the size.
5. The method for monitoring natural resource survey based on unmanned aerial vehicle measurement technology according to claim 4, wherein the step of standardizing the screened second tree image comprises:
extracting a second tree image with the largest threshold value difference;
performing image preprocessing on the second tree image with the largest threshold value difference, and performing graying processing on the second tree image with the largest threshold value difference after preprocessing to convert the second tree image into a gray image;
carrying out pixel point distinction on the gray image by using a K-means clustering algorithm, taking a pixel point clustering forming area as a target area and extracting, wherein the target area comprises a block or Q block leaf characteristic area, and Q is a positive integer which is more than 1 and less than or equal to 5;
calculating the area difference between the area D% of each target area and a preset standard area; the preset standard area is the template area P% + area optional error S%;
and taking the corresponding target area with the area difference smaller than a preset area difference threshold as a standard second tree image.
6. The method for monitoring natural resource survey based on unmanned aerial vehicle measurement technology according to claim 5, wherein comparing the third tree image with the corresponding standard tree symptom image, and identifying the corresponding cause of the symptom tree object according to the comparison result, comprises:
vectorizing the third tree image and the corresponding standard tree symptom image, and calculating the image similarity of the vectorized third tree image and the corresponding vectorized standard tree symptom image according to the cosine metric model;
judging whether the image similarity is larger than a preset similarity threshold, if so, generating a comparison result, and extracting a tree symptom cause corresponding to the corresponding standard tree symptom image according to the comparison result to serve as the identified tree symptom cause of the pathological tree object.
7. The unmanned aerial vehicle measurement technology-based natural resource survey monitoring method of claim 6, further comprising:
judging whether the comparison result has a null value or not, if so, recording that the corresponding third tree image is an unconventional image;
updating the corresponding relation between the preset standard second tree image and the tree symptom analysis information according to the unconventional image;
and acquiring corresponding coordinate data of the unmanned aerial vehicle according to the comparison result or the non-conventional image.
8. Natural resource investigation monitoring devices based on unmanned aerial vehicle measurement technique, its characterized in that includes:
the first data acquisition module is used for acquiring K second tree images of pathological tree objects in n target subareas in the monitored target forest, wherein n is more than or equal to 1, and K is more than or equal to 1; the second tree image is a leaf feature image of a corresponding pathology tree object;
the image processing module is used for extracting K pieces of basic information of the second tree images, carrying out image screening on the K pieces of second tree images according to the basic information, and carrying out standardized processing on the screened second tree images so as to obtain standard second tree images;
the information determining module is used for determining tree symptom analysis information corresponding to the standard second tree image based on the corresponding relation between the pre-stored preset history standard second tree image and the tree symptom analysis information; the tree symptom analysis information comprises m tree symptom causes and standard tree symptom images corresponding to each tree symptom cause, wherein m is more than or equal to 1;
the second data acquisition module is used for acquiring a third tree image acquired by the unmanned aerial vehicle, wherein the third tree image is an actual branch symptom image;
and the symptom cause identification module is used for comparing the third tree image with the corresponding standard tree symptom image, and identifying and determining the identification tree symptom cause of the symptom tree object according to the comparison result.
9. A natural resource diagnostic analysis device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method for natural resource investigation and monitoring based on unmanned aerial vehicle measurement technique according to any of claims 1 to 7 when executing the computer program.
10. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the method for natural resource investigation and monitoring based on unmanned aerial vehicle measurement technology according to any of claims 1 to 7.
CN202310533299.6A 2023-05-12 2023-05-12 Natural resource investigation monitoring method and device based on unmanned aerial vehicle measurement technology Active CN116310913B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310533299.6A CN116310913B (en) 2023-05-12 2023-05-12 Natural resource investigation monitoring method and device based on unmanned aerial vehicle measurement technology

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310533299.6A CN116310913B (en) 2023-05-12 2023-05-12 Natural resource investigation monitoring method and device based on unmanned aerial vehicle measurement technology

Publications (2)

Publication Number Publication Date
CN116310913A true CN116310913A (en) 2023-06-23
CN116310913B CN116310913B (en) 2023-07-25

Family

ID=86829080

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310533299.6A Active CN116310913B (en) 2023-05-12 2023-05-12 Natural resource investigation monitoring method and device based on unmanned aerial vehicle measurement technology

Country Status (1)

Country Link
CN (1) CN116310913B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116740651A (en) * 2023-08-11 2023-09-12 南京吾悦农业科技有限公司 Edible fungus cultivation monitoring method and system based on intelligent decision

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109684914A (en) * 2018-11-09 2019-04-26 武汉伊莱维特电力科技有限公司 Based on unmanned plane image intelligent identification Method
CN110097107A (en) * 2019-04-23 2019-08-06 安徽大学 Alternaria mali roberts disease recognition and classification method based on convolutional neural networks
CN110598532A (en) * 2019-07-31 2019-12-20 长春市万易科技有限公司 Tree pest and disease damage monitoring system and method
CN111598001A (en) * 2020-05-18 2020-08-28 哈尔滨理工大学 Apple tree pest and disease identification method based on image processing
CN112633161A (en) * 2020-12-21 2021-04-09 重庆英卡电子有限公司 Pine wood nematode disease withered and dead tree detection and positioning method based on high-altitude pan-tilt recognition
CN113033520A (en) * 2021-05-25 2021-06-25 华中农业大学 Tree nematode disease wood identification method and system based on deep learning
CN114266979A (en) * 2021-12-31 2022-04-01 湖北省农业科学院农业质量标准与检测技术研究所 Tea tree pest and disease intelligent identification method, device, equipment and storage medium
CN114387528A (en) * 2021-12-29 2022-04-22 浙江同创空间技术有限公司 Pine nematode disease monitoring space-air-ground integrated monitoring method
CN115661647A (en) * 2022-10-25 2023-01-31 新乡市林业技术推广站(新乡市林业工作站) Timed inspection method for tree planting growth conditions
CN115989766A (en) * 2023-02-15 2023-04-21 湖北九森林业股份有限公司 Seedling cultivation method with high survival rate

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109684914A (en) * 2018-11-09 2019-04-26 武汉伊莱维特电力科技有限公司 Based on unmanned plane image intelligent identification Method
CN110097107A (en) * 2019-04-23 2019-08-06 安徽大学 Alternaria mali roberts disease recognition and classification method based on convolutional neural networks
CN110598532A (en) * 2019-07-31 2019-12-20 长春市万易科技有限公司 Tree pest and disease damage monitoring system and method
CN111598001A (en) * 2020-05-18 2020-08-28 哈尔滨理工大学 Apple tree pest and disease identification method based on image processing
CN112633161A (en) * 2020-12-21 2021-04-09 重庆英卡电子有限公司 Pine wood nematode disease withered and dead tree detection and positioning method based on high-altitude pan-tilt recognition
CN113033520A (en) * 2021-05-25 2021-06-25 华中农业大学 Tree nematode disease wood identification method and system based on deep learning
CN114387528A (en) * 2021-12-29 2022-04-22 浙江同创空间技术有限公司 Pine nematode disease monitoring space-air-ground integrated monitoring method
CN114266979A (en) * 2021-12-31 2022-04-01 湖北省农业科学院农业质量标准与检测技术研究所 Tea tree pest and disease intelligent identification method, device, equipment and storage medium
CN115661647A (en) * 2022-10-25 2023-01-31 新乡市林业技术推广站(新乡市林业工作站) Timed inspection method for tree planting growth conditions
CN115989766A (en) * 2023-02-15 2023-04-21 湖北九森林业股份有限公司 Seedling cultivation method with high survival rate

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
张冲;李淼;张健;董俊;高会议;陈雷;吴娜;: "基于改进中值滤波的梨树病害彩色图像分割", 计算机应用, no. 2, pages 202 - 204 *
张帅堂;王紫烟;邹修国;钱燕;余磊;: "基于高光谱图像和遗传优化神经网络的茶叶病斑识别", 农业工程学报, no. 22, pages 208 - 215 *
张衡;潘洁;廖振峰;: "森林病虫害高光谱遥感监测的研究进展", 中国森林病虫, no. 02, pages 30 - 34 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116740651A (en) * 2023-08-11 2023-09-12 南京吾悦农业科技有限公司 Edible fungus cultivation monitoring method and system based on intelligent decision
CN116740651B (en) * 2023-08-11 2023-10-17 南京吾悦农业科技有限公司 Edible fungus cultivation monitoring method and system based on intelligent decision

Also Published As

Publication number Publication date
CN116310913B (en) 2023-07-25

Similar Documents

Publication Publication Date Title
CN110796284B (en) Method and device for predicting pollution level of fine particulate matters and computer equipment
CN112101159B (en) Multi-temporal forest remote sensing image change monitoring method
CN113705478B (en) Mangrove single wood target detection method based on improved YOLOv5
Grinand et al. Estimating deforestation in tropical humid and dry forests in Madagascar from 2000 to 2010 using multi-date Landsat satellite images and the random forests classifier
Hallau et al. Automated identification of sugar beet diseases using smartphones
Eng et al. The use of VARI, GLI, and VIgreen formulas in detecting vegetation in aerial images
Ahmed et al. Plant disease detection using machine learning approaches
CN111666855B (en) Animal three-dimensional parameter extraction method and system based on unmanned aerial vehicle and electronic equipment
CN114219847B (en) Method and system for determining crop planting area based on phenological characteristics and storage medium
CN116310913B (en) Natural resource investigation monitoring method and device based on unmanned aerial vehicle measurement technology
Ninomiya High-throughput field crop phenotyping: current status and challenges
CN104680185B (en) Hyperspectral image classification method based on boundary point reclassification
US11790410B2 (en) System and method for natural capital measurement
CN115439654B (en) Method and system for finely dividing weakly supervised farmland plots under dynamic constraint
CN117173590A (en) Water body abnormality monitoring method based on multisource time sequence remote sensing image
Liu et al. Automated classification of stems and leaves of potted plants based on point cloud data
CN112702408A (en) Internet of things system and method based on multi-sensing function
CN117171533B (en) Real-time acquisition and processing method and system for geographical mapping operation data
JP5164802B2 (en) Recognition system, recognition method, and recognition program
CN117351659A (en) Hydrogeological disaster monitoring device and monitoring method
CN109636194B (en) Multi-source cooperative detection method and system for major change of power transmission and transformation project
CN114283367B (en) Artificial intelligent open fire detection method and system for garden fire early warning
CN113240340B (en) Soybean planting area analysis method, device, equipment and medium based on fuzzy classification
Chellasamy et al. Automatic training sample selection for a multi-evidence based crop classification approach
CN109886142B (en) Crop interpretation method based on SAR technology

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant