CN116663408A - Establishment method of optimal digging pose of pseudo-ginseng - Google Patents

Establishment method of optimal digging pose of pseudo-ginseng Download PDF

Info

Publication number
CN116663408A
CN116663408A CN202310621573.5A CN202310621573A CN116663408A CN 116663408 A CN116663408 A CN 116663408A CN 202310621573 A CN202310621573 A CN 202310621573A CN 116663408 A CN116663408 A CN 116663408A
Authority
CN
China
Prior art keywords
image
point
steps
digging
pseudo
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202310621573.5A
Other languages
Chinese (zh)
Other versions
CN116663408B (en
Inventor
王成琳
韩启宇
李春江
王浩名
王法安
张兆国
解开婷
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Kunming University of Science and Technology
Original Assignee
Kunming University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Kunming University of Science and Technology filed Critical Kunming University of Science and Technology
Priority to CN202310621573.5A priority Critical patent/CN116663408B/en
Publication of CN116663408A publication Critical patent/CN116663408A/en
Application granted granted Critical
Publication of CN116663408B publication Critical patent/CN116663408B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F30/00Computer-aided design [CAD]
    • G06F30/20Design optimisation, verification or simulation
    • G06F30/27Design optimisation, verification or simulation using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
    • AHUMAN NECESSITIES
    • A01AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
    • A01DHARVESTING; MOWING
    • A01D33/00Accessories for digging harvesters
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84Systems specially adapted for particular applications
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N27/00Investigating or analysing materials by the use of electric, electrochemical, or magnetic means
    • G01N27/02Investigating or analysing materials by the use of electric, electrochemical, or magnetic means by investigating impedance
    • G01N27/04Investigating or analysing materials by the use of electric, electrochemical, or magnetic means by investigating impedance by investigating resistance
    • G01N27/041Investigating or analysing materials by the use of electric, electrochemical, or magnetic means by investigating impedance by investigating resistance of a solid body
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10Complex mathematical operations
    • G06F17/18Complex mathematical operations for evaluating statistical data, e.g. average values, frequency distributions, probability functions, regression analysis
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00Geometric image transformations in the plane of the image
    • G06T3/14Transformations for image registration, e.g. adjusting or mapping for alignment of images
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00Geometric image transformations in the plane of the image
    • G06T3/40Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4038Image mosaicing, e.g. composing plane images from plane sub-images
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/50Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/30Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G06T7/337Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods involving reference images or patches
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/30Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/35Determination of transform parameters for the alignment of images, i.e. image registration using statistical methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/761Proximity, similarity or dissimilarity measures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/84Arrangements for image or video recognition or understanding using pattern recognition or machine learning using probabilistic graphical models from image or video features, e.g. Markov models or Bayesian networks
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84Systems specially adapted for particular applications
    • G01N2021/8466Investigation of vegetal material, e.g. leaves, plants, fruits
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00Indexing scheme for image data processing or generation, in general
    • G06T2200/32Indexing scheme for image data processing or generation, in general involving image mosaicing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10028Range image; Depth image; 3D point clouds
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20076Probabilistic image processing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20172Image enhancement details
    • G06T2207/20192Edge enhancement; Edge preservation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20212Image combination
    • G06T2207/20221Image fusion; Image merging
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30181Earth observation
    • G06T2207/30188Vegetation; Agriculture
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02ATECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A90/00Technologies having an indirect contribution to adaptation to climate change
    • Y02A90/30Assessment of water resources

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Software Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Chemical & Material Sciences (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Probability & Statistics with Applications (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Analysis (AREA)
  • Computing Systems (AREA)
  • Computational Mathematics (AREA)
  • Mathematical Optimization (AREA)
  • Pathology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Pure & Applied Mathematics (AREA)
  • Immunology (AREA)
  • Mathematical Physics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Biology (AREA)
  • Biochemistry (AREA)
  • Multimedia (AREA)
  • Analytical Chemistry (AREA)
  • Computer Hardware Design (AREA)
  • Environmental Sciences (AREA)
  • Geometry (AREA)
  • Algebra (AREA)
  • Operations Research (AREA)
  • Chemical Kinetics & Catalysis (AREA)
  • Electrochemistry (AREA)
  • Image Processing (AREA)

Abstract

The application provides a method for establishing an optimal digging pose of pseudo-ginseng, which comprises the following steps: generating an above-ground stem and leaf part point cloud image, establishing an underground rhizome part detection image, reconstructing the image and acquiring an optimal pose; the method comprises the steps that point cloud images of the above-ground stem and leaf parts are obtained by splicing point clouds of depth images, detection images of the below-ground stem and leaf parts are converted through electric signals of an electric impedance sensor, the point cloud images and the detection images are fused through image reconstruction, and the optimal pose is obtained through building an excavation function model. The method can obtain the optimal position of Notoginseng radix during the process of digging into soil.

Description

Establishment method of optimal digging pose of pseudo-ginseng
Technical Field
The application relates to the technical field of pseudo-ginseng excavation, in particular to a method for establishing an optimal pseudo-ginseng digging pose.
Background
Notoginseng radix, also called Notoginseng radix, has effects of removing blood stasis, stopping bleeding, promoting blood circulation and relieving pain, and has high edible and medicinal value of its leaves, stems and rhizomes; along with the increasing significance of the economic value and social benefit of the pseudo-ginseng, the planting area of the pseudo-ginseng is enlarged year by year. At present, the harvesting of the pseudo-ginseng is mainly performed by manual excavation and auxiliary mechanical excavation: the manual excavation requires larger labor force, so that the labor force and the material resources are greatly wasted, the excavation cost is increased, the excavation efficiency is low, the excavation time is long, and the loss of pseudo-ginseng is easy to increase; although the auxiliary mechanical excavation can obviously improve the excavation efficiency of the pseudo-ginseng, save the excavation time and effectively reduce the excavation cost and the labor amount, the root stem parts of the pseudo-ginseng are positioned below the ground, and the root stems of each plant of pseudo-ginseng are different (namely the shape of the root stem parts, the depth extending into the soil layer and the like), the position, the shape and the like of the root stems of the pseudo-ginseng below the ground are difficult to accurately judge by the auxiliary mechanical excavation, and the position, the shape and the like of the root stems of the pseudo-ginseng cannot be adjusted according to the different shapes and the soil depths of the root stems of each plant of pseudo-ginseng, so that the root stems of the pseudo-ginseng are easily damaged in the excavation process, the root damage rate is high, and the edible and medicinal values of the pseudo-ginseng are further influenced.
Disclosure of Invention
Aiming at the problems existing in the prior art, the application aims to provide a method for establishing the optimal digging pose of pseudo-ginseng, which is characterized in that the method is used for respectively acquiring the point cloud image of the aerial stem and leaf part of pseudo-ginseng and the detection image of the lower rhizome part of pseudo-ginseng, and acquiring the reconstructed image of root-earth-channel of pseudo-ginseng through the fusion of the point cloud image of the aerial stem and leaf part of pseudo-ginseng and the detection image of the lower rhizome part of pseudo-ginseng, so that the optimal position and pose of the digging of pseudo-ginseng are obtained, and the problems of damage to pseudo-ginseng, high root injury rate and the like in the digging process are avoided.
The aim of the application is achieved by the following technical scheme:
a method for establishing an optimal digging pose of pseudo-ginseng is characterized by comprising the following steps: comprising the following steps: generating an above-ground stem and leaf part point cloud image, establishing an underground rhizome part detection image, reconstructing the image and acquiring an optimal pose;
the generation of the above-ground stem and leaf part point cloud image specifically comprises the following steps: firstly, taking images of stem leaves (namely aerial parts) of pseudo-ginseng from opposite sides by adopting two groups of binocular stereo vision cameras to obtain two groups of depth images; then, an improved iterative closest point (Iterative Closest Point, ICP) algorithm is adopted to splice point clouds in the two groups of depth images, and an overground stem leaf part point cloud image is generated;
the underground rhizome part detection image is established specifically as follows: according to an electrical impedance detection method, a plurality of electrical impedance sensors are inserted in the root and stem parts (namely underground parts) of pseudo-ginseng in a ring mode, an improved modified Newton Laplasen (Modified Newton Rapson, MNR) algorithm is adopted, electrical signals acquired by the electrical impedance sensors are reconstructed into image signals, and an underground root and stem part detection image is built;
the image reconstruction specifically comprises the following steps: taking the plane of the minimum coordinate of the root part bottom as the soil surface, and fusing the above-ground stem and leaf part point cloud image with the underground root part detection image to obtain a pseudo-ginseng root-soil-stem reconstructed image;
the optimal pose acquisition specifically comprises the following steps: the method comprises the steps of (1) taking the soil entering angle and edge point coordinates of a digging shovel as independent variables, taking the consumption energy of the digging shovel as dependent variables, and establishing a digging function model; and obtaining independent variables (namely the soil entering angle and the edge point coordinates of the digging shovel) which minimize the dependent variables (namely the energy consumed by the digging shovel) through a digging function model, so as to obtain the optimal pose of the soil entering digging.
As a preferable scheme of the application, the step of splicing the point clouds in the two groups of depth images by adopting the improved iterative nearest point algorithm comprises the following steps:
s01, extracting feature points: for two point clouds to be registered, firstly extracting some characteristic points from the point clouds to be registered, and taking the characteristic points as a matching basis, wherein the characteristic points are extracted specifically as follows:
wherein: p represents a set of all points in the point cloud;or->Representing a certain point p i Or p j Is a feature vector of (1);or->Represents p i Or p j Feature scores for points; />Representing the selected set of points with salient features;
s02, selecting dense point pairs: selecting dense point pairs from the characteristic point set for matching when carrying out point cloud registration; a distance-based neighborhood search method is generally used to obtain dense point pairs, that is, the neighborhood range of each point is obtained by the following method assuming that the mean variances of the distances between corresponding point pairs in two sets of point clouds are equal:
N i =j|d(p i ,q j )≤r;
wherein: d (p) i ,q j ) Representing point p i And q j A Euclidean distance between them; r represents a neighborhood search range; n (N) i Representation and point p i A set of adjacent points;
s03, weighted average error: after registering dense pairs of points, minimizing the distance error between these pairs of points, the registration effect is measured using a weighted average error, as follows:
wherein: n represents the number of pairs of points; p is p i 、q i ' represents the corresponding point pairs in the two point clouds to be registered, respectively; d (p) i ,q i ' indicates p i And q i ' Euclidean distance between; w (w) i Representing a weight coefficient for adjusting the contribution of each point pair;
s04, least square optimization: obtaining a rigid transformation between the two sets of point clouds by minimizing a weighted average error; by using a 4x4 matrix to represent the rigid transformation of two groups of point clouds, the method is as follows:
wherein: r represents a rotation matrix and t represents a translation vector, which are obtained by the following stepwise iterative formula:
wherein: and (3) carrying out multiple iterations in the solving process until the error approaches 0.
As a preferred scheme of the present application, the "reconstructing the electrical signal acquired by the electrical impedance sensor into the image signal by using the modified newton-rapson algorithm" specifically includes:
s11, data acquisition: dividing the surface of an object to be detected (namely the surface of the pseudo-ginseng lower rhizome part) into a plurality of separated areas (electrode pairs) by adopting an electrical impedance imaging technology, and collecting voltage values, namely electrical impedance measured values, from the interfaces of each pair of electrodes;
s12, data preprocessing: preprocessing the data acquired in the step S11;
s13, solving the distributed impedance: firstly, reconstructing the electrical impedance data preprocessed in the step S12 by using a film method; the electrical impedance is then characterized as a function of the conductivity, geometry and electrode position of the medium by solving ohm's law, expressed as a function of the resistivity of the medium by ohm's method; then using a nonlinear optimization algorithm to iteratively solve the distributed impedance;
s14, image reconstruction: image reconstruction is carried out by adopting an improved MNR algorithm;
s15, data post-processing: and (5) performing post-processing operation on the image signals in the step (S14) to obtain an underground rhizome part detection image.
As a preferred embodiment of the present application, the data preprocessing in step S12 includes filtering and denoising, so as to improve the signal-to-noise ratio.
As a preferred embodiment of the present application, the step S13 specifically includes:
and (3) reconstructing the electrical impedance data preprocessed in the step (S12) by using a film method, wherein the method specifically comprises the following steps:
V=HZ;
wherein: h represents the total energy guiding matrix, Z represents the electrical impedance vector to, V represents the potential distribution vector; the matrix H and the vector V are solved by a thin film method, so that an electric potential distribution image is obtained;
the electrical impedance is then characterized by solving ohm's law as a function of the medium conductivity, geometry and electrode position, in particular:
wherein: σ represents the electrical conductivity and,representing the potential field distribution;
it is expressed by ohmic methods as a function of the resistivity of the medium, in particular:
AR=b;
wherein: r represents a resistivity distribution vector, A represents a measurement matrix, and b represents a current distribution vector;
and then using a nonlinear optimization algorithm to iteratively solve the distributed impedance, wherein the method specifically comprises the following steps:
wherein: Φ represents the objective function, λ represents the canonical optimization parameter, and c represents the morphological regularization operator.
As a preferred embodiment of the present application, the step S14 specifically includes:
s141, initializing data, setting the input test data as y and the initial image as x 0 The iteration times are t, and the punishment parameters are e;
s142, constructing derivative operator D by using Sobel operator x And D y The gradient information is used for calculating the image;
s143, for each pixel point, acquiring a weight matrix W of the pixel point, wherein W i,j Representing weights at reconstructed pixel points (i, j); according to a local weighting strategy, a weight value is determined by the distance between the pixel point and surrounding pixel points and the distance between the measured data;
s144, image x i Decomposition into multiple dimensions x i (1) ,x i (2) ,…,x i (s) And calculateGradient information G for each scale image i (1) ,G i (2) ,…,G i (s)
S145, carrying out gradient calculation on each scale image:
s146, balancing smoothing effect and reserving edge information by weighting the image gradient information;
s147, updating the image x by minimizing the cost function i
Wherein: i k Measurement data representing a kth scale;
s148, repeating the steps S144-S147 until the preset iteration times are reached or convergence conditions are met.
As a preferred embodiment of the present application, the post-processing of data in step S15 includes artifact removal, contrast enhancement, and the like.
As a preferable scheme of the application, the process of fusing the above-ground stem and leaf part point cloud image and the underground rhizome part detection image to obtain the pseudo-ginseng root-soil-stem reconstructed image is specifically as follows:
firstly, the above-ground stem and leaf part point cloud image and the underground rhizome part detection image are respectively regarded as random variables, and fusion is realized by carrying out joint conditional probability distribution on the above-ground stem and leaf part point cloud image and the underground rhizome part detection image, specifically:
P(X|Y)P(Y)=P(Y|X)P(X);
wherein: x represents an above-ground stem and leaf part point cloud image, and Y represents an below-ground rhizome part detection image; p (x|y) represents the probability distribution of X given Y; p (y|x) represents the probability distribution of Y given X; p (X) and P (Y) each represent a priori probability distribution of X, Y;
then, a Bayesian filtering algorithm is utilized to estimate posterior probability distribution of the above-ground stem and leaf part point cloud image, namely:
and (3) convolving the P (Y|X) and the P (X) to obtain a post probability distribution of the P (X|Y), and further obtaining a pseudo-ginseng root-soil-stem reconstructed image after fusion.
As a preferable scheme of the application, the method for establishing the excavation function model by taking the coordinates of the soil entering angle and the edge point of the excavation shovel as independent variables and the consumption energy of the excavation shovel as dependent variables comprises the following specific steps:
firstly, setting the soil entering angle of a digging shovel as u, the coordinates of edge points as (v, w), and the energy consumption of the digging shovel as F, wherein the digging function model is as follows:
F(u,v,w)=β 01 ·u+β 2 ·v+β 3 ·w;
wherein: beta 0 、β 1 、β 2 、β 3 Respectively representing multiple regression coefficients;
wherein:
wherein: a, a 0 、a 1 、a 2 、a 3 Representing fitting plane coefficients; (u) 0 ,v 0 ,w 0 ) Representing the coordinates of a point on the plane.
As a preferable scheme of the present application, the "obtaining the optimal pose of the soil excavation by obtaining the independent variable minimizing the dependent variable through the excavation function model" specifically includes:
s21: randomly initializing values of u, v and w;
s22: the gradient of the mining function model is calculated respectively, specifically:
wherein: x is x i 、y i 、z i Values of corresponding input variables U, v, w are respectively represented, U representing corresponding output variable F ();
s23: the parameter updating is carried out, specifically:
wherein: lambda represents the learning rate, obtained from experimental data;
s24: and repeating the steps S22 to S23 until the convergence condition is reached, outputting u, v and w as optimal solutions when the convergence is carried out, and obtaining the optimal digging pose when the digging is carried out.
The application has the following technical effects:
according to the application, two groups of binocular stereo vision cameras are utilized to acquire images of the above-ground stem and leaf parts of the pseudo-ginseng, an electrical impedance sensor is utilized to construct images of the underground rhizome parts of the pseudo-ginseng, and the images are fused, so that the complete images of the pseudo-ginseng are obtained, and the problem that the root and stem parts of the pseudo-ginseng are damaged by excavation caused by the fact that the images below the ground of the pseudo-ginseng cannot be obtained is effectively avoided. When the aerial stem leaf part image of pseudo-ginseng is acquired, an improved iterative nearest point algorithm is utilized, firstly, a matching result is more accurate and can reach a convergence state quickly, secondly, the number of data points is reduced by screening sparse point pairs and dense point pairs, the calculation speed is further improved, and therefore a more accurate stem leaf part point cloud image is obtained quickly; in the process of establishing the pseudo-ginseng rhizome part image, the improved modified Newton Laplasen algorithm is utilized to reduce redundant calculation, noise and artifacts, so that a clearer image signal is obtained, the accuracy of establishing the image is ensured, the damage of the pseudo-ginseng rhizome part in the digging process caused by inaccuracy is avoided, and meanwhile, the method has stronger robustness, stability and adaptability. In addition, the application also utilizes the establishment of the excavation function model to obtain the optimal digging pose through repeated iteration, so that firstly, the damage to pseudo-ginseng in the excavating process is effectively avoided, and secondly, the energy loss in the excavating process is ensured to be small, thereby saving the excavating time, improving the excavating efficiency and reducing the excavating cost.
Drawings
Fig. 1 is a flowchart of a method for establishing an optimal digging pose in an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments.
Examples:
as shown in fig. 1, a method for establishing an optimal digging pose of pseudo-ginseng is characterized in that: comprising the following steps: generating an above-ground stem and leaf part point cloud image, establishing an underground rhizome part detection image, reconstructing the image and acquiring an optimal pose;
the generation of the above-ground stem and leaf part point cloud image specifically comprises the following steps: firstly, two groups of binocular stereo vision cameras (the binocular stereo vision cameras are only of the type common to the market in the field, and the embodiment does not have excessive limitation) are adopted to shoot images of stem leaves (namely aerial parts) of pseudo-ginseng from opposite sides, so that two groups of depth images are obtained;
then, the point clouds in the two groups of depth images are spliced by adopting an improved iterative closest point (Iterative Closest Point, ICP) algorithm to generate an aerial stem and leaf part point cloud image, which specifically comprises the following steps:
s01, extracting feature points: for two point clouds to be registered, firstly extracting some characteristic points from the point clouds to be registered, and taking the characteristic points as a matching basis, wherein the characteristic points are extracted specifically as follows:
wherein: p represents all of the point cloudsA collection of points;or->Representing a certain point p i Or p j Is a feature vector of (1);or->Represents p i Or p j Feature scores for points; />Representing the selected set of points with salient features;
s02, selecting dense point pairs: selecting dense point pairs from the characteristic point set for matching when carrying out point cloud registration; a distance-based neighborhood search method is generally used to obtain dense point pairs, that is, the neighborhood range of each point is obtained by the following method assuming that the mean variances of the distances between corresponding point pairs in two sets of point clouds are equal:
N i =j|d(p i ,q j )≤r;
wherein: d (p) i ,q j ) Representing point p i And q j A Euclidean distance between them; r represents a neighborhood search range; n (N) i Representation and point p i A set of adjacent points;
s03, weighted average error: after registering dense pairs of points, minimizing the distance error between these pairs of points, the registration effect is measured using a weighted average error, as follows:
wherein: n represents the number of pairs of points; p is p i 、q i ' represents two point clouds to be registered separatelyCorresponding pairs of points in (a); d (p) i ,q i ' indicates p i And q i ' Euclidean distance between; w (w) i Representing weight coefficients, for adjusting the contribution of each point pair, weight coefficients commonly used in the art include distance weight coefficients and normal vector weight coefficients, etc.;
s04, least square optimization: obtaining a rigid transformation between the two sets of point clouds by minimizing a weighted average error; by using a 4x4 matrix to represent the rigid transformation of two groups of point clouds, the method is as follows:
wherein: r represents a rotation matrix and t represents a translation vector, which are obtained by the following stepwise iterative formula:
wherein: and (3) carrying out multiple iterations in the solving process until the error approaches 0.
The underground rhizome part detection image is established specifically as follows: according to the electrical impedance detection method, a plurality of electrical impedance sensors are inserted in the ring of the root stem (namely the underground part) of the pseudo-ginseng (the electrical impedance detection method can be detected by adopting a Sciospec EIT16 electrical impedance imaging system which is specially designed for electrical impedance tomography or multiport impedance measurement application, key parameters are that the system is provided with 16 double-acting electrode connections which are used as voltage measurement and current injection ports, a synchronous data acquisition block can simultaneously sample all sixteen channels, the frequency range is 100 Hz-1 MHz, spectral measurement can be carried out through frequency scanning, 128 frequencies are supported at most, the current excitation range is 100 nA-10 mA, the frame rate is 100fps, the ring insertion is that the plurality of electrical impedance sensors take the central line of the root stem of the pseudo-ginseng as an axis and are distributed on the outer ring of the root stem of the pseudo-ginseng in a ring shape uniformly, the electrical impedance sensors can be fixed and positioned by arranging a fixed seat, an improved correction Newton Laprenon (Modified Newton Rapson, MNR) algorithm is adopted to reconstruct electrical signals acquired by the electrical impedance sensors into image signals, and an underground root part detection image is established, and the method specifically comprises:
s11, data acquisition: dividing the surface of an object to be detected (namely the surface of the pseudo-ginseng lower rhizome part) into a plurality of separated areas (electrode pairs) by adopting an electrical impedance imaging technology, and collecting voltage values, namely electrical impedance measured values, from the interfaces of each pair of electrodes;
s12, data preprocessing: preprocessing the data obtained in the step S11, including filtering, denoising and the like (it is to be noted that the filtering, denoising and the like are all conventional means in the field and only meet the purpose of the application), so as to improve the signal-to-noise ratio;
s13, solving the distributed impedance: firstly, reconstructing the electrical impedance data preprocessed in the step S12 by using a film method, specifically:
V=HZ;
wherein: h represents the total energy guiding matrix, Z represents the electrical impedance vector to, V represents the potential distribution vector; the matrix H and the vector V are solved by a thin film method, so that an electric potential distribution image is obtained;
the electrical impedance is then characterized by solving ohm's law as a function of the medium conductivity, geometry and electrode position, in particular:
wherein: σ represents the electrical conductivity and,representing the potential field distribution;
it is expressed as a function of the resistivity of the medium by the ohmic method, which is one of the key algorithms for solving the electrical impedance imaging problem, which can be expressed as a function of the resistivity of the medium, in particular:
AR=b;
wherein: r represents a resistivity distribution vector, A represents a measurement matrix, and b represents a current distribution vector;
and then using a nonlinear optimization algorithm to iteratively solve the distributed impedance, wherein the method specifically comprises the following steps:
wherein: Φ represents the objective function, λ represents the canonical optimization parameter, and c represents the morphological regularization operator.
S14, image reconstruction: image reconstruction is carried out by adopting an improved MNR algorithm, and the method specifically comprises the following steps:
s141, initializing data, setting the input test data as y and the initial image as x 0 The iteration times are t, and the punishment parameters are e;
s142, constructing derivative operator D by using Sobel operator x And D y The gradient information is used for calculating the image;
s143, for each pixel point, acquiring a weight matrix W of the pixel point, wherein W i,j Representing weights at reconstructed pixel points (i, j); according to a local weighting strategy, a weight value is determined by the distance between the pixel point and surrounding pixel points and the distance between the measured data; the weight matrix W is an n×n matrix, where n is the size of the image;
s144, image x i Decomposition into multiple dimensions x i (1) ,x i (2) ,…,x i (s) And calculates gradient information G of each scale image i (1) ,G i (2) ,…,G i (s)
S145, carrying out gradient calculation on each scale image:
s146, balancing smoothing effect and reserving edge information by weighting the image gradient information;
s147, updating the image x by minimizing the cost function i
Wherein: i k Measurement data representing a kth scale;
s148, repeating the steps S144-S147 until the preset iteration times are reached or convergence conditions are met;
s15, data post-processing: and (2) performing post-processing operations on the image signals in the step (S14) to remove artifacts, enhance contrast and the like (it is to be noted that the processes of removing the artifacts, enhancing the contrast and the like all adopt conventional means in the field, and only the aim of the application is required to be met), so as to obtain the underground rhizome part detection image.
The image reconstruction is specifically as follows: the method comprises the steps of taking a plane where the minimum coordinates of the root bottom are located as a soil surface, fusing an above-ground stem and leaf part point cloud image with an underground root part detection image to obtain a pseudo-ginseng root-soil-stem reconstructed image, and specifically comprises the following steps:
firstly, the above-ground stem and leaf part point cloud image and the underground rhizome part detection image are respectively regarded as random variables, and fusion is realized by carrying out joint conditional probability distribution on the above-ground stem and leaf part point cloud image and the underground rhizome part detection image, specifically:
P(X|Y)P(Y)=P(Y|X)P(X);
wherein: x represents an above-ground stem and leaf part point cloud image, and Y represents an below-ground rhizome part detection image; p (x|y) represents the probability distribution of X given Y; p (y|x) represents the probability distribution of Y given X; p (X) and P (Y) each represent a priori probability distribution of X, Y;
then, a Bayesian filtering algorithm is utilized to estimate posterior probability distribution of the above-ground stem and leaf part point cloud image, namely:
wherein, P (X) and P (Y) respectively obtain probability distribution by extracting characteristic vectors of the above-ground stem and leaf part point cloud image and the below-ground rhizome part detection image and utilizing the characteristic vectors; p (Y|X) is realized by comparing the characteristic vectors of the above-ground stem and leaf part point cloud image and the below-ground rhizome part detection image, namely, a similarity function is defined to measure the similarity (such as Euclidean distance function) between the above-ground stem and leaf part point cloud image and the below-ground rhizome part detection image, so that P (Y|X) is obtained;
and (3) convolving the P (Y|X) and the P (X) to obtain a post probability distribution of the P (X|Y), and further obtaining a pseudo-ginseng root-soil-stem reconstructed image after fusion.
It should be noted that: the specific implementation mode and parameters affect the fusion effect, so that algorithm parameters need to be adjusted according to specific conditions and are properly optimized to achieve the optimal fusion effect.
The optimal pose is obtained specifically as follows: the method comprises the steps of taking the soil entering angle and the edge point coordinates of a digging shovel as independent variables, taking the consumption energy of the digging shovel as dependent variables, and establishing a digging function model, wherein the method specifically comprises the following steps:
firstly, setting the soil entering angle of a digging shovel as u, the coordinates of edge points as (v, w), and the energy consumption of the digging shovel as F, wherein the digging function model is as follows:
F(u,v,w)=β 01 ·u+β 2 ·v+β 3 ·w;
wherein: beta 0 、β 1 、β 2 、β 3 Respectively representing multiple regression coefficients;
wherein:
wherein: a, a 0 、a 1 、a 2 、a 3 Representation ofFitting a plane coefficient; (u) 0 ,v 0 ,w 0 ) Representing the coordinates of a point on the plane.
Then, obtaining an independent variable (namely, the soil entering angle and the edge point coordinates of the excavating shovel) which minimizes the dependent variable (namely, the energy consumed by the excavating shovel) through an excavating function model, so as to obtain the optimal pose of soil entering excavation, which is specifically as follows:
s21: randomly initializing values of u, v and w;
s22: the gradient of the mining function model is calculated respectively, specifically:
wherein: x is x i 、y i 、z i Values of corresponding input variables U, v, w are respectively represented, U representing corresponding output variable F ();
s23: the parameter updating is carried out, specifically:
wherein: lambda represents the learning rate, obtained from experimental data;
s24: and repeating the steps S22-S23 until the convergence condition is reached, outputting the optimal solutions u, v and w during convergence, and obtaining the optimal digging pose during digging, namely using the optimal solutions as the digging positions and the attitudes, so that the energy consumed by digging can be minimized.

Claims (7)

1. A method for establishing an optimal digging pose of pseudo-ginseng is characterized by comprising the following steps: comprising the following steps: generating an above-ground stem and leaf part point cloud image, establishing an underground rhizome part detection image, reconstructing the image and acquiring an optimal pose;
the generation of the above-ground stem and leaf part point cloud image specifically comprises the following steps: firstly, shooting stem and leaf images of pseudo-ginseng from opposite sides by adopting two groups of binocular stereo vision cameras to obtain two groups of depth images; then, an improved iterative nearest point algorithm is adopted to splice point clouds in the two groups of depth images, and an overground stem leaf part point cloud image is generated;
the underground rhizome part detection image is established specifically as follows: according to an electrical impedance detection method, a plurality of electrical impedance sensors are inserted in the root and stem part of pseudo-ginseng in a ring mode, an improved modified Newton Laplasen algorithm is adopted, electrical signals acquired by the electrical impedance sensors are reconstructed into image signals, and an underground rhizome part detection image is established;
the image reconstruction specifically comprises the following steps: taking the plane of the minimum coordinate of the root part bottom as the soil surface, and fusing the above-ground stem and leaf part point cloud image with the underground root part detection image to obtain a pseudo-ginseng root-soil-stem reconstructed image;
the optimal pose acquisition specifically comprises the following steps: the method comprises the steps of (1) taking the soil entering angle and edge point coordinates of a digging shovel as independent variables, taking the consumption energy of the digging shovel as dependent variables, and establishing a digging function model; and obtaining an independent variable which minimizes the dependent variable through the excavation function model, and obtaining the optimal pose of the soil excavation.
2. The method for establishing the optimal digging pose of pseudo-ginseng according to claim 1, wherein the method comprises the following steps: the step of splicing the point clouds in the two groups of depth images by adopting the improved iterative closest point algorithm comprises the following steps:
s01, extracting feature points: for two point clouds to be registered, firstly extracting some characteristic points from the point clouds to be registered, and taking the characteristic points as a matching basis, wherein the characteristic points are extracted specifically as follows:
wherein: p represents a set of all points in the point cloud;or->Representing a certain point p i Or p j Is a feature vector of (1); />Or (b)Represents p i Or p j Feature scores for points; />Representing the selected set of points with salient features;
s02, selecting dense point pairs: selecting dense point pairs from the characteristic point set for matching when carrying out point cloud registration; a distance-based neighborhood search method is generally used to obtain dense point pairs, that is, the neighborhood range of each point is obtained by the following method assuming that the mean variances of the distances between corresponding point pairs in two sets of point clouds are equal:
N i =j|d(p i ,q j )≤r;
wherein: d (p) i ,q j ) Representing point p i And q j A Euclidean distance between them; r represents a neighborhood search range; n (N) i Representation and point p i A set of adjacent points;
s03, weighted average error: after registering dense pairs of points, minimizing the distance error between these pairs of points, the registration effect is measured using a weighted average error, as follows:
wherein: n represents the number of pairs of points; p is p i 、q i ' represents the corresponding point pairs in the two point clouds to be registered, respectively; d (p) i ,q i ' indicates p i And q i ' Euclidean distance between; w (w) i Representing a weight coefficient for adjusting the contribution of each point pair;
s04, least square optimization: obtaining a rigid transformation between the two sets of point clouds by minimizing a weighted average error; by using a 4x4 matrix to represent the rigid transformation of two groups of point clouds, the method is as follows:
wherein: r represents a rotation matrix and t represents a translation vector, which are obtained by the following stepwise iterative formula:
wherein: and (3) carrying out multiple iterations in the solving process until the error approaches 0.
3. The method for establishing the optimal digging pose of pseudo-ginseng according to claim 1 or 2, wherein the method comprises the following steps: the method for reconstructing the electric signal acquired by the electrical impedance sensor into the image signal by adopting the improved modified Newton Laplasen algorithm comprises the following specific steps:
s11, data acquisition: dividing the surface of an object to be measured into a plurality of separated areas by adopting an electrical impedance imaging technology, and collecting voltage values, namely electrical impedance measured values, from interfaces of each pair of electrodes;
s12, data preprocessing: preprocessing the data acquired in the step S11;
s13, solving the distributed impedance: firstly, reconstructing the electrical impedance data preprocessed in the step S12 by using a film method; the electrical impedance is then characterized as a function of the conductivity, geometry and electrode position of the medium by solving ohm's law, expressed as a function of the resistivity of the medium by ohm's method; then using a nonlinear optimization algorithm to iteratively solve the distributed impedance;
s14, image reconstruction: image reconstruction is carried out by adopting an improved MNR algorithm;
s15, data post-processing: and (5) performing post-processing operation on the image signals in the step (S14) to obtain an underground rhizome part detection image.
4. A method for establishing an optimal digging pose of pseudo-ginseng according to claim 1 or 3, wherein: the step S13 specifically includes:
and (3) reconstructing the electrical impedance data preprocessed in the step (S12) by using a film method, wherein the method specifically comprises the following steps:
V=HZ;
wherein: h represents the total energy guiding matrix, Z represents the electrical impedance vector to, V represents the potential distribution vector; the matrix H and the vector V are solved by a thin film method, so that an electric potential distribution image is obtained;
the electrical impedance is then characterized by solving ohm's law as a function of the medium conductivity, geometry and electrode position, in particular:
wherein: σ represents the electrical conductivity and,representing the potential field distribution;
it is expressed by ohmic methods as a function of the resistivity of the medium, in particular:
AR=b;
wherein: r represents a resistivity distribution vector, A represents a measurement matrix, and b represents a current distribution vector;
and then using a nonlinear optimization algorithm to iteratively solve the distributed impedance, wherein the method specifically comprises the following steps:
wherein: Φ represents the objective function, λ represents the canonical optimization parameter, and c represents the morphological regularization operator.
5. A method for establishing an optimal digging pose of pseudo-ginseng according to claim 1 or 3, wherein: the step S14 specifically includes:
s141, initializing data, setting the input test data as y and the initial image as x 0 The iteration times are t, and the punishment parameters are e;
s142, constructing derivative operator D by using Sobel operator x And D y The gradient information is used for calculating the image;
s143, for each pixel point, acquiring a weight matrix W of the pixel point, wherein W i,j Representing weights at reconstructed pixel points (i, j); according to a local weighting strategy, a weight value is determined by the distance between the pixel point and surrounding pixel points and the distance between the measured data;
s144, image x i Decomposition into multiple dimensions x i (1) ,x i (2) ,…,x i (s) And calculates gradient information G of each scale image i (1) ,G i (2) ,…,G i (s)
S145, carrying out gradient calculation on each scale image:
s146, balancing smoothing effect and reserving edge information by weighting the image gradient information;
s147, updating the image x by minimizing the cost function i
Wherein: i k Measurement data representing a kth scale;
s148, repeating the steps S144-S147 until the preset iteration times are reached or convergence conditions are met.
6. The method for establishing the optimal digging pose of pseudo-ginseng according to claim 3, wherein the method comprises the following steps: the method for building the excavation function model by taking the soil entering angle and the edge point coordinates of the excavation shovel as independent variables and the consumption energy of the excavation shovel as dependent variables comprises the following specific steps:
firstly, setting the soil entering angle of a digging shovel as u, the coordinates of edge points as (v, w), and the energy consumption of the digging shovel as F, wherein the digging function model is as follows:
F(u,v,w)=β 01 ·u+β 2 ·v+β 3 ·w;
wherein: beta 0 、β 1 、β 2 、β 3 Respectively representing multiple regression coefficients;
wherein:
wherein: a, a 0 、a 1 、a 2 、a 3 Representing fitting plane coefficients; (u) 0 ,v 0 ,w 0 ) Representing the coordinates of a point on the plane.
7. The method for establishing the optimal digging pose of pseudo-ginseng according to claim 6, wherein the method comprises the following steps: the 'obtaining the optimal pose of the excavation into the soil by obtaining the independent variable which minimizes the dependent variable through the excavation function model' specifically comprises the following steps:
s21: randomly initializing values of u, v and w;
s22: the gradient of the mining function model is calculated respectively, specifically:
wherein: x is x i 、y i 、z i Values of corresponding input variables U, v, w are respectively represented, U representing corresponding output variable F ();
s23: the parameter updating is carried out, specifically:
wherein: lambda represents the learning rate, obtained from experimental data;
s24: and repeating the steps S22 to S23 until the convergence condition is reached, outputting u, v and w as optimal solutions when the convergence is carried out, and obtaining the optimal digging pose when the digging is carried out.
CN202310621573.5A 2023-05-30 2023-05-30 Establishment method of optimal digging pose of pseudo-ginseng Active CN116663408B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310621573.5A CN116663408B (en) 2023-05-30 2023-05-30 Establishment method of optimal digging pose of pseudo-ginseng

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310621573.5A CN116663408B (en) 2023-05-30 2023-05-30 Establishment method of optimal digging pose of pseudo-ginseng

Publications (2)

Publication Number Publication Date
CN116663408A true CN116663408A (en) 2023-08-29
CN116663408B CN116663408B (en) 2023-12-22

Family

ID=87727269

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310621573.5A Active CN116663408B (en) 2023-05-30 2023-05-30 Establishment method of optimal digging pose of pseudo-ginseng

Country Status (1)

Country Link
CN (1) CN116663408B (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105115976A (en) * 2015-06-24 2015-12-02 上海图甲信息科技有限公司 Detection system and method for wear defects of track
CN109345620A (en) * 2018-08-13 2019-02-15 浙江大学 Merge the improvement ICP object under test point cloud method of quick point feature histogram
CN113112588A (en) * 2021-04-07 2021-07-13 中国电建集团贵州电力设计研究院有限公司 Underground pipe well three-dimensional visualization method based on RGB-D depth camera reconstruction
CN114219912A (en) * 2021-11-22 2022-03-22 北京林业大学 Method for automatically reconstructing tree structure from foundation laser radar point cloud

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105115976A (en) * 2015-06-24 2015-12-02 上海图甲信息科技有限公司 Detection system and method for wear defects of track
CN109345620A (en) * 2018-08-13 2019-02-15 浙江大学 Merge the improvement ICP object under test point cloud method of quick point feature histogram
CN113112588A (en) * 2021-04-07 2021-07-13 中国电建集团贵州电力设计研究院有限公司 Underground pipe well three-dimensional visualization method based on RGB-D depth camera reconstruction
CN114219912A (en) * 2021-11-22 2022-03-22 北京林业大学 Method for automatically reconstructing tree structure from foundation laser radar point cloud

Also Published As

Publication number Publication date
CN116663408B (en) 2023-12-22

Similar Documents

Publication Publication Date Title
CN109934855A (en) A kind of livewire work scene power components three-dimensional rebuilding method based on cloud
CN105719278B (en) A kind of medical image cutting method based on statistics deformation model
CN107154038B (en) Rib fracture auxiliary diagnosis method based on rib visualization
CN111339989B (en) Water body extraction method, device, equipment and storage medium
CN102831614B (en) Sequential medical image quick segmentation method based on interactive dictionary migration
CN107481274A (en) A kind of three-dimensional makees the robustness reconstructing method of object point cloud
CN110136119A (en) A kind of lung based on deep learning splits the method and system of segmentation and integrity assessment
CN111047559B (en) Method for rapidly detecting abnormal area of digital pathological section
CN112288859B (en) Three-dimensional face modeling method based on convolutional neural network
CN116580068B (en) Multi-mode medical registration method based on point cloud registration
CN111797744A (en) Multimode remote sensing image matching method based on co-occurrence filtering algorithm
CN108230223A (en) Light field angle super-resolution rate method and device based on convolutional neural networks
CN116342444B (en) Dual-channel multi-mode image fusion method and electronic equipment
CN114119987A (en) Feature extraction and descriptor generation method and system based on convolutional neural network
CN116663408B (en) Establishment method of optimal digging pose of pseudo-ginseng
CN109544565B (en) Wound surface image segmentation method based on DC-GMRF level set
CN113538560A (en) Leaf area index extraction method based on three-dimensional reconstruction
CN116703018B (en) Path control method for precisely digging main root of pseudo-ginseng
CN116630217B (en) Interactive sensing type digging method for low-damage radix notoginseng main roots
CN117173437A (en) Multi-mode remote sensing image hybrid matching method and system with multi-dimensional directional self-similar characteristics
CN114596218B (en) Ancient painting image restoration method, model and device based on convolutional neural network
CN116663761B (en) Pseudo-ginseng chinese-medicinal material low-loss excavation system
Roy et al. MDL-IWS: multi-view deep learning with iterative watershed for pulmonary fissure segmentation
Yang et al. Depth super-resolution via fully edge-augmented guidance
Xie et al. Joint reconstruction and calibration using regularization by denoising

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant