CN115035415A - Garden street tree shaping, trimming and identifying method and system based on artificial intelligence - Google Patents
Garden street tree shaping, trimming and identifying method and system based on artificial intelligence Download PDFInfo
- Publication number
- CN115035415A CN115035415A CN202210952938.8A CN202210952938A CN115035415A CN 115035415 A CN115035415 A CN 115035415A CN 202210952938 A CN202210952938 A CN 202210952938A CN 115035415 A CN115035415 A CN 115035415A
- Authority
- CN
- China
- Prior art keywords
- street tree
- street
- tree
- graph
- obtaining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 57
- 238000013473 artificial intelligence Methods 0.000 title claims abstract description 27
- 238000009966 trimming Methods 0.000 title claims abstract description 13
- 238000007493 shaping process Methods 0.000 title claims abstract description 11
- 230000011218 segmentation Effects 0.000 claims abstract description 78
- 238000013138 pruning Methods 0.000 claims abstract description 33
- 238000010586 diagram Methods 0.000 claims abstract description 29
- 230000006870 function Effects 0.000 claims description 15
- 230000008569 process Effects 0.000 claims description 14
- 239000000126 substance Substances 0.000 claims description 12
- 238000012937 correction Methods 0.000 claims description 11
- 238000012545 processing Methods 0.000 claims description 7
- 230000008859 change Effects 0.000 claims description 5
- 238000001514 detection method Methods 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000012549 training Methods 0.000 description 5
- 238000004458 analytical method Methods 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 238000003708 edge detection Methods 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000011478 gradient descent method Methods 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 235000015097 nutrients Nutrition 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000002035 prolonged effect Effects 0.000 description 1
- 230000003716 rejuvenation Effects 0.000 description 1
- 241000894007 species Species 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/188—Vegetation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/56—Extraction of image or video features relating to colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Medical Informatics (AREA)
- Databases & Information Systems (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to a garden street tree shaping, trimming and identifying method and system based on artificial intelligence, which are used for acquiring RGB (red, green and blue) images of a street tree to obtain a street tree integral segmentation map; acquiring central axes of trunks of the street trees; obtaining a dividing line of crowns of two adjacent streets according to the overall street tree dividing graph; determining the shielding areas of two adjacent street trees according to the central axis, the dividing line and the edge information between the central axis and the dividing line; determining each occlusion contour information of adjacent street trees according to each occlusion area, and determining a single street tree segmentation graph and a corresponding dense degree distribution graph; meanwhile, obtaining the confidence of each street tree segmentation graph; calculating an average characteristic graph according to each confidence coefficient and the busyness degree distribution graph; subtracting the average characteristic diagram from the density distribution diagram to obtain each deviation diagram; calculating the average degree of the deviation graph, and pruning the current street tree when the average degree is greater than a set threshold value; the method and the device can accurately identify the pruning of the street tree.
Description
Technical Field
The invention relates to the field of artificial intelligence, in particular to a garden street tree shaping, trimming and identifying method and a garden street tree shaping, trimming and identifying system based on artificial intelligence.
Background
At present, greening is mainly built in public places such as gardens, residential areas, streets and the like, a correct pruning method is mastered in the pruning process of trees, and beautiful tree forms can be cultivated through reasonable pruning. The reasonable distribution of nutrient substances is further adjusted through pruning, the spindly growth is inhibited, the flower bud differentiation is promoted, the young trees can bloom and bear fruits early, the full-bloom period and the full-fruited period can be prolonged, and the old trees can be rejuvenated; meanwhile, the living conditions of people can be improved by the neatly-divided greening environment.
In particular, for trimming street trees, regular tree crowns are generally adopted, but due to different growth trends of the street trees, in order to ensure the attractiveness of the street trees and the normal traffic of vehicles and pedestrians, the street trees need to be trimmed irregularly. The street tree is a tree species which is used for shading vehicles and pedestrians and forming garden scenes on two sides of a road and a vehicle dividing zone. The street tree requires the extension of branches, the wide crown and the dense branches and leaves, and the crown shape is determined by overhead lines and traffic conditions of the planting place.
Therefore, how to determine whether the street tree needs to be pruned is a problem to be solved, and the street tree can be pruned conveniently and timely.
Disclosure of Invention
In order to solve the technical problems, the invention aims to provide a garden street tree shaping and trimming method and a garden street tree shaping and trimming system based on artificial intelligence, and the adopted technical scheme is as follows:
the invention provides a technical scheme of a garden street tree shaping, trimming and identifying method based on artificial intelligence, which comprises the following steps:
collecting RGB images of a street tree, and obtaining an integral street tree segmentation map by utilizing a semantic segmentation network;
carrying out image processing on the street tree integral segmentation graph to obtain integral border information of the street trees and obtain a central shaft of a trunk of each street tree, wherein the central shaft is along the trunk growth direction and makes the trunks symmetrical;
obtaining a crown segmentation chart according to the pavement tree integral segmentation chart, and obtaining a segmentation line of crowns of two adjacent pavement trees;
taking a region surrounded by a central axis of a trunk of one street tree, the dividing line and edge information between the central axis of the trunk of one street tree and the dividing line as a shielding region of the other street tree, and taking a region surrounded by a central axis of the trunk of the other street tree, the dividing line and edge information between the central axis of the trunk of the other street tree and the dividing line as a shielding region of one street tree, so as to obtain shielding regions of two adjacent street trees;
determining each shielding contour information of adjacent street trees according to each shielding area, obtaining a single street tree segmentation graph according to the shielding contour information and the crown edge information, and further obtaining a density distribution graph of the single street tree;
obtaining the staggered area of two adjacent street trees according to the shielding outline information of the two adjacent street trees, calculating the area of the staggered area, and obtaining the confidence coefficient of each street tree segmentation graph according to the area of the staggered area and the area of the corresponding street tree;
calculating an average characteristic graph according to each confidence coefficient and the density distribution graph of the corresponding single street tree; subtracting the average characteristic diagram from the density distribution diagram to obtain deviation diagrams; and calculating the average degree of the deviation graph, and pruning the current street tree when the average degree is greater than a set threshold value.
Further, the process of obtaining the dividing line is as follows:
dividing the street tree crown segmentation drawing along a direction vertical to the central axis to obtain upper edge information of the upper crown segmentation drawing and lower edge information of the lower crown segmentation drawing;
obtaining valley points according to the upper edge information, and forming an upper intersection point sequence by the valley points
(ii) a Obtaining a peak point according to the lower edge information, and forming a lower junction point sequence by the peak point
；
Matching each peak point and each valley point according to the corresponding abscissa to obtain matched peak points and valley points, and connecting the matched peak points and valley points, wherein the connecting line is a dividing line of two adjacent street trees.
Further, the method also comprises a step of correcting the upper junction point sequence or the lower junction point sequence of the upper sequence;
the process of correcting the upper exchange point sequence comprises the following steps:
constructing a correction model; the correction model is as follows:
wherein the content of the first and second substances,
respectively being a sequence of upper and lower junction points
The corresponding abscissa when the middle pixel value is 1;
is composed of
A neighborhood range of
With a central, left-right distance of 4 steps
A rectangular region of (a);
to allow for the deviation;
and judging whether the abscissa of a valley point in the upper junction sequence meets the correction model, if so, reserving the abscissa, otherwise, setting a pixel value corresponding to the abscissa to be 0, and obtaining the updated upper junction sequence.
Further, the obtaining process of the occlusion contour information is as follows:
multiplying each shielding area with the RGB image respectively to obtain a corresponding color image;
calculating saturation and brightness of the color image;
according to the saturation and the brightness, calculating to obtain the density degree of the pixels in the color image;
and acquiring contour points with the maximum density change according to the density of the pixel points, wherein each contour point forms contour information corresponding to the street tree.
Further, the method for obtaining the density distribution map of the single street tree comprises the following steps:
obtaining scattered points of a region between the central axis of the other street tree and the outline information according to the outline information of one street tree and the central axis of the other street tree, and fitting the scattered points by using a least square method to obtain a density distribution polynomial function of an unshielded region of the other street tree;
obtaining the density degree corresponding to each pixel point of the street tree segmentation graph according to the other street tree segmentation graph and the density distribution polynomial function; and replacing the pixel value in the single street tree segmentation graph with each density degree to obtain a density degree distribution graph of the single street tree.
Furthermore, the area of the interlaced region is the counted number of the pixels in the interlaced region.
Further, the confidence is:
wherein the content of the first and second substances,
representing an area of a single street tree segmentation graph;
representing the single street tree
Two street trees adjacent to the street trees divide the staggered area of the graph.
Further, the image processing process includes:
(a) obtaining linear information in the whole edge information of the street tree by using Hough linear detection to obtain a linear equation;
wherein the content of the first and second substances,xwhich is the abscissa of the edge information,k、bslope and intercept, respectively;
(b) performing intersection calculation on the whole edge information and the straight line information of the street tree to obtain the length of the straight line segment;
(c) respectively comparing the slope of the straight line segment with the set slope, and the length of the straight line segment with the set length, if the slope and the set slope meet the requirements
And is
Then, the straight line segment is the edge information of the trunk of the street tree; further obtaining a plurality of straight-line segments of the main trunk of the street tree in the overall segmentation graph;
(d) and calculating the distance between every two straight line segments, wherein when the distance is smaller than a set threshold value, the two straight line segments are two straight line segments of the trunk of the same street tree, and the central axis of each street tree is obtained by taking the central lines of the two straight line segments.
Further, the average degree of each deviation graph is as follows:
wherein the content of the first and second substances,
the difference of the density degree in the deviation graph is shown, W and H are the size of the deviation graph, and Delta T is the deviation graph.
The invention also provides a technical scheme of the artificial intelligence-based garden street tree pruning recognition system, which comprises a memory and a processor, wherein the processor is used for executing the technical scheme of the artificial intelligence-based garden street tree pruning recognition method stored in the memory.
The invention has the following beneficial effects:
according to the scheme, the method and the device have the advantages that the tree crowns between the adjacent street trees are mutually staggered to influence the characteristic detection of a single street tree, the contour information and the density distribution information of each street tree are obtained by utilizing the overall segmentation graph and the density change of the tree crowns, and the accuracy of the shaping and trimming time and scheme of the subsequent street trees is ensured.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions and advantages of the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 is a flowchart of a method of an embodiment of the method for identifying garden street tree pruning based on artificial intelligence of the present invention;
FIG. 2 is a schematic diagram of a central axis of a trunk of a street tree obtained by the garden street tree pruning recognition method based on artificial intelligence;
FIG. 3 is a schematic diagram of obtaining occlusion contour information in the method for pruning and identifying garden street trees based on artificial intelligence;
fig. 4 is a schematic diagram of another embodiment of obtaining occlusion contour information of the garden street tree pruning recognition method based on artificial intelligence.
Detailed Description
To further explain the technical means and effects of the present invention adopted to achieve the predetermined objects, the embodiments, structures, features and effects thereof according to the present invention will be described in detail below with reference to the accompanying drawings and preferred embodiments. In the following description, the different references to "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
The following describes a specific scheme of the garden street tree pruning recognition method based on artificial intelligence in detail with reference to the accompanying drawings.
Referring to fig. 1, a flow chart of steps of a garden street tree pruning recognition method based on artificial intelligence according to an embodiment of the present invention is shown, where the method includes the following steps:
step 1, collecting RGB images of a street tree, and obtaining an integral street tree segmentation image by utilizing a semantic segmentation network;
in the embodiment, monitoring cameras on two sides of a garden street are used for collecting RGB images of each street tree, the collected RGB images are input into a semantic segmentation network to obtain Mask regions of the street trees, and the Mask region images of the street trees on the same road are spliced to obtain an integral segmentation map F of the street trees on the same street.
The Mask region is a binary map, that is, the pixel value of each street tree region is 1, and the pixel values of the other regions are 0. In addition, since image stitching is a known technology, it is not described herein again.
The semantic segmentation network in this embodiment is an Encoder-Decoder structure, and may be implemented by using existing semantic segmentation networks such as UNet and deep lab.
The training method of the semantic segmentation network comprises the following steps: and (3) marking a label on each piece of training data by using the RGB image acquired by the monitoring camera as the training data to obtain label data.
In the embodiment, the label data is obtained by setting the pixel value of the street tree region to 1 and the pixel values of other regions to 0; and the loss function adopts a cross entropy function, and network training is carried out by using a gradient descent method to obtain a trained semantic segmentation network.
The step of obtaining the street tree integral segmentation map is that the spacing of the street trees during planting is fixed, the spacing between adjacent street trees is large, and the street trees are easy to distinguish, but along with the growth of the street trees, the crowns of the adjacent street trees are mutually staggered, so that the step directly obtains the integral segmentation map of the street trees on the same street for the feature analysis of a single subsequent street tree.
Step 2, carrying out image processing on the street tree integral segmentation graph to obtain integral border information of the street trees and obtain a central shaft of a trunk of each street tree, wherein the central shaft is along the trunk growth direction and makes the trunks symmetrical;
wherein the process of obtaining the central axis of the main trunk of each street tree is:
(a) acquiring integral edge information of a street tree, and acquiring straight line information in the integral edge information of the street tree by using Hough straight line detection to obtain a straight line equation;
wherein the content of the first and second substances,xwhich is the abscissa of the edge information and,k、bslope and intercept, respectively;
in this embodiment, the Canny edge detection algorithm is used to obtain the whole edge information of the street tree.
(b) Performing intersection calculation on the edge street tree whole edge information and the straight line information to obtain the length of the straight line segment;
(c) respectively comparing the slope of the straight line segment with the set slope, and the length of the straight line segment with the set length, if the slope and the set slope meet the requirements
And is
Then, the straight line segment is the edge information of the trunk of the street tree; further obtaining a plurality of straight line sections of the main trunk of the street tree in the overall segmentation graph;
set slope and set length in the present embodimentk 0 ，C 0 Is an empirical thresholdk 0 =5，C 0 =10, implementer can make changes according to camera's pose. The set length represents the height of the trunk of the street tree, and the height of the trunk of the street tree can be counted through historical data, and the set length in the image, which is a unit cm, can be obtained according to the zoom ratio of the shot image.
(d) Calculating the distance between every two straight line segments, when the distance is smaller than a set threshold, the two straight line segments are two straight line segments of the trunk of the same street tree, taking the central line of the two straight line segments to obtain the central axis of each street tree, and recording as Z, as shown in fig. 2, Z1, Z2, and Z3 are the central axes of the street trees respectively.
Wherein, the straight line segmentiAnd the straight line segmentjIs a distance of
，
Wherein, the first and the second end of the pipe are connected with each other,b i is a straight line segmentiThe intercept of (d);b j is a straight line segmentjThe intercept of (2).
In this embodiment, the closer the distance between the two straight line segments, the higher the matching degree of the two straight line segments, the grouping of a plurality of straight line segments can be completed, and each group is two straight line segments of the same trunk.
Step 3, obtaining an integral crown segmentation chart according to the pavement tree integral segmentation chart to obtain a segmentation line of two adjacent pavement tree crowns;
in this embodiment, the method for obtaining the dividing line includes:
(1) dividing the whole crown segmentation chart of the street tree along a direction vertical to the central shaft to obtain upper edge information of the upper crown segmentation chart and lower edge information of the lower crown segmentation chart;
the dividing method in the embodiment is as follows:
in this embodiment, as shown in fig. 3, the circumscribed rectangles are equally divided into 2 parts, specifically, the upper edge information of the upper crown segmentation graph and the lower edge information of the lower crown segmentation graph, so as to obtain the contour points of the upper edge information and the contour points of the lower edge information.
The number of the above-described parts may be 3 parts as long as the upper edge information of the upper crown split map and the lower edge information of the lower crown split map are satisfied.
(2) Obtaining a valley point according to the upper edge information to obtain an upper junction point sequence; obtaining a peak point according to the lower edge information to obtain a lower junction point sequence;
specifically, the process of acquiring the upper exchange point sequence is as follows:
determining a sequence based on the upper edge informationQ 1 ；
Setting a sliding window, to the sequenceQ 1 Performing sliding window, multiplying elements of the sliding window and elements of a wave trough sequence under a corresponding window first and then adding the multiplied elements to obtain fluctuation points, judging the size of each fluctuation point and a set fluctuation threshold, and when the fluctuation point is smaller than the set fluctuation threshold, setting the value of the fluctuation point to be 0, and keeping the other fluctuation points unchanged to obtain a fluctuation sequence;
taking a point which changes from positive to negative in the fluctuation sequence as a valley point, taking the position of the valley point as an intersection point above the crown of the adjacent street tree, marking the intersection point as 1, setting the values of the rest positions as 0, and obtaining an upper intersection point sequence
。
Wherein, the sequenceQ 1 Has a size of
，
As an image
Width information of (a), which reflects the profile distribution over the crown of the street tree; the sliding window has a size of
The size of the fluctuation threshold is set to 3.
In this example, the sequences were obtained in the same manner as described aboveQ 2 The point which is changed from negative to positive in the fluctuation sequence is obtained as a peak point, and the position of the point is taken as a lower junction point of the tree crowns of the adjacent street trees; marking the junction point below the crown as 1, and marking the values of the rest positions as 0 to obtain a junction point sequence
The detailed process is not described herein.
It should be noted that, in the invention, it is considered that when the crowns of the adjacent street trees are staggered together, a valley is formed at the intersection above the crowns of the adjacent street trees, the crown intersection of the adjacent street trees is obtained according to the characteristic, when the crowns of the adjacent street trees are staggered together, a peak is formed at the intersection below the crowns of the adjacent street trees, and the crown intersection of the adjacent street trees can also be obtained according to the characteristic.
(3) And matching each upper junction point and each lower junction point according to the corresponding position abscissa to obtain the matched upper junction point and lower junction point, and connecting the matched upper junction point and lower junction point, wherein the connecting line is a dividing line of two adjacent rows of trees.
It should be noted that, the dividing line of the connecting line of the matched upper junction and the lower junction is parallel to the central axis of the trunk of the street tree in principle, but since the abscissa of the corresponding position of the matched upper junction and the lower junction may not be equal, the connecting line may not be parallel to the central axis, which may be an error caused by the detection of the intersection of the crown due to the fluctuation of the profile of the crown of the street tree, the present invention further includes a step of correcting the upper junction sequence or the lower junction sequence.
Specifically, the correction step is as follows:
constructing a correction model;
the correction model is as follows:
wherein the content of the first and second substances,
respectively being a sequence of upper and lower junction points
The corresponding abscissa when the middle pixel value is 1;
is composed of
A neighborhood range of
With a central, left-right distance of 4 steps
A rectangular region of (a);
in order to allow for the deviation, the invention takes a value of 2, which is an empirical value set for humans.
And judging whether the abscissa of the upper junction in the upper junction sequence meets the correction model, if so, reserving the abscissa, otherwise, setting the pixel value corresponding to the abscissa to be 0, and obtaining the updated upper junction sequence.
According to the updated junctionObtaining the dividing line of the junction of the crowns of the adjacent street trees in sequence and recording the dividing line asD(ii) a The dividing line isDSlope of and central axis of trunk of street treeZThe slopes of (a) and (b) are uniform.
Meanwhile, the method is the same as above for correcting the lower junction point sequence, and redundant description is omitted here.
So far, the central axis of each street tree trunk in the overall segmentation graph is obtainedZAnd adjacent crown junction dividing lineDAs shown in fig. 2 as D12 and D23.
Step 4, taking a region surrounded by the central axis of the trunk of one street tree, the dividing line and the edge information between the central axis of the trunk of one street tree and the dividing line as a shielding region of the other street tree, and taking a region surrounded by the central axis of the trunk of the other street tree, the dividing line and the central axis of the trunk of the other street tree and the edge information between the dividing line as a shielding region of one street tree, so as to obtain shielding regions of two adjacent street trees;
determining each shielding contour information of adjacent street trees according to each shielding area, obtaining a single street tree graph according to the shielding contour information and the edge information, and further obtaining a density distribution map of the single street tree;
obtaining the staggered area of two adjacent street trees according to the shielding outline information of the two adjacent street trees, calculating the area of the staggered area, and calculating the confidence coefficient of each street tree segmentation graph according to the area of the staggered area and the area of the corresponding street tree;
in this embodiment, the obtaining process of the occlusion profile information is as follows:
multiplying each shielding area with the RGB image respectively to obtain a corresponding color image;
calculating saturation and brightness of the color image;
according to the saturation and the brightness, the density degree of the pixels in the color image is calculated;
and determining the contour information of each street tree according to the density of the pixel points.
In this embodiment, the method for obtaining the density distribution map of the single street tree includes:
obtaining scattered points of a region between the contour point and the symmetry axis of the other street tree according to the contour points in the contour information of one street tree and the symmetry axis of the other street tree, and fitting the scattered points by using a least square method to obtain a density distribution polynomial function of an unoccluded region of the other street tree;
obtaining a density index of the street tree according to the determined other street tree segmentation graph and the density distribution polynomial function; and replacing the pixel values in the single street tree segmentation graph with the density degrees to obtain a single street tree density degree distribution graph.
The confidence of a single street tree in this embodiment is:
wherein the content of the first and second substances,
representing an area of a single street tree segmentation graph;
representing the single street tree
Two street trees adjacent to the street trees divide the staggered area of the graph.
To specifically introduce the above step 4, three street trees are taken as an example for explanation:
according to the abscissa, the central shafts are arranged from large to smallZAnd a dividing lineDCarry out the numbering, noteuThe central axis of each street tree isZuOf 1 atvThe central axis of each street tree isZv(ii) a The dividing line between two street trees is marked as
(ii) a As shown in fig. 4, includes a street tree 1 (corresponding to the central axis)Z1）Street tree 2 (corresponding to the central shaft)Z 2) And a street tree 3 (corresponding to the central axis)Z3）。
First, it is determined whether occlusion exists between adjacent street trees:
judgment and judgmentZ1 andZ2 whether or not there is a dividing line
If the tree crown does not exist, the phenomenon that the tree crowns of the street tree 1 and the street tree 2 are not staggered mutually is shown, and the method is based on the principle thatZ1 andZ2, the line with the least number of foreground pixel points is segmented as a segmentation line to obtain a street tree 1 segmentation graph
(ii) a When in useZ1 andZ2 there is a dividing line
And the time indicates that the crowns of two adjacent street trees are mutually staggered, namely, a shielding area exists, wherein the shielding area comprises the contour information of the corresponding street tree.
Secondly, determining the occlusion area of the adjacent street tree:
according to the obtained dividing line of the whole road tree dividing graph
And the central shaftZ2, then, dividing line
Central shaftZ2 and a dividing line
And the central shaftZThe edge information between 2 forms the sheltered area of the street tree 1; by analogy, the sheltered area of the street tree 2 is a parting line
Central shaftZ1 and a dividing line
And the central shaftZ1, in the region between.
It should be noted that the obtained occlusion region of the street tree 2 is only one occlusion region of the street tree 2, and the occlusion region on the other side needs to be obtained according to the related information of the street tree 3.
Further, as another preferred embodiment, the division line may be used
Coordinates of two intersection points with crown segmentation chart
Respectively cross two intersections
Two perpendicular auxiliary lines (see the solid line parallel to the ground and passing through two coordinate points in fig. 4), and dividing the lines
Central shaftZAnd 2, acquiring a region surrounded by the two auxiliary lines as a final interested shielding region, wherein the interested shielding region contains all contour information of the shaded street tree 1.
It should be noted that the interested occlusion region at this time is a region in the binary image, and the pixel values outside the interested occlusion region are set to 0, so as to shield the interference of the irrelevant region.
Then, acquiring outline information of the occlusion area and a segmentation map of a single street tree:
multiplying each occlusion area by the RGB image in the step 1 to obtain a color image in the occlusion area;
changing the color image in the shielding area into an HSV color space to obtain values of three channels, wherein the values respectively correspond to a hue channel H, a saturation channel S and a lightness channel V;
calculating the density degree corresponding to each pixel point of the color image in the shielding area according to the values of the saturation channel S and the lightness channel V;
in this embodiment, a threshold of the hue channel is set to obtain a green region in the shielding region, and further the density of each pixel point in the green region is:
wherein the content of the first and second substances,
respectively indicate positions
Hue, saturation, lightness and density of the pixel point.
Replacing the pixel value at each pixel point position with the intensity degree to obtain an intensity degree distribution map in the shielding area; and determining the contour information in the area according to the density degree. In order to avoid the influence of the density index of 0 on the contour points, interpolation is used
The points are filled to obtain a density distribution map in the shielded area.
In the embodiment, it is considered that when two crowns are staggered, the crown density index at the staggered critical point is greatly changed, and therefore, the contour point of the corresponding street tree in the shielding area is determined according to the change of the density in the density distribution diagram.
Specifically, taking the occlusion area of the street tree 1 as an example, the analysis is performed:
A. from near the central axis of the street tree 2Z2 one side of statisticsmThe density of each point in the row, the abscissa being the central axisZ2, obtaining a scatter diagram by taking the vertical coordinate as the density degree; the point with the maximum change of the density degree in the scatter diagram is obtained as the first pointmOf street trees 1 in the rowContour points, noted
(ii) a It should be noted that the axis of symmetry is defined as
To the contour point
The area between is the density distribution of the trees 2; while the contour points
To
The density distribution of the area between the trees 1 and 2 is generated by staggering;
B. obtaining the contour points of each line in the occlusion area according to the step A, and combining all the contour points to obtain the contour information of the occlusion area of the street tree 1, namely the sequenceP。
It is noted that the center axisZ2 to sequencePThe area between the two is the dense degree distribution area of the street tree 2; and sequence ofPTo
The area between the two is the dense degree distribution of the staggered area generated by the mutual staggering of the street tree 1 and the street tree 2. Meanwhile, the central shaft can be obtained according to the stepsZ1 and the dividing line
Outline information of the occlusion region of the street tree 2.
In this embodiment, a segmentation map of a single tree can be obtained according to the obtained contour information. The segmentation graph of each tree is a binary graph, the pixel value of the tree region is 1, and the pixel values of other regions are 0. It should be noted that, in the foregoing, image segmentation is performed according to the density of trees, which is actually based on occlusion between trees, so that a color of a green region in an occluded region is different from that of an unoccluded portion, that is, the deeper the color of the green region is, it is determined that branches are dense, and there may be interleaving between trees; however, it should be noted that the tree may also have its own leaf color changed due to environmental factors, but the present invention does not consider other factors, and only determines the approximate outline information of a single tree for a basis of subsequent tree pruning.
And obtaining a density distribution diagram corresponding to the segmentation diagram of the single street tree:
obtaining the central axis according to the dense degree distribution area of the street tree 2Z2 to sequencePThe scatter point in the region between, the central axis of the street tree 22 is obtained by fitting the scatter point by the least square methodZ2 to sequencePDensity distribution polynomial function of the region in between
(ii) a And obtaining a single street tree busyness degree distribution diagram according to the determined single street tree segmentation diagram and the density distribution polynomial function.
In this embodiment, taking the street tree 2 as an example, a single street tree segmentation map and a street tree density distribution polynomial function are obtained
And then substituting all abscissa coordinates in the single street tree segmentation graph into a density distribution polynomial function to obtain the density degree of the corresponding position, and replacing the corresponding pixel value in the single street tree segmentation graph with each density degree to obtain a single street tree density degree distribution graph.
The highest power of the distribution polynomial function in this embodiment is set empirically, and takes a value of 4.
Meanwhile, for the street tree 1 and the street tree 3, scattered points of the non-occluded area are obtained first according to the method, polynomial fitting is carried out, and the corresponding density distribution graph is obtained.
And finally, obtaining confidence:
determining the area of the staggered area of a single street tree according to the determined staggered area of two adjacent street trees, and calculating the ratio of the area of the staggered area of the single street tree to the total area of the single street tree segmentation graph to obtain the confidence of each single street tree segmentation graph, wherein the street tree segmentation graph
Degree of confidence of
The calculation formula of (a) is as follows:
wherein, the first and the second end of the pipe are connected with each other,
showing a division diagram
The area of the middle foreground region;
representing street trees
And the intersection area between the two adjacent street tree segmentation maps.
It should be noted that, when the intersection area is larger, the intersection area represents a larger tree crown intersection area, and the confidence of the segmentation map is smaller.
The area in this embodiment is determined by counting the number of pixels in the interleaved region.
It should be noted that, in the above embodiment, only 3 trees are selected for analysis in order to simplify the analysis, and there are many road trees on both sides of the road, such as 10, 20, etc., which need to be determined according to the actual road condition.
Step 5, calculating an average characteristic graph according to each confidence coefficient and the density distribution graph of the corresponding street tree; subtracting the average characteristic diagram from the density distribution diagram to obtain deviation diagrams; and calculating the average degree of the deviation graph, and pruning the current street tree when the average degree is greater than a set degree threshold value.
In this example, the average feature map
Comprises the following steps:
wherein the content of the first and second substances,Numthe number of street trees on the same street,
to be aligned withuThe confidence of the individual street trees is normalized to the standard confidence,
is as followsuThe density distribution graph of each street tree.
It should be noted that, the average feature map in the above description may reflect the average density distribution and the average profile information; the average feature map is the same size as the feature map of each street tree. Meanwhile, the average feature map in this embodiment is the sum of feature values of feature maps of street trees corresponding to weight correction determined according to the degree of occlusion of different street trees, that is, it may be considered as an average value of feature maps of all street trees, and the average value is used as a standard map and compared with feature maps of all street trees, so as to implement pruning of corresponding street trees, and ensure that all street trees are kept consistent after pruning as much as possible.
The average degree of each deviation plot is:
wherein the content of the first and second substances,
is the difference of density degree in the deviation diagram, W and H are the size of the deviation diagram, and Delta T is the deviation diagram
When in use
It indicates that the current street tree needs to be pruned,
to set a threshold level. The set threshold in this embodiment is obtained by processing the trimmed street tree. Of course, as another embodiment, the setting may be performed directly based on human experience, for example, the setting may be performed
。
Deviation diagram in the present embodiment
The pixel values of the regions can reflect the difference of the contour and the density, and the pruning scheme is determined according to the deviation graph of each street tree.
Further, the number of street trees to be pruned is counted
When is coming into contact with
And when the street trees do not reach the corresponding quantity, the trimming is not carried out firstly.
Further, for the above-mentioned street tree for determining pruninguThe pruning scheme can be realized by means of a neural network, and the input is a street treeuDeviation map of street tree
(ii) a The output is the area to be trimmed.
The neural network is an Encoder-Decoder semantic segmentation structure, the training data is a street tree deviation map of each street tree, the label data is an artificially calibrated region needing to be cut, the pixel value of the region needing to be cut is 1, and the pixel values of other regions are 0; the loss function is trained by adopting a cross entropy function.
Based on the same inventive concept as the method, the invention also provides an artificial intelligence-based garden street tree pruning recognition system, which comprises a processor and a memory, wherein the processor is used for executing the program of the artificial intelligence-based garden street tree pruning recognition method embodiment stored in the memory; since the embodiments of the garden street tree pruning and identification method based on artificial intelligence are already described in the above embodiments, redundant description is omitted here.
It should be noted that: the precedence order of the above embodiments of the present invention is only for description, and does not represent the merits of the embodiments. And specific embodiments thereof have been described above. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (8)
1. The garden street tree shaping, trimming and identifying method based on artificial intelligence is characterized by comprising the following steps of:
collecting RGB images of the street tree, and obtaining a whole street tree segmentation image by utilizing a semantic segmentation network;
carrying out image processing on the street tree integral segmentation graph to obtain integral border information of the street trees and obtain a central shaft of a trunk of each street tree, wherein the central shaft is along the trunk growth direction and makes the trunks symmetrical;
obtaining a crown segmentation chart according to the pavement tree integral segmentation chart to obtain a segmentation line of crowns of two adjacent pavement trees;
taking a region surrounded by a central axis of a trunk of one street tree, the dividing line and edge information between the central axis of the trunk of one street tree and the dividing line as a shielding region of the other street tree, and taking a region surrounded by a central axis of the trunk of the other street tree, the dividing line and edge information between the central axis of the trunk of the other street tree and the dividing line as a shielding region of one street tree, so as to obtain shielding regions of two adjacent street trees;
determining each shielding contour information of adjacent street trees according to each shielding area, obtaining a single street tree segmentation graph according to the shielding contour information and the whole edge information of the street trees, and further obtaining a density distribution graph of the single street tree;
obtaining the staggered area of two adjacent street trees according to the shielding outline information of the two adjacent street trees, calculating the area of the staggered area, and obtaining the confidence coefficient of each street tree segmentation graph according to the area of the staggered area and the area of the corresponding street tree;
calculating an average characteristic graph according to each confidence coefficient and the density distribution graph of the corresponding single street tree; subtracting the average characteristic diagram from the density distribution diagram to obtain deviation diagrams; calculating the average degree of the deviation graph, and pruning the current street tree when the average degree is greater than a set threshold value;
the obtaining process of the shielding outline information comprises the following steps:
multiplying each shielding area with the RGB image respectively to obtain a corresponding color image;
calculating saturation and brightness of the color image;
according to the saturation and the brightness, the density degree of the pixels in the color image is calculated;
acquiring contour points with the maximum density change according to the density of the pixel points, wherein each contour point forms contour information of a corresponding street tree;
the method for acquiring the density distribution map of the single street tree comprises the following steps:
obtaining scattered points of a region between the central axis of the other street tree and the outline information according to the outline information of one street tree and the central axis of the other street tree, and fitting the scattered points by using a least square method to obtain a density distribution polynomial function of an unshielded region of the other street tree;
obtaining the density degree corresponding to each pixel point of the street tree segmentation graph according to the other street tree segmentation graph and the density distribution polynomial function; and replacing the pixel values in the single street tree segmentation graph with the density degrees to obtain a density degree distribution graph of the single street tree.
2. The method for pruning and identifying garden street trees based on artificial intelligence according to claim 1, wherein the process for obtaining the dividing line is as follows:
dividing the street tree crown segmentation drawing along a direction vertical to the central axis to obtain upper edge information of the upper crown segmentation drawing and lower edge information of the lower crown segmentation drawing;
obtaining valley points according to the upper edge information, and forming the valley points into an upper junction sequence
(ii) a Obtaining a peak point according to the lower edge information, and forming a lower junction point sequence by the peak point
；
Matching each peak point and each valley point according to the corresponding abscissa to obtain matched peak points and valley points, and connecting the matched peak points and valley points, wherein the connecting line is a dividing line of two adjacent street trees.
3. The artificial intelligence-based garden street tree shaping, trimming and identifying method according to claim 2, characterized by further comprising the step of correcting the upper junction sequence or the lower junction sequence of the upper sequence;
the process of correcting the upper exchange point sequence comprises the following steps:
constructing a correction model; the correction model is as follows:
wherein the content of the first and second substances,
respectively being a sequence of upper and lower junction points
The abscissa corresponding to the middle pixel value of 1;
is composed of
A neighborhood range of (2), the neighborhood range of
With a central, left-right distance of 4 steps
A rectangular region of (a);
to allow for the deviation;
and judging whether the abscissa of the valley point in the upper junction point sequence meets the correction model, if so, keeping the abscissa, otherwise, setting the pixel value corresponding to the abscissa to be 0, and obtaining the updated upper junction point sequence.
4. The method for pruning and identifying garden street tree based on artificial intelligence according to claim 1, wherein the area of the staggered area is the counted number of pixels in the staggered area.
5. The artificial intelligence-based garden street tree pruning recognition method according to claim 4, wherein the confidence is as follows:
wherein, the first and the second end of the pipe are connected with each other,
representing the area of a single street tree segmentation graph;
representing the single street tree
Two street trees adjacent to the street trees divide the staggered area of the graph.
6. The method for pruning and identifying garden street trees based on artificial intelligence according to claim 1, wherein the image processing process comprises:
(a) obtaining linear information in the whole edge information of the street tree by using Hough linear detection to obtain a linear equation;
wherein, the first and the second end of the pipe are connected with each other,xwhich is the abscissa of the edge information and,k、bslope and intercept, respectively;
(b) performing intersection calculation on the whole edge information and the straight line information of the street tree to obtain the length of a straight line segment;
(c) respectively comparing the slope of the straight line segment with a set slope, and the length of the straight line segment with a set length, and if the slope and the set slope meet the requirements
And is provided with
Then, the straight line segment is the edge information of the trunk of the street tree; further obtaining a plurality of straight line sections of the main trunk of the street tree in the overall segmentation graph;
(d) and calculating the distance between every two straight line segments, wherein when the distance is smaller than a set threshold value, the two straight line segments are two straight line segments of the trunk of the same street tree, and the central axis of each street tree is obtained by taking the central lines of the two straight line segments.
7. The method for pruning and identifying garden street trees based on artificial intelligence according to claim 1, wherein the average degree of each deviation map is as follows:
wherein, the first and the second end of the pipe are connected with each other,
is the difference of density in the deviation diagram, W and H are the size of the deviation diagram, and DeltaTIs a deviation graph.
8. An artificial intelligence based garden street tree pruning recognition system, comprising a memory and a processor, wherein the processor is used for executing the steps of the artificial intelligence based garden street tree pruning recognition method stored in the memory according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210952938.8A CN115035415B (en) | 2022-08-10 | 2022-08-10 | Garden street tree shaping, trimming and identifying method and system based on artificial intelligence |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210952938.8A CN115035415B (en) | 2022-08-10 | 2022-08-10 | Garden street tree shaping, trimming and identifying method and system based on artificial intelligence |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115035415A true CN115035415A (en) | 2022-09-09 |
| CN115035415B CN115035415B (en) | 2022-11-11 |
Family
ID=83131019
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210952938.8A Active CN115035415B (en) | 2022-08-10 | 2022-08-10 | Garden street tree shaping, trimming and identifying method and system based on artificial intelligence |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115035415B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110041396A1 (en) * | 2009-08-19 | 2011-02-24 | Stafford Robert N | Convex Cutting and Trimming Method for Shaping Trees |
| CN110199706A (en) * | 2019-06-05 | 2019-09-06 | 安徽工程大学 | A kind of artificial intelligence greening pruner machine |
| CN112819830A (en) * | 2021-01-24 | 2021-05-18 | 南京林业大学 | Individual tree crown segmentation method based on deep learning and airborne laser point cloud |
| CN114241324A (en) * | 2021-12-31 | 2022-03-25 | 北京精英路通科技有限公司 | Tree green leaf trimming early warning method, electronic equipment and program product |
-
2022
- 2022-08-10 CN CN202210952938.8A patent/CN115035415B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110041396A1 (en) * | 2009-08-19 | 2011-02-24 | Stafford Robert N | Convex Cutting and Trimming Method for Shaping Trees |
| CN110199706A (en) * | 2019-06-05 | 2019-09-06 | 安徽工程大学 | A kind of artificial intelligence greening pruner machine |
| CN112819830A (en) * | 2021-01-24 | 2021-05-18 | 南京林业大学 | Individual tree crown segmentation method based on deep learning and airborne laser point cloud |
| CN114241324A (en) * | 2021-12-31 | 2022-03-25 | 北京精英路通科技有限公司 | Tree green leaf trimming early warning method, electronic equipment and program product |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115035415B (en) | 2022-11-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109711325B (en) | Mango picking point identification method | |
| CN105718945B (en) | Apple picking robot night image recognition method based on watershed and neural network | |
| CN108710840B (en) | Visual navigation path identification method for farmland pesticide spraying robot | |
| Lv et al. | A method to obtain the near-large fruit from apple image in orchard for single-arm apple harvesting robot | |
| CN110598532B (en) | Tree pest and disease damage monitoring system and method | |
| CN110569786A (en) | fruit tree identification and quantity monitoring method and system based on unmanned aerial vehicle data acquisition | |
| CN113575388A (en) | Agricultural intelligent irrigation system based on artificial intelligence and big data | |
| CN113255434B (en) | Apple identification method integrating fruit characteristics and deep convolutional neural network | |
| CN110827273A (en) | Tea disease detection method based on regional convolution neural network | |
| CN115908371B (en) | Plant leaf disease and pest degree detection method based on optimized segmentation | |
| CN108629289A (en) | The recognition methods in farmland and system, applied to the unmanned plane of agricultural | |
| CN116309670B (en) | Bush coverage measuring method based on unmanned aerial vehicle | |
| CN102542560A (en) | Method for automatically detecting density of rice after transplantation | |
| WO2024109419A1 (en) | Elevation information-fused processing method for remote sensing images | |
| CN115641412A (en) | Hyperspectral data-based three-dimensional semantic map generation method | |
| CN111798470A (en) | Crop image entity segmentation method and system applied to intelligent agriculture | |
| CN109086823A (en) | A kind of wheat scab disease tassel yield method for automatically counting | |
| CN103226709B (en) | A kind of network curtain image recognition method of fall webworm larvae | |
| CN115035415B (en) | Garden street tree shaping, trimming and identifying method and system based on artificial intelligence | |
| CN112561844B (en) | Automatic generation method of digital camouflage pattern fused with texture structure | |
| CN111275698B (en) | Method for detecting visibility of road in foggy weather based on unimodal offset maximum entropy threshold segmentation | |
| CN111950349A (en) | Semantic segmentation based field navigation line extraction method | |
| CN111428990A (en) | Deep neural network-based method for evaluating flower grade of water-cultured flowers in flowering period | |
| CN115953686A (en) | Peanut pest detection method and system based on image processing | |
| CN111079530A (en) | Mature strawberry identification method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20240118 Address after: Group 13, Sanwei Village, Haifu Town, Qidong City, Nantong City, Jiangsu Province 226000 Patentee after: Nantong Xindi Fishing Tackle Co.,Ltd. Address before: 226200 Zhongshi village, Wangbao Town, Qidong City, Nantong City, Jiangsu Province Patentee before: Nantong Zhonghuang Tools Co.,Ltd. |