CN108961286B - Unmanned aerial vehicle image segmentation method considering three-dimensional and edge shape characteristics of building - Google Patents
Unmanned aerial vehicle image segmentation method considering three-dimensional and edge shape characteristics of building Download PDFInfo
- Publication number
- CN108961286B CN108961286B CN201810680919.8A CN201810680919A CN108961286B CN 108961286 B CN108961286 B CN 108961286B CN 201810680919 A CN201810680919 A CN 201810680919A CN 108961286 B CN108961286 B CN 108961286B
- Authority
- CN
- China
- Prior art keywords
- image
- objects
- segmentation
- edge
- merged
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/187—Segmentation; Edge detection involving region growing; involving region merging; involving connected component labelling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10032—Satellite or aerial image; Remote sensing
Abstract
An unmanned aerial vehicle image segmentation method considering three-dimensional and edge shape characteristics of a building comprises the steps of setting segmentation scale parameters according to the area of a maximum object to be extracted, carrying out mask processing on an invalid area, taking single pixels of an input orthorectified image, an elevation orthorectified image and an SLIC label image as objects, and initializing a segmentation flow; finding adjacent pixel objects for the initial pixel objects, and if SLIC labels are the same, merging until all the pixel objects with the same labels are merged together; adding Canny edge straight line information and vegetation mask information, and carrying out edge marking and vegetation marking on a single superpixel segmentation object in a superpixel pre-segmentation result; performing loop iteration, searching the most similar object of the object under various constraint conditions, judging whether the most similar object is combined with the current object properly, and repeating iteration until no object which can be combined again exists; and optimally merging the small-area areas which are not merged in the result all the time.
Description
Technical Field
The invention relates to the technical field of remote sensing image processing, in particular to an unmanned aerial vehicle image segmentation method considering three-dimensional and edge shape characteristics of a building.
Background
With the improvement of the spatial resolution of remote sensing images, Object Based Image Analysis (OBIA) gradually becomes an effective high spatial resolution Image Analysis tool. The image analysis of the object level can weaken the spectrum difference inside the ground object and reduce the salt and pepper noise in the result. Image segmentation is a crucial step in object-level image analysis such as feature recognition, information extraction, or image classification. Unmanned aerial vehicle flying height is generally lower, and image spatial resolution is very high, more is close to the close-range image, therefore single ground object is more clear, the inside spectral difference grow of ground object, and "the foreign matter is with the spectrum" phenomenon also more obvious. In the traditional segmentation method based on spectrum and shape heterogeneity, as the 'averaging' of the features of the segmented objects is subject to greater uncertainty in the merging process of the objects, scale parameters in the segmentation method are difficult to select and have no determined physical significance, edge shape features in images are difficult to utilize, and three-dimensional elevation information is only used as other one-dimensional 'two-dimensional' waveband information, so that building areas and non-building areas in segmentation results are possibly wrongly merged, and the subsequent object-oriented image analysis is difficult to generate a correct analysis object.
Disclosure of Invention
Aiming at the problems, the invention provides an unmanned aerial vehicle image segmentation technical scheme considering the three-dimensional and edge shape characteristics of the building, the unmanned aerial vehicle image segmentation technical scheme has strong operability, the segmentation parameters have certain physical significance, a trial and error mode is not used in parameter selection, and fuzzy constraint is adopted to represent the area of a segmentation object with the largest area possibly obtained in a segmentation result. Meanwhile, elevation information of the building can be fully considered, the DSM is converted into an elevation orthographic image by using the imaging geometric model to participate in segmentation, and the influence of projection difference generated by the central projection building on a segmentation result is weakened, so that the segmentation result can keep the building object and other non-building areas from being adhered mistakenly, and the final image segmentation result can better represent the real building.
The invention discloses an unmanned aerial vehicle image segmentation method considering three-dimensional and edge shape characteristics of a building, which comprises the following steps of:
step 1, setting segmentation scale parameters according to the maximum object area to be extracted, performing mask processing on an invalid area, taking single pixels of an input orthorectified image, an elevation orthorectified image and an SLIC label image as objects, and initializing a segmentation process;
step 2, performing pre-segmentation based on SLIC superpixels; finding adjacent pixel objects for the initial pixel objects, and if SLIC labels are the same, merging until all the pixel objects with the same labels are merged together;
step 3, extracting Canny edge straight lines and vegetation from the orthophoto image, and performing edge marking and vegetation marking on a single superpixel segmentation object in the superpixel pre-segmentation result by using Canny edge straight line information and vegetation mask information;
step 4, searching the most similar object of the current object, further judging whether the most similar object is combined with the current object or not by combining the segmentation scale parameter, the vegetation mark information and other constraint conditions, repeating iteration until no object which can be combined again exists, wherein the first N isitSkipping the superpixels of the edge marks during the secondary iteration to find the most similar object;
and 5, optimally combining the small-area areas which are not combined in the result all the time.
Further, in step 1, the elevation orthographic image is obtained by,
(1) acquiring the pixel size of an original unmanned aerial vehicle image, generating a virtual image with the same size, wherein the data type of the virtual image is a double type, the number of wave bands is 1, and the initial values of all pixels are set to be infinite;
(2) calculating the actual ground positions corresponding to the four corners of the image according to the internal and external orientation elements and the DTM of the original image, and obtaining the maximum value of the range; in order to ensure that the edges of the virtual image can be calculated to obtain an elevation value, carrying out external expansion on the obtained maximum range to obtain an external expansion range;
(3) acquiring test area DSM data in the external expansion range, sequencing three-dimensional points of a DSM grid from high to low according to elevation, and simultaneously recording the plane coordinate XY of each grid point;
(4) in order to enable the generated virtual image to have the relative shielding condition the same as the real condition, a series of three-dimensional points are generated in a descending manner according to a certain height difference on the plane position of the current grid point and are re-projected on the virtual image one by one until the elevation is reduced to the DTM elevation value of the corresponding position, wherein the re-projection process is calculated by utilizing a collinear equation:
in the above formula, X, Y and Z are three-dimensional point coordinates, X0,y0F is an internal orientation element of the original unmanned aerial vehicle image, ai,bi,ci(i is 1,2,3) is the cosine of the orientation in the outward direction, XS,YS,ZSThe method comprises the following steps that (1) x and y are elements of an outer orientation line of an original unmanned aerial vehicle image, are coordinates of a three-dimensional point re-projected image plane, and are converted into pixel coordinates according to the pixel size of a camera; and at the same pixel position on the re-projected image, taking the maximum value of all elevation values, and correcting the virtual image again by using the internal and external orientation elements and the measurement area DTM of the original image to obtain an elevation orthographic image.
Further, in step 3, Canny edge straight line information is obtained as follows,
(1) extracting Canny edge straight lines of the orthoimage, then carrying out 4-communication region connection on the edge detection result, and connecting 8-communication pixels into 4-communication;
(2) scanning the whole Canny edge detection result image to obtain the interconnected regions of edge pixels;
(3) and (3) re-rasterizing the pixels which accord with the linear model in the current connection region into an image, checking whether the pixels are mutually communicated or not, and if the pixels are mutually communicated and the number of the pixels in the connection region is greater than a given linear length threshold, determining that the detected pixels are linear.
Furthermore, in step 4, the most similar object of the current object is found by using a method with a spectral similarity measure as a primary measure and a shape similarity measure as a secondary measure, and the specific implementation manner is as follows,
(1) when the spectrum similarity is calculated, an area difference penalty term and a public side length penalty term are added in the similarity calculation process, and the similarity between two adjacent objects is calculated by the following formula:
Dr＝D·exp(lp -1+|n1-n2|·(n1+n2)-1)
in the above formula, D is the spectral similarity of two adjacent objects, B is the set of data bands participating in the segmentation, wbIs the weight of the b band, cibIs the mean value of the b band of the object i, cjbB-band mean, D, of object jrCalculating the similarity of the area difference punishment and the public edge length punishmentpLength of a common edge of two adjacent objects, n1And n2Is the area of two objects; then measuring the spectral similarity D between the objectsrThe object smaller than 10 is taken as the most similar object of the current object;
(2) the shape factor calculation aims at selecting a more appropriate merging object from two adjacent objects which are close to each other, actually, the calculation is carried out on the objects after the virtual merging, if the current object is Ri, and the objects which are similar to each other in two spectrums are Rj and Rk respectively, the Ri and the Rj are virtually merged firstly, the shape factor is calculated, then the Ri and the Rk are virtually merged, the shape factor is calculated, and then the object with the small shape factor is used as the most similar object;
after the current object and the adjacent similar object are combined in a 'virtual' mode, the weighted average of the compactness and the smoothness is smaller and better, and the shape factor calculation formula comprehensively considering the compactness and the smoothness is as follows:
h＝wcmpt·hcmpt+wsmooth·hsmooth
the compactness calculation formula is as follows:
the smoothness calculation formula is:
wherein the content of the first and second substances,wcmptto compact weight, wsmoothAnd f, representing smoothness weight, wherein l is the perimeter of the merged object, n is the area of the merged object, and b is the perimeter of the minimum bounding rectangle of the merged object.
Further, in step 4, it is determined whether the most similar object really needs to be merged with the current object, where the determination conditions are as follows:
(1) the most similar objects exceed the scale constraint, namely the segmentation scale parameters set in the step 1, and are not combined;
(2) the most similar objects accord with the judgment of the convex model and are not merged;
(3) the most similar object vegetation mask marks are inconsistent and not combined;
(4) the common edge of the most similar object and the current object is a super short edge and is not merged;
(5) the current object is very close to a regular shape, and the most similar object is not in the regular shape after being merged with the current object and is not merged;
(6) the elevation of the most similar object is truncated and not merged;
(7) after the most similar object and the current object are combined, the internal consistency measure exceeds a threshold (spectrum and elevation), and the most similar object and the current object are not combined;
when the most similar object and the current object are combined and meet the regular shape constraint, or the conditions are not met, the current object and the most similar object are considered to be required to be combined; the internal spectrum consistency measure threshold in the (7) th condition is the maximum value of the standard deviation of the RGB bands in the current image, and the internal elevation consistency measure threshold is the standard deviation of the elevation in the current image.
Further, NitIs 3 or 4.
The processing method is clear and strong in operability, and can really take the problem that projection difference caused by central projection does not correspond to high-range information in the image into account, so that the segmentation result can keep building objects and other non-building areas from being adhered mistakenly, and the final image segmentation result can better represent a real building.
Drawings
FIG. 1 is a flow chart of an embodiment of the present invention.
FIG. 2 is a graph of the segmentation result according to the embodiment of the present invention.
FIG. 3 is a flowchart illustrating an exemplary process of generating an elevation ortho image according to the present invention.
FIG. 4 is an elevation orthographic view of an embodiment of the invention.
FIG. 5 is an overlay of an elevation ortho image and a corresponding original ortho image according to an embodiment of the present invention.
Detailed Description
The unmanned aerial vehicle image segmentation method considering the three-dimensional and edge shape characteristics of the building, provided by the invention, comprises the steps of setting segmentation scale parameters for the maximum object area to be extracted, carrying out mask processing on an invalid area, taking single pixels of an input orthorectified image, an elevation orthorectified image and an SLIC label image as objects, and initializing a segmentation flow; pre-segmentation based on SLIC superpixels is carried out, adjacent pixel objects are found for the initial pixel objects, if SLIC labels are the same, merging is carried out until all the pixel objects with the same labels are merged together; adding Canny edge straight line information and vegetation mask information, and carrying out edge marking and vegetation marking on a single superpixel segmentation object in a superpixel pre-segmentation result; searching the most similar object of the object under various constraint conditions, judging whether the most similar object is combined with the current object, repeating iteration until no object which can be combined again exists, and skipping the superpixel of the edge mark during the previous 3 or 4 iterations to search the most similar object; and optimally merging the small-area areas which are not merged in the result all the time. In specific implementation, a computer software technology can be adopted to realize automatic process operation, and the technical scheme of the invention is described in detail below by combining the drawings and the embodiments.
As shown in fig. 1, the flow of the embodiment specifically includes the following steps:
step 1, setting segmentation scale parameters according to the maximum object area to be extracted, performing mask processing on an invalid area, taking single pixels of an input ortho-corrected image, an elevation ortho-image and an SLIC label image as objects, and initializing a segmentation process:
the segmentation scale parameter of the method is set according to the area of the maximum object to be segmented in the image, and the scale parameter is a fuzzy constraint and represents the segmented object with the maximum possible area in the segmentation result. Since the orthoimage has an invalid data region around the image during the correction process, it is necessary to mask this invalid region during the initialization of segmentation, i.e., pixels in the invalid region are not to be initially segmented into individual pixels during the initialization of segmentation.
Obtaining of single ortho-image: after all unmanned aerial vehicle images are subjected to space-three processing, a single image can be obtained by orthorectification by utilizing inner orientation elements of the image, restored outer orientation elements of the image and a DEM (digital elevation model) of a measuring area;
the flowchart of the generation of a single elevation ortho image is shown in fig. 3, and the specific steps are as follows:
(5) acquiring the pixel size of an original unmanned aerial vehicle image, generating a virtual image with the same size, wherein the data type of the virtual image is a double type, the number of wave bands is 1, and the initial values of all pixels are set to be infinite;
(6) and calculating the actual ground positions corresponding to the four corners of the image according to the internal and external orientation elements and the DTM of the original unmanned aerial vehicle image, and obtaining the maximum value of the range. In order to ensure that the edges of the virtual image can be calculated to obtain an elevation value, carrying out external expansion on the obtained maximum range to obtain an external expansion range;
(7) acquiring test area DSM data in the external expansion range, sequencing three-dimensional points of a DSM grid from high to low according to elevation, and simultaneously recording the plane coordinate XY of each grid point;
(8) and (4) re-projecting each grid point on the virtual image from high to low. In order to enable the generated virtual image to have the same relative shielding condition as the real condition, a series of three-dimensional points are generated on the plane position of the current grid point in a descending manner according to a certain height difference and are projected on the virtual image again one by one until the elevation is reduced to the DTM elevation value of the corresponding position. The reprojection process is calculated using the collinearity equation:
in the above formula, X, Y and Z are three-dimensional point coordinates, X0,y0F is an internal orientation element of the original unmanned aerial vehicle image, ai,bi,ci(i is 1,2,3) is the cosine of the orientation in the outward direction, XS,YS,ZSThe method is characterized in that the method is an original unmanned aerial vehicle image outer orientation line element, x and y are image plane coordinates of a three-dimensional point after re-projection, and the image plane coordinates are converted into pixel coordinates according to the pixel size of a camera. And (3) taking the maximum value of all elevation values at the same pixel position on the re-projected image, and correcting the virtual image again by using the internal and external orientation elements and the measurement area DTM of the original unmanned aerial vehicle image (the principle of performing orthorectification on the single original unmanned aerial vehicle image is the same) to obtain the elevation orthoimage. FIG. 4 is an elevation orthographic image generated by the above steps, wherein the pixel values in the image are elevation values of pixels corresponding to the original image; fig. 5 is a display comparison of the elevation ortho image and the corresponding ortho image of the drone.
The SLIC super-pixel label image is obtained from the corrected orthoimage of the unmanned aerial vehicle, wherein the SLIC super-pixel label image can be found in documents during specific implementation: achanta R, ShajiA, Smith K, et al.2012.SLIC superpixels matched to state-of-the-art superpixel methods IEEE transactions on pattern analysis and machine interaction, 34: 2274-.
Step 2, performing pre-segmentation based on SLIC superpixels; finding adjacent pixel objects for the initial pixel object, if the SLIC labels are the same, merging until all pixel objects with the same labels are merged together:
and in the initial merging process, the initial pixel-level merging is only carried out according to the marking result of SLIC super-pixel pre-segmentation, and the characteristic parameters of the object, such as the mean variance and the like, also need to be counted and updated in the merging process.
Step 3, adding Canny edge straight line information and vegetation mask information; performing edge marking and vegetation marking on a single superpixel segmentation object in the superpixel pre-segmentation result:
after the super-pixel pre-segmentation is combined, marking pre-segmentation intermediate result segmentation objects of all single super-pixels according to a straight line edge extraction result in the orthographic image and a vegetation mask acquisition result, wherein the Canny edge straight line extraction step comprises the following steps:
(1) extracting Canny edge straight lines of the orthoimage, then carrying out 4-communication region connection on the edge detection result, and connecting 8-communication pixels into 4-communication;
(2) and scanning the whole Canny edge detection result image to obtain the interconnected regions of the edge pixels. Since image pixels are usually large, the Canny edge detection result image needs to be blocked in the algorithm implementation. All pixel points which are mutually communicated by 4 in the edge detection result are used for carrying out linear detection based on the RANSAC method, and because the real edge in the image is not necessarily a strict straight line, the distance from the midpoint of the RANSAC to the linear model is set to be 2.5 (pixel unit) in the embodiment of the invention;
(3) and (3) re-rasterizing the pixels which accord with the linear model in the current connection region into an image, checking whether the pixels are mutually communicated or not, and if the pixels are mutually communicated and the number of the pixels in the connection region is greater than a given linear length threshold, determining that the detected pixels are linear. The straight line length threshold value set according to the pixel length of the edge of the actual building in the embodiment of the present invention is 60 (pixel unit).
Canny, among others, can be found in the literature: canny J.1986.A computational aspect to edge detection. IEEE transactions on pattern analysis and machine interaction 679-698; the RANSAC method can be practiced in the literature: fischler M A, Bolles R C.1981.random sample presentation a part for model fixing with applications to image analysis and automatic card graphics. communications of the ACM,24: 381-395; the vegetation mask acquisition method can be specifically implemented in the literature: meyer G E, Net J C.2008.verification of color vision indexes for automatic cropping imaging applications, computers and Electronics in the Agriculture,63: 282-.
Step 4, searching the most similar object of the object under various constraint conditions, judging whether the most similar object is combined with the current object according to the following constraint conditions by combining information such as segmentation scale parameters, vegetation mark information and the like, and repeating iteration until no object which can be combined again exists, wherein the first N isitAt the next iteration, the edge-marked superpixel is skipped to find the most similar object:
in the initial stage of iteration, the edge superpixel does not search for the most similar object, and is used to prevent the possible mismerge of two edge-crossing superpixels in the initial stage of merging, where the initial stage of merging defines: the first 3-4 iterations, 3-4 in this example are empirical values, are performed on a superpixel partition basis.
In the process of searching the most similar object of the current object under various constraint conditions, because the similarity measurement dimension of the shape factors is not uniform with the similarity measurement dimension of the spectrum, the similarity judgment in the merging process of the embodiment of the invention is different from the traditional method, mainly based on the similarity of the spectrum and assisted by the similarity measurement of the shape. And in the merging process, the closer the areas of the adjacent objects are or the longer the common edge of the adjacent objects is, the more likely the adjacent objects are merged in the iteration, an area difference penalty term and a common edge length penalty term need to be added in the similarity calculation process, and the similarity between the two adjacent objects is calculated by the following formula:
Dr＝D·exp(lp -1+|n1-n2|·(n1+n2)-1)
in the above formula, D is the spectral similarity of two adjacent objects, B is the set of data bands participating in the segmentation, wbWeight of b bandHeavy, cibIs the mean value of the b band of the object i, cjbB-band mean, D, of object jrCalculating the similarity of the area difference punishment and the public edge length punishmentpLength of a common edge of two adjacent objects, n1And n2Is the area of two objects. In the implementation, the objects with the spectral similarity metric value smaller than 10 are taken into consideration as very similar objects.
The shape factor constraint is only considered when the current object is very similar to both of the two neighboring objects (the absolute value of the difference in similarity from the two similar neighboring objects is less than 10 or the ratio of the difference in similarity to the smaller similarity is less than 0.1), and the shape factor calculation objective is to select a more appropriate object to be merged among the two neighboring objects that are very close.
The method actually calculates the 'virtually' combined object, and supposing that the current object is Ri, and the objects which are very similar on two spectrums are Rj and Rk respectively, virtually combines Ri and Rj, calculates the shape factor, virtually combines Ri and Rk, calculates the shape factor, and then uses the small shape factor as the object to be combined. After the current object and the adjacent similar object are combined in a 'virtual' mode, the weighted average of the compactness and the smoothness is smaller and better, and the shape factor calculation formula comprehensively considering the compactness and the smoothness is as follows:
h＝wcmpt·hcmpt+wsmooth·hsmooth
the compactness calculation formula is as follows:
the smoothness calculation formula is:
wherein, wcmptTo compact weight, wsmoothFor smoothness weights, l is the perimeter of the merged object, n isThe area of the merged object, b is the perimeter of the smallest circumscribed rectangle of the merged object. In specific implementation, the two weights may be adjusted accordingly according to actual requirements, and the general empirical values may be 0.5 each.
In a bottom-up merge process, finding the most similar object alone does not mean that it must be merged with, since there will always be one of the most similar objects among all the objects that are contiguous. Therefore, it is necessary to further determine whether the most similar object really needs to be merged with the current object, where the determination conditions are as follows:
(1) the most similar objects exceed the scale constraint, namely the set scale parameters, and are not combined;
(2) the most similar object accords with convex model judgment, and is not merged, the convex model can refer to literature SunYing.2008, ground target change detection based on the object [ D ]: Boshi ], Wuhan university;
(3) the most similar object vegetation mask marks are inconsistent and not combined;
(4) the common edge of the most similar object and the current object is a super short edge and is not merged;
(5) the current object is very close to a regular shape, and the most similar object is not in the regular shape after being merged with the current object and is not merged;
(6) the elevation of the most similar object is truncated and not merged;
(7) after the most similar object and the current object are combined, the internal consistency measure exceeds a threshold (spectrum and elevation), and the most similar object and the current object are not combined;
and when the most similar object and the current object are merged and meet the regular shape constraint or the conditions are not met, the current object and the most similar object are considered to be merged. The internal spectrum consistency measurement threshold in the 7 th condition is the maximum value of the standard deviation of the RGB bands in the current image, and the internal elevation consistency measurement threshold is the standard deviation of the elevation in the current image.
And 5, optimally combining the small-area areas which are not combined in the result all the time.
Through the above constraints, small-area objects may not be merged all the time (small-area objects refer to the remaining initial superpixel labeled objects which are not merged), and at this time, it is reasonable to believe that these small objects should be part of some large objects adjacent to the small objects, so that the small objects are finally merged once with a weakened merging condition (i.e., the above 7 un-merging constraints are not judged, and a most similar adjacent object is found and merged), and a final segmentation result is obtained. In addition, in the merging process, when the areas of the two objects are both larger than a certain value, the two objects can be merged only if the two objects are the most similar objects, and the area threshold value can be half of the area of the interested ground object.
Referring to fig. 2, when the method of the embodiment of the present invention is used for unmanned aerial vehicle image segmentation considering the three-dimensional and edge shape characteristics of a building, it can be seen that under various constraint conditions, small-scale stable segmentation objects in a segmentation result are not merged on a large scale. Theoretically, the larger the scale parameter is set, the more the segmentation object in the segmentation result is matched with the real building. The large-scale parameters do not cause the stable building segmentation objects of small scale to continue to be combined wrongly, but for the large-scale ground features themselves, the segmentation objects get closer to the real ground features as the scale parameters increase.
Claims (6)
1. An unmanned aerial vehicle image segmentation method considering three-dimensional and edge shape characteristics of a building is characterized by comprising the following steps:
step 1, setting segmentation scale parameters according to the maximum object area to be extracted, performing mask processing on an invalid area, taking single pixels of an input orthorectified image, an elevation orthorectified image and an SLIC label image as objects, and initializing a segmentation process;
step 2, performing pre-segmentation based on SLIC superpixels; finding adjacent pixel objects for the initial pixel objects, and if SLIC labels are the same, merging until all the pixel objects with the same labels are merged together;
step 3, extracting Canny edge straight lines and vegetation from the orthophoto image, and performing edge marking and vegetation marking on a single superpixel segmentation object in the superpixel pre-segmentation result by using Canny edge straight line information and vegetation mask information;
step 4, searching the most similar object of the current object, further judging whether the most similar object is combined with the current object or not by combining the segmentation scale parameter, the vegetation mark information and other constraint conditions, repeating iteration until no object which can be combined again exists, wherein the first N isitSkipping the superpixels of the edge marks during the secondary iteration to find the most similar object;
and 5, optimally combining the small-area areas which are not combined in the result all the time.
2. The unmanned aerial vehicle image segmentation method considering the three-dimensional and edge shape characteristics of the building as claimed in claim 1, wherein: in step 1, the elevation orthographic image is obtained by,
(1) acquiring the pixel size of an original unmanned aerial vehicle image, generating a virtual image with the same size, wherein the data type of the virtual image is a double type, the number of wave bands is 1, and the initial values of all pixels are set to be infinite;
(2) calculating the actual ground positions corresponding to the four corners of the image according to the internal and external orientation elements and the DTM of the original image, and obtaining the maximum value of the range; in order to ensure that the edges of the virtual image can be calculated to obtain an elevation value, carrying out external expansion on the obtained maximum range to obtain an external expansion range;
(3) acquiring test area DSM data in the external expansion range, sequencing three-dimensional points of a DSM grid from high to low according to elevation, and simultaneously recording the plane coordinate XY of each grid point;
(4) in order to enable the generated virtual image to have the relative shielding condition the same as the real condition, a series of three-dimensional points are generated in a descending manner according to a certain height difference on the plane position of the current grid point and are re-projected on the virtual image one by one until the elevation is reduced to the DTM elevation value of the corresponding position, wherein the re-projection process is calculated by utilizing a collinear equation:
in the above formula, X, Y and Z are three-dimensional point coordinates, X0,y0F is an internal orientation element of the original unmanned aerial vehicle image, ai,bi,ciIs the cosine of the direction of orientation, i is 1,2,3, XS,YS,ZSThe method comprises the following steps that (1) x and y are elements of an outer orientation line of an original unmanned aerial vehicle image, are coordinates of a three-dimensional point re-projected image plane, and are converted into pixel coordinates according to the pixel size of a camera; and at the same pixel position on the re-projected image, taking the maximum value of all elevation values, and correcting the virtual image again by using the internal and external orientation elements and the measurement area DTM of the original image to obtain an elevation orthographic image.
3. The unmanned aerial vehicle image segmentation method considering the three-dimensional and edge shape characteristics of the building as claimed in claim 1, wherein: in step 3, Canny edge straight line information is obtained in the following manner,
(1) extracting Canny edge straight lines of the orthoimage, then carrying out 4-communication region connection on the edge detection result, and connecting 8-communication pixels into 4-communication;
(2) scanning the whole Canny edge detection result image to obtain the interconnected regions of edge pixels;
(3) and (3) re-rasterizing the pixels which accord with the linear model in the current connection region into an image, checking whether the pixels are mutually communicated or not, and if the pixels are mutually communicated and the number of the pixels in the connection region is greater than a given linear length threshold, determining that the detected pixels are linear.
4. The unmanned aerial vehicle image segmentation method considering the three-dimensional and edge shape characteristics of the building as claimed in claim 1, wherein: in step 4, the most similar object of the current object is found by using a method with the spectral similarity measure as a primary measure and the shape similarity measure as a secondary measure, the specific implementation manner is as follows,
(1) when the spectrum similarity is calculated, an area difference penalty term and a public side length penalty term are added in the similarity calculation process, and the similarity between two adjacent objects is calculated by the following formula:
Dr＝D·exp(lp -1+|n1-n2|·(n1+n2)-1)
in the above formula, D is the spectral similarity of two adjacent objects, B is the set of data bands participating in the segmentation, wbIs the weight of the b band, cibIs the mean value of the b band of the object i, cjbB-band mean, D, of object jrCalculating the similarity of the area difference punishment and the public edge length punishmentpLength of a common edge of two adjacent objects, n1And n2Is the area of two objects; then measuring the spectral similarity D between the objectsrThe object smaller than 10 is taken as the most similar object of the current object;
(2) the shape factor calculation aims at selecting a more appropriate merging object from two adjacent objects which are close to each other, actually, the calculation is carried out on the objects after the virtual merging, if the current object is Ri, and the objects which are similar to each other in two spectrums are Rj and Rk respectively, the Ri and the Rj are virtually merged firstly, the shape factor is calculated, then the Ri and the Rk are virtually merged, the shape factor is calculated, and then the object with the small shape factor is used as the most similar object;
after the current object and the adjacent similar object are combined in a 'virtual' mode, the weighted average of the compactness and the smoothness is smaller and better, and the shape factor calculation formula comprehensively considering the compactness and the smoothness is as follows:
h＝wcmpt·hcmpt+wsmooth·hsmooth
the compactness calculation formula is as follows:
the smoothness calculation formula is:
wherein, wcmptTo compact weight, wsmoothAnd f, representing smoothness weight, wherein l is the perimeter of the merged object, n is the area of the merged object, and b is the perimeter of the minimum bounding rectangle of the merged object.
5. The unmanned aerial vehicle image segmentation method considering the three-dimensional and edge shape characteristics of the building as claimed in claim 1, wherein: in step 4, whether the most similar object is really needed to be merged with the current object is judged, and the judgment conditions are as follows:
(1) the most similar objects exceed the scale constraint, namely the segmentation scale parameters set in the step 1, and are not combined;
(2) the most similar objects accord with the judgment of the convex model and are not merged;
(3) the most similar object vegetation mask marks are inconsistent and not combined;
(4) the common edge of the most similar object and the current object is a super short edge and is not merged;
(5) the current object is very close to a regular shape, and the most similar object is not in the regular shape after being merged with the current object and is not merged;
(6) the elevation of the most similar object is truncated and not merged;
(7) after the most similar object and the current object are combined, the internal consistency measure exceeds a threshold value and is not combined;
when the most similar object and the current object are combined and meet the regular shape constraint, or the conditions are not met, the current object and the most similar object are considered to be required to be combined; the internal spectrum consistency measure threshold in the (7) th condition is the maximum value of the standard deviation of the RGB bands in the current image, and the internal elevation consistency measure threshold is the standard deviation of the elevation in the current image.
6. The unmanned aerial vehicle image segmentation method considering the three-dimensional and edge shape characteristics of the building as claimed in claim 1, wherein: n is a radical ofitIs 3 or 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810680919.8A CN108961286B (en) | 2018-06-27 | 2018-06-27 | Unmanned aerial vehicle image segmentation method considering three-dimensional and edge shape characteristics of building |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810680919.8A CN108961286B (en) | 2018-06-27 | 2018-06-27 | Unmanned aerial vehicle image segmentation method considering three-dimensional and edge shape characteristics of building |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108961286A CN108961286A (en) | 2018-12-07 |
| CN108961286B true CN108961286B (en) | 2021-11-16 |
Family
ID=64487288
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810680919.8A Active CN108961286B (en) | 2018-06-27 | 2018-06-27 | Unmanned aerial vehicle image segmentation method considering three-dimensional and edge shape characteristics of building |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108961286B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109919944B (en) * | 2018-12-29 | 2022-09-27 | 武汉大学 | Combined superpixel graph-cut optimization method for complex scene building change detection |
| CN111311750B (en) * | 2020-01-17 | 2022-06-21 | 武汉大学 | Mosaic line network global optimization method based on constrained triangulation network |
| CN111833445A (en) * | 2020-06-24 | 2020-10-27 | 浙江省测绘科学技术研究院 | Regional terrain segmentation and digital elevation model acquisition method based on multi-source data |
| CN111831010A (en) * | 2020-07-15 | 2020-10-27 | 武汉大学 | Unmanned aerial vehicle obstacle avoidance flight method based on digital space slice |
| CN112580493B (en) * | 2020-12-16 | 2021-11-09 | 广东省林业科学研究院 | Plant identification method, device and equipment based on unmanned aerial vehicle remote sensing and storage medium |
| CN116051976B (en) * | 2022-11-23 | 2023-09-19 | 河南理工大学 | Processing method of remote sensing image fused with elevation information |
| CN116309670B (en) * | 2023-05-06 | 2023-08-11 | 中国林业科学研究院资源信息研究所 | Bush coverage measuring method based on unmanned aerial vehicle |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102855627A (en) * | 2012-08-09 | 2013-01-02 | 武汉大学 | City remote sensing image shadow detection method based on spectral characteristic and topological relation |
| CN104091369A (en) * | 2014-07-23 | 2014-10-08 | 武汉大学 | Unmanned aerial vehicle remote-sensing image building three-dimensional damage detection method |
| CN104794688A (en) * | 2015-03-12 | 2015-07-22 | 北京航空航天大学 | Single image defogging method and device based on depth information separation sky region |
| CN105956542A (en) * | 2016-04-28 | 2016-09-21 | 武汉大学 | Structure wiring harness counting and matching high-resolution remote-sensing image road extraction method |
| CN107203757A (en) * | 2017-06-02 | 2017-09-26 | 重庆市地理信息中心 | Building extracting method based on binary features grader |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2013175051A1 (en) * | 2012-05-25 | 2013-11-28 | Nokia Corporation | Method and apparatus for producing a cinemagraph |
-
2018
- 2018-06-27 CN CN201810680919.8A patent/CN108961286B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102855627A (en) * | 2012-08-09 | 2013-01-02 | 武汉大学 | City remote sensing image shadow detection method based on spectral characteristic and topological relation |
| CN104091369A (en) * | 2014-07-23 | 2014-10-08 | 武汉大学 | Unmanned aerial vehicle remote-sensing image building three-dimensional damage detection method |
| CN104794688A (en) * | 2015-03-12 | 2015-07-22 | 北京航空航天大学 | Single image defogging method and device based on depth information separation sky region |
| CN105956542A (en) * | 2016-04-28 | 2016-09-21 | 武汉大学 | Structure wiring harness counting and matching high-resolution remote-sensing image road extraction method |
| CN107203757A (en) * | 2017-06-02 | 2017-09-26 | 重庆市地理信息中心 | Building extracting method based on binary features grader |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108961286A (en) | 2018-12-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108961286B (en) | Unmanned aerial vehicle image segmentation method considering three-dimensional and edge shape characteristics of building | |
| CN110443836B (en) | Point cloud data automatic registration method and device based on plane features | |
| Song et al. | Road extraction using SVM and image segmentation | |
| CN104574347B (en) | Satellite in orbit image geometry positioning accuracy evaluation method based on multi- source Remote Sensing Data data | |
| Gonçalves et al. | HAIRIS: A method for automatic image registration through histogram-based image segmentation | |
| US8401333B2 (en) | Image processing method and apparatus for multi-resolution feature based image registration | |
| JP5830546B2 (en) | Determination of model parameters based on model transformation of objects | |
| CN104200461B (en) | The remote sensing image registration method of block and sift features is selected based on mutual information image | |
| CN111340701B (en) | Circuit board image splicing method for screening matching points based on clustering method | |
| US10062005B2 (en) | Multi-scale correspondence point matching using constellation of image chips | |
| CN111507901B (en) | Aerial image splicing and positioning method based on aerial GPS and scale invariant constraint | |
| CN104751465A (en) | ORB (oriented brief) image feature registration method based on LK (Lucas-Kanade) optical flow constraint | |
| US7747106B2 (en) | Method and system for filtering, registering, and matching 2.5D normal maps | |
| EP3549094A1 (en) | Method and system for creating images | |
| US20120076409A1 (en) | Computer system and method of matching for images and graphs | |
| Urban et al. | Finding a good feature detector-descriptor combination for the 2D keypoint-based registration of TLS point clouds | |
| CN110084743B (en) | Image splicing and positioning method based on multi-flight-zone initial flight path constraint | |
| US11804025B2 (en) | Methods and systems for identifying topographic features | |
| CN108960267A (en) | System and method for model adjustment | |
| JP2005234603A (en) | Map information updating method and map updating device | |
| CN110569861A (en) | Image matching positioning method based on point feature and contour feature fusion | |
| CN114022459A (en) | Multi-temporal satellite image-based super-pixel change detection method and system | |
| US7602943B2 (en) | Image processing apparatus, image processing method, and image processing program | |
| Lu et al. | Multiperspective image stitching and regularization via hybrid structure warping | |
| JP3863014B2 (en) | Object detection apparatus and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |