CN108090913B - Image semantic segmentation method based on object-level Gauss-Markov random field - Google Patents

Image semantic segmentation method based on object-level Gauss-Markov random field Download PDF

Info

Publication number
CN108090913B
CN108090913B CN201711316006.XA CN201711316006A CN108090913B CN 108090913 B CN108090913 B CN 108090913B CN 201711316006 A CN201711316006 A CN 201711316006A CN 108090913 B CN108090913 B CN 108090913B
Authority
CN
China
Prior art keywords
segmentation
level
field
region
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201711316006.XA
Other languages
Chinese (zh)
Other versions
CN108090913A (en
Inventor
郑晨
姚鸿泰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Suzhou Qingchen Technology Co ltd
Original Assignee
Henan University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Henan University filed Critical Henan University
Priority to CN201711316006.XA priority Critical patent/CN108090913B/en
Publication of CN108090913A publication Critical patent/CN108090913A/en
Application granted granted Critical
Publication of CN108090913B publication Critical patent/CN108090913B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/143—Segmentation; Edge detection involving probabilistic approaches, e.g. Markov random field [MRF] modelling
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Probability & Statistics with Applications (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)
  • Coloring Foods And Improving Nutritive Qualities (AREA)

Abstract

The invention provides an image semantic segmentation method based on an object-level Gauss-Markov random field, which comprises the following steps of: performing initialization over-segmentation on the pixel-level image to obtain an object-level image and a region adjacency graph, and respectively defining an adjacent domain system, an observation characteristic field and a segmentation marker field on the region adjacency graph; performing Gauss-Markov modeling on the characteristics of each region of the observation characteristic field and the characteristics of the neighborhood thereof according to the object-level segmentation marker field and the neighborhood system, and constructing an object-level linear regression equation for each region; and respectively carrying out probability modeling on the characteristic field and the marker field, obtaining posterior distribution of the segmented marker field according to Bayes criterion, and obtaining a final segmentation result according to the maximum posterior probability criterion. The method can be used in a system for carrying out semantic segmentation on the images in batches under the background of complex semantics and high spatial resolution, and greatly improves the working efficiency compared with manual detection.

Description

Image semantic segmentation method based on object-level Gauss-Markov random field
Technical Field
The invention relates to the technical field of image semantic segmentation, in particular to an image semantic segmentation method based on an object-level Gauss-Markov random field.
Background
Image semantic segmentation refers to grouping pixels in an image according to different semantics expressed in the image, and the process is performed autonomously by a machine.
With the continuous development of modern sensor manufacturing processes and imaging techniques, the spatial resolution of the processed images is higher and higher, and the number of acquired images grows exponentially, which is inefficient if manual segmentation is used. The previous pixel level segmentation method cannot consider spatial information in a wider range, and a large amount of information is wasted. In recent years, object-level geographic analysis technology has become a hotspot technology for extracting image information, and is applied to image semantic segmentation, so that spatial information in a wider range can be considered. But the interaction relation between the region features is ignored, and the segmentation precision needs to be improved. Therefore, there is a need for an image semantic segmentation method that can not only ensure the full utilization of spatial information, but also consider the interaction between the region features.
Disclosure of Invention
Aiming at the technical problem that the existing image semantic segmentation method cannot fully utilize and guarantee the spatial information and consider the interaction between the regional characteristics, the invention provides the image semantic segmentation method based on the object-level Gauss-Markov random field, which not only ensures the full utilization of the spatial information, but also considers the interaction between the regional characteristics.
In order to achieve the purpose, the technical scheme of the invention is realized as follows: an image semantic segmentation method based on an object-level Gauss-Markov random field is characterized by comprising the following steps of:
the method comprises the following steps: performing initial over-segmentation on the read pixel-level image to obtain an object-level image consisting of over-segmented regions and a corresponding object-level region adjacency graph RAG, and defining a neighborhood system N of the image according to the region adjacency graph RAGOObject-level observation feature field YOAnd object level segmentation marker field XO；
Step two: segmenting the marker field X according to the object levelOAnd neighbor system NOTo the observation characteristic field YOEach region r ofiThe features of (a) and the features of its neighborhood are modeled by Gauss-Markov, and the structure is constructed for each region riI 1, …, l;
step three: respectively to observe characteristic field YOAnd dividing the mark field XOCarrying out probability modeling and obtaining a segmentation marking field X according to Bayes criterionOThe iterative segmentation is updated by applying the maximum posterior probability criterion and the final segmentation is obtained by solving the updated iterative segmentation.
The specific implementation steps of the first step are as follows:
1) performing position index set definition and pixel-level observation feature set definition on an input high-spatial-resolution three-channel image I (R, G, B), and assuming that the resolution of the image I (R, G, B) is mxn, obtaining: position index set S ═ SxyX is not less than m and not more than 1 ≦ x; y is more than or equal to 1 and less than or equal to n, and a pixel-level observation feature set
Figure BDA0001503809700000021
Wherein,
Figure BDA0001503809700000022
representing the observed eigenvalue of the pixel at position s,
Figure BDA0001503809700000023
the values of R, G, B components of the image are respectively, m is the length of the image, n is the width of the image, and (x, y) are the position coordinates of pixel points in the image;
2) and performing over-segmentation processing on the pixel level image by using a mean-shift method according to the set minimum area: over-dividing the image I (R, G, B) into l minimum areas sminEach region is assigned a label, resulting in a label matrix Ls＝{lsS belongs to S, wherein the element l belongs to SsE {1, …, l }, S e S; thus, the position index set R ═ R of the object-level image is obtained1,r2,…,rlWherein, the area ri＝{s|ls＝i}；
3) Obtaining an object-level region adjacency graph G (R, E) according to the over-segmentation processing, wherein the position index set R is an object-level element, each element represents an over-segmentation region, and E (E)ijI is less than or equal to 1, j is less than or equal to l represents an adjacency relation, and the element eijIndicating the region riNeutralization region rjNumber of adjacent pixels, eijNot equal to 0 and only if the element RiAnd RjAre adjacent;
4) defining an object-level observation characteristic field on a region adjacency graph G
Figure BDA0001503809700000024
And object level segmentation marker field
Figure BDA0001503809700000025
Wherein,
Figure BDA0001503809700000026
indicating the region riIs observed as | riI denotes the region riThe number of internal pixel points; xOIs a random field that is generated by the field,
Figure BDA0001503809700000027
is a random variable that is a function of time,
Figure BDA0001503809700000028
wherein K is a division classA classification set, wherein k is the preset number of segmentation classifications;
5) and giving an object-level neighborhood system according to the object-level region adjacency graph G ═ R, E):
Figure BDA0001503809700000029
wherein,
Figure BDA00015038097000000210
the second step comprises the following specific steps:
1) in the region adjacency graph G ═ (R, E), the area parameter whose target level element is the number of pixels included in each over-divided region can be obtained from the position index set R, and the area matrix RS ═ RS can be obtainediI is more than or equal to 1 and less than or equal to l, wherein RSi＝|ri|；
2) Let xOIs an object level segmentation marker field XOAccording to xOObtaining the characteristic mean value and the characteristic covariance matrix of each category, and the realization process is as follows:
(a) realization of the known object-level segmentation marker field as xOCalculating the segmentation class corresponding to each pixel point in the original image, namely a pixel-level segmentation mark matrix
Figure BDA0001503809700000031
Wherein
Figure BDA0001503809700000032
(b) Respectively calculating characteristic mean values m ═ miI is not less than 1 and not more than k and a characteristic covariance matrix sigma { ∑ sigmai|1≤i≤k}：
Figure BDA0001503809700000033
Figure BDA0001503809700000034
3) For each object level element riGiven its segmentation marker implementationIs composed of
Figure BDA0001503809700000035
Then, a linear regression equation is constructed as follows:
Figure BDA0001503809700000036
Figure BDA00015038097000000311
wherein e isi～N(0,∑h) Is a gaussian white noise.
The concrete method of the third step is as follows:
2) for object level observation feature field YOInstead of directly modeling the joint probabilities for observed features, each object-level element r is modelediAnd performing combined modeling on residual terms in the constructed object-level linear regression equation to obtain a likelihood function of the characteristic field, namely:
Figure BDA0001503809700000037
2) object level segmentation marker field XOProbability modeling is carried out, and the Markov-Gibbs equivalence shows that the object-level segmentation mark field conforms to Gibbs distribution, and the prior distribution of the mark field is obtained as follows:
Figure BDA0001503809700000038
Figure BDA0001503809700000039
Figure BDA00015038097000000310
Figure BDA0001503809700000041
Figure BDA0001503809700000042
Figure BDA0001503809700000043
wherein Z is a normalization constant, U (x)O) Representing a split field implementation as xOEnergy of time, K is the set of segmentation classes, V2(. cndot.) is a function of the potential energy of the group, given by the Potts model, i.e.:
Figure BDA0001503809700000044
3) the posterior distribution of the marker field from Bayes' formula can be found as:
Figure BDA0001503809700000045
therefore, the optimal result of the segmentation mark is converted into the segmentation mark field XOThe problem of posterior distribution maximization, namely:
Figure BDA0001503809700000046
and updating the segmentation marks through loop iteration to finally obtain a segmentation result.
The specific implementation process of the loop iteration is as follows:
5) firstly, a pixel level MRF method is realized by a classical ICM algorithm, and the segmentation class of each pixel point is obtained, namely the pixel level segmentation field result: x is the number ofP＝{xsL S belongs to S, and then the realization of the object segmentation mark field of the initial iteration is obtained
Figure BDA0001503809700000047
Wherein
Figure BDA0001503809700000048
mode is a mode function；
6) Implementation of segmentation of the marking field at the t-th step from the object level
Figure BDA0001503809700000049
Obtaining the characteristic mean value corresponding to each category according to the following formula
Figure BDA00015038097000000410
Sum feature covariance
Figure BDA00015038097000000411
Figure BDA00015038097000000412
Figure BDA00015038097000000413
7) Separately computing each object-level element riThe object-level linear regression equation of (1):
Figure BDA0001503809700000051
8) respectively calculating the object and characteristic field probability and the marker field probability, and updating the segmentation markers object by object, specifically:
Figure BDA0001503809700000052
the invention has the beneficial effects that: a semantic segmentation method for the formed RGB image with high spatial resolution is provided; the method can be used for semantic segmentation of batch processing of the RGB images with high spatial resolution, the segmentation efficiency is far higher than the traditional manual segmentation level, and the efficiency is higher than that of most of the existing object-oriented segmentation modes; the fixed value is directly given to the parameter to be estimated in the linear regression equation, so that the method is simple, convenient and quick to calculate and high in precision.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a flow chart of the present invention.
FIG. 2 is a flow chart of initialization of the present invention.
FIG. 3 is an exemplary diagram of the initialization process of the present invention.
FIG. 4 is a flow chart of the linear regression equation construction of the present invention.
FIG. 5 is a flow chart of the joint modeling of the present invention.
FIG. 6 is a simulation diagram of the experiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without inventive effort based on the embodiments of the present invention, are within the scope of the present invention.
As shown in fig. 1, an image semantic segmentation method based on an object-level Gauss-Markov random field includes the following steps:
the method comprises the following steps: performing initial over-segmentation on the read pixel-level image to obtain an object-level image consisting of over-segmented regions and a corresponding object-level region adjacency graph RAG, and defining a neighborhood system N of the image according to the region adjacency graph RAGOObject-level observation feature field YOAnd object level segmentation marker field XO。
In order to perform object-level image analysis and improve algorithm efficiency, initial over-segmentation is required to obtain a region adjacency graph RAG. The region adjacency graph RAG is obtained from the spatial relationship between the respective over-segmented regions of the object-level image. The method for carrying out initialization over-segmentation on the image is a mean-shift method with the minimum area factor. And according to the pixel-level image characteristics, obtaining object-level image representation by using a mean-shift method related to a minimum area parameter (namely the number of pixels contained in the over-segmentation region), and finally obtaining the object-level image characteristics. As shown in fig. 2, the specific implementation steps are as follows:
1) performing position index set definition and pixel-level observation feature set definition on an input high-spatial-resolution three-channel image I (R, G, B), and assuming that the resolution of the image I (R, G, B) is mxn, obtaining: position index set S ═ SxyX is not less than m and not more than 1 ≦ x; y is more than or equal to 1 and less than or equal to n, and a pixel-level observation feature set
Figure BDA0001503809700000061
Wherein,
Figure BDA0001503809700000062
representing the observed eigenvalue of the pixel at position s,
Figure BDA0001503809700000063
the values of the R, G, B components of the image, m the length of the image, n the width of the image, and (x, y) the position coordinates of the pixel points in the image.
2) And performing over-segmentation processing on the pixel level image by using a mean-shift method according to the set minimum area: over-dividing the image I (R, G, B) into l minimum areas sminEach region is assigned a label, resulting in a label matrix Ls＝{lsS belongs to S, wherein the element l belongs to SsE {1, …, l }, S e S. Thus, the position index set R ═ R of the object-level image is obtained1,r2,…,rlWherein, the area ri＝{s|lsI }. As shown in fig. 3(a), the processing result is grayed, and the line in the figure is the result of division.
3) The object level region adjacency graph G is obtained by the over-segmentation processing (R, E). Wherein the position index set R is an object level element, each element representing an over-segmented region. E ═ EijI is less than or equal to 1, j is less than or equal to l represents an adjacency relation, and the element eijIndicating the region riNeutralization region rjNumber of adjacent pixels, eijNot equal to 0 and only if the element RiAnd RjAre adjacent.
4) Defining an object-level observation characteristic field on a region adjacency graph G
Figure BDA0001503809700000064
Object level segmentation marker field
Figure BDA0001503809700000065
Wherein,
Figure BDA0001503809700000066
indicating the region riIs observed as | riI denotes the region riThe number of pixels in the column. XOIs a random field that is generated by the field,
Figure BDA0001503809700000067
is a random variable representing the over-segmented region riThe classification of (2) is performed,
Figure BDA0001503809700000068
where K is a set of segmentation classes and K is a predetermined number of segmentation classes.
5) And giving an object-level neighborhood system according to the object-level region adjacency graph G ═ R, E):
Figure BDA0001503809700000069
wherein,
Figure BDA00015038097000000610
fig. 3(b) is an enlarged view of a part of the rectangular frame in fig. 3(a), and the neighborhood labeling of each region in fig. 3(b) is as shown in fig. 3 (c).
Step two: dividing each region r of the marker field XO and the neighborhood system NO to the observation characteristic field YO according to the object leveliThe features of (a) and the features of its neighborhood are modeled by Gauss-Markov, and the structure is constructed for each region riI 1, …, l.
The object-level linear regression equation uses the area size and the boundary length of the object-level elements as parameters of the linear regression equation, and constructs a linear regression equation for each object-level element, as shown in fig. 4, the specific steps are as follows:
1) in the region adjacency graph G ═ (R, E), the number of pixels included in each over-divided region can be obtained from the position index set R, and the number of pixels is regarded as the area parameter of the object-level element, so that the area matrix RS ═ RS is obtainediI is more than or equal to 1 and less than or equal to l, wherein RSi＝|ri|。
2) Let x beOIs an object level segmentation marker field XOAccording to xOObtaining the characteristic mean value and the characteristic covariance matrix of each category, and the realization process is as follows:
(a) realization of the known object-level segmentation marker field as xOCalculating the segmentation class corresponding to each pixel point in the original image, namely a pixel-level segmentation mark matrix
Figure BDA0001503809700000071
Wherein
Figure BDA0001503809700000072
(b) Respectively calculating a characteristic mean value m { mi |1 ≦ i ≦ k } and a characteristic covariance matrix sigma { ∑ sigma { [i|1≤i≤k}：
Figure BDA0001503809700000073
Figure BDA0001503809700000074
3) For each object level element riGiven its segmentation marker implemented as xiAfter O, a linear regression equation was constructed as follows:
Figure BDA0001503809700000075
Figure BDA0001503809700000076
wherein, for the convenience of calculation, assume ei～N(0,∑h) Is a gaussian white noise.
Step three: respectively to observe characteristic field YOAnd dividing the mark field XOCarrying out probability modeling and obtaining a segmentation marking field X according to Bayes criterionOThe iterative segmentation is updated by applying the maximum posterior probability criterion and the final segmentation is obtained by solving the updated iterative segmentation.
Probabilistic modeling includes constructing an observed feature field Y from error terms in an object-level linear regression equationOThe multivariate normal distribution and the construction of a segmentation marker field X by adopting a Potts modelOGibbs distribution of (1). The final segmentation result is: and updating iterative segmentation by using Gibbs distributed sampling, and finally outputting a convergence solution. As shown in fig. 5, the specific operation is as follows:
1) for object level observation feature field YOInstead of directly modeling the joint probabilities for observed features, each object-level element r is modelediAnd performing combined modeling on residual terms in the constructed object-level linear regression equation to obtain a likelihood function of the characteristic field, namely:
Figure BDA0001503809700000081
2) object level segmentation marker field XOAnd (3) performing probability modeling, wherein the mark field has Markov property, and the mark field accords with Gibbs distribution according to Markov-Gibbs equivalence, so that the prior distribution of the mark field is obtained as follows:
Figure BDA0001503809700000082
Figure BDA0001503809700000083
Figure BDA0001503809700000084
Figure BDA0001503809700000085
Figure BDA0001503809700000086
Figure BDA0001503809700000087
wherein Z is a normalization constant, U (x)O) Representing a split field implementation as xOEnergy of time, V2(. cndot.) is a function of the potential energy of the group, given by the Potts model, i.e.:
Figure BDA0001503809700000088
3) the posterior distribution of the marker field from Bayes' formula can be found as:
Figure BDA0001503809700000089
therefore, the optimal result of the segmentation mark is converted into the segmentation mark field XOThe problem of posterior distribution maximization, namely:
Figure BDA0001503809700000091
and updating the segmentation marks through loop iteration to finally obtain a result. The specific loop iteration process is as follows:
1) firstly, a pixel-level MRF (Markovrandom field) method is realized by a classical ICM (iterative condition model) algorithm, and the classification of each pixel point is obtained, namely the pixel-level segmentation field result: x is the number ofP＝{xsL S belongs to S, and then the realization of the object segmentation mark field of the initial iteration is obtained
Figure BDA0001503809700000092
Wherein
Figure BDA0001503809700000093
I.e. for the over-divided region riThe segmentation mark is the mode of the segmentation mark of the internal pixel point.
2) Implementation of segmentation of the marking field at the t-th step from the object level
Figure BDA0001503809700000094
Obtaining the characteristic mean value corresponding to each category according to the following formula
Figure BDA0001503809700000095
Sum feature covariance
Figure BDA0001503809700000096
Figure BDA0001503809700000097
Figure BDA0001503809700000098
3) Separately computing each object-level element riThe object-level linear regression equation of (1):
Figure BDA0001503809700000099
4) respectively calculating the object and characteristic field probability and the marker field probability, and updating the segmentation markers object by object, specifically:
Figure BDA00015038097000000910
the present invention is a platform that is operated such that core i3-4160@3.6GHz, RAM: 4G, 64-bit win10 system, 2015a version matlab, the color image of aerial image 1024_1 is as shown in fig. 6(a1) (the color image is grayed), real manual segmentation is as shown in fig. 6(a2), ICM method is used for image 1024_1, β is 0.5, the color image of the segmentation result is as shown in fig. 6(a3), GMRF method is used for image 1024_1, β is 0.5, the color image of the segmentation result is as shown in fig. 6(a4), mrmrmrmrmrf method is used for image 1024_1, wavelet decomposition is three-layer, β is 0.5, the color image of the segmentation result is as shown in fig. 6(a5), the image 1024_1 is as shown in fig. 6, and mr5 is as shown in fig. 256, r 5 is used for image 1024 b, r 5, the image is as shown in fig. 6(a 583) and the image is as shown in fig. 26, r 3, r 5, r 3, r 5, r 3, r 5, r 5, r, r 5, r 5, r 5, r 5, r g. 7, r, G5, r 5, g. 7, G5, G3, G5, G3, G7, G7, G6 a r, G7, G7, G7, G7, G.
TABLE 1 Kappa coefficient of segmentation results
Figure BDA0001503809700000101
TABLE 2 Total Accuracy of segmentation results (OA Accuracy)
Figure BDA0001503809700000102
As can be seen from the data in fig. 6 and tables 1-2, the segmentation accuracy of the present invention is the best. The aerial image contains more texture information, and the sub-objects in the same class have larger differences in spectral values, while sub-objects in different classes may have similar spectral values. For example, in the urban sector, roofs and courtyards have different spectral values, but the spectral values of trees in the urban and forest sectors are similar. For these reasons, there are many finely divided misclassifications for the three pixel-based approaches. Compared to the pixel-based approach, the object-based approach treats the over-segmented regions as basic units, thus significantly optimizing the segmentation accuracy. The OMRF method models the feature domain using the probability distribution of the features of the object, while the OGMRF-RC method models the feature domain using the probability distribution of the residual terms in the object-level linear regression equation. The OGMRF-RC method has the advantage that the influence of spectral variation between the same classes on the segmentation in the iterative process can be reduced. For example, in the upper half of fig. 6(a7), the large bare land and forest are accurately divided into idle sections, unlike the OMRF divided into house sections in fig. 6(a 6).
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.

Claims (4)

1. An image semantic segmentation method based on an object-level Gauss-Markov random field is characterized by comprising the following steps of:
the method comprises the following steps: performing initial over-segmentation on the read pixel-level image to obtain an object-level image consisting of over-segmented regions and a corresponding object-level region adjacency graph RAG, and defining a neighborhood system N of the image according to the region adjacency graph RAGOObject-level observation feature field YOAnd object level segmentation marker field XO；
Step two: segmenting the marker field X according to the object levelOAnd neighbor system NOTo the observation characteristic field YOEach region r ofiThe features of (a) and the features of its neighborhood are modeled by Gauss-Markov, and the structure is constructed for each region ri1.. times.l;
step three: respectively to observe characteristic field YOAnd dividing the mark field XOGo to outlineRate modeling and obtaining a segmentation marking field X according to Bayes criterionOThe posterior distribution of the segmentation is realized by updating iterative segmentation by applying a maximum posterior probability criterion and solving the iterative segmentation to obtain final segmentation;
the second step comprises the following specific steps:
1) in the region adjacency graph G ═ (R, E), the area parameter whose target level element is the number of pixels included in each over-divided region can be obtained from the position index set R, and the area matrix RS ═ RS can be obtainediI is more than or equal to 1 and less than or equal to l, wherein RSi＝|ri|；
2) Let xOIs an object level segmentation marker field XOAccording to xOObtaining the characteristic mean value and the characteristic covariance matrix of each category, and the realization process is as follows:
(a) realization of the known object-level segmentation marker field as xOCalculating the segmentation class corresponding to each pixel point in the original image, namely a pixel-level segmentation mark matrix
Figure FDA0002351894940000011
Wherein
Figure FDA0002351894940000012
(b) Calculating the characteristic mean value mu ═ mu respectivelyiI is not less than 1 and not more than k and a characteristic covariance matrix sigma { ∑ sigmai|1≤i≤k}：
Figure FDA0002351894940000013
Figure FDA0002351894940000014
Wherein,
Figure FDA0002351894940000015
representing the observation characteristic value of the pixel point at the position s;
3) for each object level element riGiven its segmentation markingNow is
Figure FDA0002351894940000016
Then, a linear regression equation is constructed as follows:
Figure FDA0002351894940000017
Figure FDA0002351894940000021
wherein,
Figure FDA0002351894940000022
is a gaussian white noise; element eijIndicating the region riNeutralization region rjThe number of adjacent pixels.
2. The method for image semantic segmentation based on the object-level Gauss-Markov random field according to claim 1, wherein the first step is implemented by the following steps:
1) performing position index set definition and pixel-level observation feature set definition on an input high-spatial-resolution three-channel image I (R, G, B), and assuming that the resolution of the image I (R, G, B) is mxn, obtaining: position index set S ═ SxyX is not less than m and not more than 1 ≦ x; y is more than or equal to 1 and less than or equal to n, and a pixel-level observation feature set
Figure FDA0002351894940000023
Wherein,
Figure FDA0002351894940000024
representing the observed eigenvalue of the pixel at position s,
Figure FDA0002351894940000025
the values of R, G, B components of the image are respectively, m is the length of the image, n is the width of the image, and (x, y) are the position coordinates of pixel points in the image;
2) and performing over-segmentation processing on the pixel level image by using a mean-shift method according to the set minimum area: over-dividing the image I (R, G, B) into l minimum areas sminEach region is assigned a label, resulting in a label matrix Ls＝{lsS belongs to S, wherein the element l belongs to SsE {1, …, l }, S e S; thus, the position index set R ═ R of the object-level image is obtained1,r2,…,rlWherein, the area ri＝{s|ls＝i}；
3) Obtaining an object-level region adjacency graph G (R, E) according to the over-segmentation processing, wherein the position index set R is an object-level element, each element represents an over-segmentation region, and E (E)ijI is less than or equal to 1, j is less than or equal to l represents an adjacency relation, and the element eijIndicating the region riNeutralization region rjNumber of adjacent pixels, eijNot equal to 0 and only if the element RiAnd RjAre adjacent;
4) defining an object-level observation characteristic field on a region adjacency graph G
Figure FDA0002351894940000026
And object level segmentation marker field
Figure FDA0002351894940000027
Wherein,
Figure FDA0002351894940000028
indicating the region riIs observed as | riI denotes the region riThe number of internal pixel points; xOIs a random field that is generated by the field,
Figure FDA0002351894940000029
is a random variable that is a function of time,
Figure FDA00023518949400000210
wherein, K is a segmentation class set, and K is a preset segmentation class number;
5) given according to the object level region adjacency graph G ═ R, EObject level neighborhood system:
Figure FDA00023518949400000211
wherein,
Figure FDA00023518949400000212
3. the image semantic segmentation method based on the object-level Gauss-Markov random field according to claim 1, wherein the specific method of the third step is as follows:
1) for object level observation feature field YOInstead of directly modeling the joint probabilities for observed features, each object-level element r is modelediAnd performing combined modeling on residual terms in the constructed object-level linear regression equation to obtain a likelihood function of the characteristic field, namely:
Figure FDA0002351894940000031
wherein,
Figure FDA0002351894940000032
indicating the region riThe observed characteristic of (a);
2) object level segmentation marker field XOProbability modeling is carried out, and the Markov-Gibbs equivalence shows that the object-level segmentation mark field conforms to Gibbs distribution, and the prior distribution of the mark field is obtained as follows:
Figure FDA0002351894940000033
Figure FDA0002351894940000034
Figure FDA0002351894940000035
Figure FDA0002351894940000036
Figure FDA0002351894940000037
Figure FDA0002351894940000038
wherein Z is a normalization constant, U (x)O) Representing a split field implementation as xOEnergy of time, K is the set of segmentation classes, V2(. cndot.) is a function of the potential energy of the group, given by the Potts model, i.e.:
Figure FDA0002351894940000039
3) the posterior distribution of the marker field from Bayes' formula can be found as:
Figure FDA00023518949400000310
therefore, the optimal result of the segmentation mark is converted into the segmentation mark field XOThe problem of posterior distribution maximization, namely:
Figure FDA0002351894940000041
and updating the segmentation marks through loop iteration to finally obtain a segmentation result.
4. The image semantic segmentation method based on the object-level Gauss-Markov random field according to claim 3, wherein the loop iteration is realized by the following specific process:
1) firstly, a pixel level MRF method is realized by a classical ICM algorithm, and the segmentation class of each pixel point is obtained, namely the pixel level segmentation field result: x is the number ofP＝{xsL S belongs to S, and then the realization of the object segmentation mark field of the initial iteration is obtained
Figure FDA0002351894940000042
Wherein
Figure FDA0002351894940000043
mode is a mode function;
2) implementation of segmentation of the marking field at the t-th step from the object level
Figure FDA0002351894940000044
Obtaining the characteristic mean value corresponding to each category according to the following formula
Figure FDA0002351894940000045
Sum feature covariance
Figure FDA0002351894940000046
Figure FDA0002351894940000047
Figure FDA0002351894940000048
Wherein, | riI denotes the region riThe number of internal pixel points;
3) separately computing each object-level element riThe object-level linear regression equation of (1):
Figure FDA0002351894940000049
4) respectively calculating the object and characteristic field probability and the marker field probability, and updating the segmentation markers object by object, specifically:
Figure FDA00023518949400000410
CN201711316006.XA 2017-12-12 2017-12-12 Image semantic segmentation method based on object-level Gauss-Markov random field Active CN108090913B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201711316006.XA CN108090913B (en) 2017-12-12 2017-12-12 Image semantic segmentation method based on object-level Gauss-Markov random field

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201711316006.XA CN108090913B (en) 2017-12-12 2017-12-12 Image semantic segmentation method based on object-level Gauss-Markov random field

Publications (2)

Publication Number Publication Date
CN108090913A CN108090913A (en) 2018-05-29
CN108090913B true CN108090913B (en) 2020-06-19

Family

ID=62173916

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201711316006.XA Active CN108090913B (en) 2017-12-12 2017-12-12 Image semantic segmentation method based on object-level Gauss-Markov random field

Country Status (1)

Country Link
CN (1) CN108090913B (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108830319B (en) * 2018-06-12 2022-09-16 北京合众思壮科技股份有限公司 Image classification method and device
CN109615637A (en) * 2019-01-16 2019-04-12 中国科学院地理科学与资源研究所 A kind of improved remote sensing image Hybrid Techniques
CN111210433B (en) * 2019-04-16 2023-03-03 河南大学 Markov field remote sensing image segmentation method based on anisotropic potential function
CN110136143A (en) * 2019-05-16 2019-08-16 河南大学 Geneva based on ADMM algorithm multiresolution remote sensing image segmentation method off field

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103295236B (en) * 2013-05-29 2015-12-23 湘潭大学 Markov multiple features random field models construction method and brain MR image cutting techniques thereof
CN106600611B (en) * 2016-12-23 2020-04-14 西安电子科技大学 SAR image segmentation method based on sparse triple Markov field
CN106951830B (en) * 2017-02-23 2020-12-18 北京联合大学 Image scene multi-object marking method based on prior condition constraint
CN107180434B (en) * 2017-05-23 2019-10-18 中国地质大学（武汉） Polarization SAR image segmentation method based on super-pixel and fractal net work evolution algorithmic

Also Published As

Publication number Publication date
CN108090913A (en) 2018-05-29

Similar Documents

Publication Publication Date Title
Wang et al. Superpixel segmentation: A benchmark
CN106981068B (en) A kind of interactive image segmentation method of joint pixel pait and super-pixel
CN108537239B (en) Method for detecting image saliency target
CN108052966B (en) Remote sensing image scene automatic extraction and classification method based on convolutional neural network
CN108090913B (en) Image semantic segmentation method based on object-level Gauss-Markov random field
CN113706487B (en) Multi-organ segmentation method based on self-supervision characteristic small sample learning
CN102013017B (en) Method for roughly sorting high-resolution remote sensing image scene
Montazer et al. An improved radial basis function neural network for object image retrieval
CN103400151B (en) The optical remote sensing image of integration and GIS autoregistration and Clean water withdraw method
Alidoost et al. A CNN-based approach for automatic building detection and recognition of roof types using a single aerial image
CN110163239B (en) Weak supervision image semantic segmentation method based on super-pixel and conditional random field
CN106340016A (en) DNA quantitative analysis method based on cell microscope image
CN103985112B (en) Image segmentation method based on improved multi-objective particle swarm optimization and clustering
CN103345744B (en) A kind of human body target part automatic analytic method based on many images
CN112784869B (en) Fine-grained image identification method based on attention perception and counterstudy
He et al. Interactive segmentation techniques: algorithms and performance evaluation
CN111695636A (en) Hyperspectral image classification method based on graph neural network
CN111950406A (en) Finger vein identification method, device and storage medium
CN110853070A (en) Underwater sea cucumber image segmentation method based on significance and Grabcut
Wang et al. Adaptive nonlocal random walks for image superpixel segmentation
Li et al. A fast level set algorithm for building roof recognition from high spatial resolution panchromatic images
Khan et al. A modified adaptive differential evolution algorithm for color image segmentation
Farid et al. DOST: A distributed object segmentation tool
CN116343048A (en) Accurate land block boundary extraction method and system for plain crop type complex region
CN113409335B (en) Image segmentation method based on strong and weak joint semi-supervised intuitive fuzzy clustering

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right

Effective date of registration: 20240117

Address after: Building 43, Zone A, Energy Conservation and Environmental Protection Innovation Park, No. 199 Hongwu Avenue, Tangqiao Town, Zhangjiagang City, Jiangsu Province, 215000

Patentee after: Suzhou Qingchen Technology Co.,Ltd.

Address before: No.85, Minglun street, Shunhe District, Kaifeng City, Henan Province

Patentee before: Henan University

TR01 Transfer of patent right