CN111242918B - Image segmentation method and system based on Kalman filtering and Markov random field - Google Patents
Image segmentation method and system based on Kalman filtering and Markov random field Download PDFInfo
- Publication number
- CN111242918B CN111242918B CN202010026633.5A CN202010026633A CN111242918B CN 111242918 B CN111242918 B CN 111242918B CN 202010026633 A CN202010026633 A CN 202010026633A CN 111242918 B CN111242918 B CN 111242918B
- Authority
- CN
- China
- Prior art keywords
- image
- pixel
- background
- value
- filtering
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
- G06T2207/20032—Median filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The invention provides an image segmentation method and system based on Kalman filtering and Markov random fields, wherein the image segmentation method comprises the following steps: carrying out prior processing on the image, partitioning the image, acquiring a background pixel and an object image pixel to be detected in each sub-image, setting a Markov random field, setting a Kalman filter, setting parameters, and initializing the Kalman filter; filtering each line and each column in two directions perpendicular to each other on the whole image, carrying out local statistics on the marked region of the Kalman filter under the Markov random field neighborhood system, and calculating the current pixelI(I,j)The distance from the background to the object to be measured, and the current pixel is judgedI(I,j)And (4) belonging to the category, fusing the marking results in the two directions, and removing abnormal points by median filtering to obtain a defect segmentation result. The technical scheme of the invention improves the segmentation precision of the image of the object to be detected and avoids the occurrence of the hole phenomenon.
Description
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to an image segmentation method and system based on Kalman filtering and Markov random fields.
Background
Alum is basically adopted by waterworks as a water purifying agent, whether the treated water is qualified or not depends on the addition amount of the alum, aluminum hydroxide colloid precipitate is generated in the alum hydrolysis process, impurities in the water are precipitated together to form alum flocs, the amount and volume of the alum flocs in the water reflect the addition condition of the alum, and therefore the alum flocs in a sedimentation tank of the waterworks need to be detected to realize the detection of the addition amount of the alum. Currently, there are three main detection methods: (1) The artificial detection method is characterized in that after the alum is dissolved in water, the state of the alum floc formed in a sedimentation tank is observed by naked eyes of people, so as to judge whether the input amount of the alum is proper or not; (2) Detecting the electrolyte concentration by using a probe, and judging whether the alum input amount is proper or not according to the electrolyte concentration; (3) And acquiring an alum blossom image in the sedimentation tank by using a CCD (charge coupled device) camera, and judging the alum input amount according to an image processing result.
With the development of CCD technology and image processing technology, the online real-time alum blossom detection method becomes the mainstream technology of online alum blossom detection at present. The alum blossom detection method based on the optical image processing technology is gradually replacing manual detection due to low requirements on detection efficiency and working environment. The purpose of image segmentation is to divide an image into regions with various characteristics and extract interested regions, so as to provide an important and key basis for subsequent target feature extraction, target identification and classification and subsequent high-level processing. The state of alum floc in the sedimentation tank is observed based on a machine vision method to judge the alum input amount. The method comprises the following processing flows: the method comprises the steps of CCD camera image acquisition, alum blossom image segmentation, image preprocessing and alum blossom particle size and number identification. Wherein the alum blossom image segmentation is an important basis of the whole monitoring system. Image segmentation is to separate the object of interest from the background image. The target in the alum blossom image is alum blossom. The existing Alum image segmentation algorithms mainly comprise: 1) Automatic threshold thresholding methods, such as: performing threshold segmentation on inter-class variance, performing iterative optimal threshold segmentation, and performing two-dimensional information entropy threshold segmentation; 2) And image segmentation based on edge detection, such as Sobel edge extraction threshold segmentation method.
Because various interference factors, such as uneven illumination, can automatically operate all the day and night 24 hours all the day, the day and night change of natural illumination can generate image gray level concentration; also, for example, in the case of a serious contrast insufficiency, the reflection and refraction of water and the sun shadow phenomenon due to a change in natural light tend to be disordered and thus cannot be controlled in a program. Due to various noises, the collected alum blossom images have the characteristics of low contrast and random distribution of alum blossom particles, which brings difficulty to positioning of alum blossom. In the existing threshold segmentation algorithm, the global threshold segmentation algorithm is simplest and has the fastest operation speed, but the global threshold segmentation algorithm is not suitable for the non-uniform image and has poor non-uniform resistance. Local threshold processing is adopted for image blocks, and obvious connecting lines are easily generated at the joints of the blocks. The local threshold segmentation technique has the disadvantages that when the image background is complex and in a dynamic process, the selection of the optimal threshold is difficult, and the method is easily interfered by noise because the method only considers the gray scale of the pixel itself and ignores the spatial correlation characteristic of the image.
Edge detection image segmentation techniques rely essentially on image discontinuities. The serial edge detection and segmentation technology has the problems of low starting point selection and low execution efficiency; boundary discontinuity and non-single-pel nature of parallel edge detection. And the method based on edge detection is easy to generate a hole phenomenon and needs subsequent filling. Therefore, in practical application, if the method is to obtain the ideal segmentation effect, other auxiliary means or corresponding post-processing technology must be used.
The defect segmentation method is usually set according to a specific detection object, and is not necessarily suitable for defect image segmentation similar to alum blossom.
Disclosure of Invention
Aiming at the characteristics that expression forms of the alum blossom have diversity, such as uneven texture thickness, irregular particle size and shape, local inconsistency of the background due to external interference factors and the characteristics of a camera, and the like, the invention discloses an image segmentation method and an image segmentation system based on Kalman filtering and Markov random fields, which are used for fully utilizing the gray scale and the spatial information of image pixels, so that the segmentation precision of the alum blossom image is improved, and the optimal segmentation result graph is provided for subsequent alum blossom particle statistics and alum blossom state identification.
In contrast, the technical scheme adopted by the invention is as follows:
an image segmentation method based on Kalman filtering and Markov random fields comprises the following steps:
step S1, performing prior processing on an image, partitioning the image containing an object to be detected into a plurality of sub-images, setting the size of the image as M multiplied by N, M as a row, N as a column, (I, j) representing a space position, and I (I, j) representing a gray value of the sub-image at the space position (I, j); collecting background pixels and image pixels of an object to be detected in each sub-image, and calculating the mean value and standard deviation u of the background pixels b1 ,u b2 ...,u bn ,σ b1 ,σ b2 ,...,σ bn And calculating the mean and standard deviation u of the pixels of the object to be measured a1 ,u a2 ...,u an ,σ a1 ,σ a2 ,...,σ an Establishing a statistical table;
step S2, a Markov random field is set up, and the size of the neighborhood of the Markov random field is (2L + 1) × (2R + 1), wherein 2L +1 is the transverse length and 2R +1 is the longitudinal length; the values of the integers L and R may take an empirical value depending on the actual image situation.
S3, setting a Kalman filter, and setting parameters of the Kalman filter, including a measurement matrix H k State transition matrix F k State noise covariance Q k Measuring the variance R k (ii) a Wherein the content of the first and second substances,
H k =(0 1),
k represents the kth pixel, and T is the sampling time;
the algorithm of the Kalman filter is as follows:
wherein, Y k A measurement vector of pixel values for the kth pixel in the image; system control matrix G k =0, so
K k Is a Kalman gain matrix, ζ k In order to measure the residual error (innovation),is an innovation covariance matrix; ζ represents a unit k Obeying zero mean and variance of S k Of a Gaussian distribution, i.e.. Zeta k ~N(0,S k ) To do soComplianceDistribution, n Y = dim (ζ) is the dimensionality of ζ;
s4, initializing the Kalman filter by adopting the initial background and the mean value and the standard deviation of the image of the object to be detected obtained in the step S1;
s5, filtering each row and each column of the whole image in two mutually perpendicular directions by adopting a Kalman filter to obtain a residual value zeta of the kth pixel on each row and each column k Stored in a matrix; the kalman filter is used to filter the whole image in other directions. By adopting the technical scheme, the whole image is filtered in the column direction and the row direction,and each column and each row can be operated in parallel, so that the efficiency is improved.
S6, under the Markov random field neighborhood system, carrying out local statistics on the marked region of the Kalman filter, and updating the mean value u of the pixel of the k-1 step of the background according to the filtering result before the k-1 step in two directions respectively b Sum standard deviation σ b And the mean value u of the pixels of the step k-1 of the object to be measured a And standard deviation σ a ;
Step S7, calculating the distance D between the pixel I (I, j) and the background by adopting the following formula respectively according to two directions b And a distance D to the object to be measured a :
Wherein u is b 、σ b Local mean and standard deviation of the background, u, respectively a 、σ a Local mean and standard deviation of the background, respectively; when D is b (i,j)≥D a (i, j), if the marked pixel (i, j) is 1;
step S8, aiming at two directions, respectively according to the statistical characteristics of the measurement residual error and the distances between the background of the local area and the object to be measured, judging the category of the current pixel according to the following conditions:
wherein, the label 0 represents the background pixel, and the label 1 represents the target pixel;
step S9, fusing the marking results in the two directions by adopting the following functions to obtain a calibrated binary image,
if the pixel (i, j) is a background pixel, it is marked as f (i, j) =0, and if the pixel (i, j) is a target pixel of the object to be measured, it is marked as f (i, j) =1.
As a further improvement of the present invention, the image segmentation method based on kalman filtering and markov random field further includes:
and S10, performing median filtering on the calibrated binary image by adopting an n-order neighborhood system, and removing isolated points or noise points to obtain a final defect segmentation image of the object to be detected.
As a further improvement of the present invention, the setting of the kalman filter includes:
let I (I, j) denote the pixel values of j row of I column of the image, where I =1,2, 3.., M, j =1,2, 3.., N;
the pixels of each line of the image being taken as a measurement sequence Y k K =1, 2.., M, one kalman filter per line;
the pixels of each column of the image being taken as a measurement sequence Y k K =1,2,., N, one kalman filter per column;
the initial state values of each kalman filter are set according to the area of the sub-image to which they belong.
As a further improvement of the present invention, the step S4 of initializing the kalman filter includes:
filter of ith row of image, when filtering from left to right, initial state value is set toWherein u is b0 =u k ,ifI(i,0)∈A k ,
The filter of the jth column of the image, when filtering from top to bottom, has an initial state value set toWherein u is b0 =u k ,ifI(0,j)∈A k 。
As a further improvement of the present invention,
with one measurement residual ζ at each position for each filter filtering k ,ζ k Obeying a mean value of zero and a variance of S k Gaussian distribution of (i.e.. Zeta.) k ~N(0,S k ) To is thatComplianceDistribution (chi-square distribution), n Y = dim (ζ) is the dimension of ζ. When the temperature is higher than the set temperatureWhen the abnormal value appears at the position of the image, the abnormal value is marked as a defective pixel, and the residual value zeta of the two directions k Respectively stored in matrix sigma up Sum Σ left In matrix sigma up 、∑ left Is the same size as the image, where α is the level of saliency. 1-alpha is the confidence probability that,the value of (c) can be looked up from the chi-square table.
As a further improvement of the present invention, the local statistics of the marked regions of the kalman filter under the markov random field neighborhood system in step S6 includes:
for filtering in the left-to-right direction, the current point (i, j), the region Σ already marked by the Kalman filter col Updating the mean value and variance of the last background and the pixel of the object to be detected by calculating the mean value and standard deviation of the background and the pixel of the object to be detected in L x (2R + 1) pixels on the left part of the pixel, and if the template does not contain one type of pixels, continuing to use the last statistical value;
for filtering in the up-down direction, the current point (i, j), the region Σ that the Kalman filter has marked row Updating the last back by calculating the mean and standard deviation of the background and the pixels of the object to be measured in (2L + 1) xR pixels at the upper part of the computerIf the template does not contain one type of pixels, the type continues to use the last statistical value.
By adopting the technical scheme, firstly, regional background sampling is carried out on the acquired image, the image of each region is used as a subgraph, and a background region statistical table is established and used as initialization information of a Kalman filter. Since the background gray scale of each sub-image changes slowly and can be regarded as following a gaussian distribution, a Constant (CV) model is used in the technical scheme to describe the gray scale change characteristic. The kalman filter is a kalman filter based on a Constant (CV) model. Then filtering the image from different directions through a Kalman filter of a Constant (CV) model, finding out heterogeneous regions according to the statistical distribution characteristics of residual errors from Kalman filtering, and judging abnormal points. This filter may also be referred to as a residual detector based on Kalman filtering. Because the detector has a hysteresis phenomenon, a hole phenomenon easily occurs, namely, a hole appears in the defect. The reason is that the whole inside of part of defect targets is stable, and the gray level change is not abnormal, so that abnormal points are not marked inside the defects, and the hole phenomenon is presented.
And simultaneously introducing a Markov Random Field (MRF), establishing a neighborhood system, carrying out local statistics on two types of pixels, namely an object to be detected and a background, of a region which is already filtered by the Kalman filter, and combining the spatial correlation of the pixels with a Kalman filtering residual error detector to mark a current pixel defect target.
The MRF model can effectively describe the local statistical characteristics of the image by considering the condition distribution of each pixel point about a group of adjacent pixel points. The introduction of the historical local pixel distribution with the spatial correlation can avoid the hole phenomenon and eliminate the hysteresis phenomenon to a certain extent. The historical spatial correlation pixels are determined by a matrix template representing a neighborhood system, and statistics of the background and the target object to be measured are updated in real time through the matrix template. And judging the category of the current point according to the distance between the current pixel and the background and the object to be detected. And finally, eliminating the noise points by using median filtering under an adjacent domain system to obtain the final image segmentation result of the object to be detected.
The invention also discloses an image segmentation system based on the Kalman filtering and the Markov random field, which comprises the following steps:
the image priori information processing module is used for partitioning an image containing an object to be detected into a plurality of sub-images, the size of the image is M multiplied by N, M is a row, N is a column, (I, j) represents a space position, and I (I, j) represents a gray value of a first sub-image at the space position (I, j); collecting background pixels and image pixels to be detected in each sub-image, and calculating the mean value and standard deviation u of the background pixels b1 ,u b2 ...,u bn ,σ b1 ,σ b2 ,...,σ bn And calculating the mean and standard deviation u of the pixels of the object to be measured a1 ,u a2 ...,u an ,σ a1 ,σ a2 ,...,σ an Establishing a statistical table;
a Markov random field establishing module, wherein the neighborhood size of the Markov random field is (2L + 1) x (2R + 1), wherein 2L +1 is the transverse length, and 2R +1 is the longitudinal length;
a Kalman filter module, wherein an algorithm of the Kalman filter is as follows:
wherein, Y k A measurement vector of pixel values for a kth pixel in the image; system control matrix G k =0;
K k Is a Kalman gain matrix, ζ k To measure the residual error (innovation),is an innovation covariance matrix; zeta k Obey zero mean and variance of S k Of a Gaussian distribution, i.e.. Zeta k ~N(0,S k ) To do soComplianceDistribution, n Y = dim (ζ) is the dimensionality of ζ;
a Kalman filter parameter setting module for setting parameters of the Kalman filter, including a measurement matrix H k State transition matrix F k State noise covariance Q k Measuring the variance R k (ii) a Wherein, the first and the second end of the pipe are connected with each other,
H k =(0 1),
k represents the kth pixel, and T is the sampling time;
the Kalman filter initialization module is used for initializing the Kalman filter by adopting the initial background obtained by the image prior information processing module and the mean value and standard deviation of the image of the object to be detected;
a filtering module for filtering the whole image in each row and each column in two mutually perpendicular directions by adopting a Kalman filter to obtain a residual value zeta of the kth pixel in each row and each column k Stored in a matrix;
a local statistic module for locally counting the marked region of the Kalman filter under the Markov random field neighborhood system, and updating the mean value u of the pixel of the k-1 step of the background according to the filtering result before the k-1 step for two directions b Sum standard deviation σ b And the mean value u of the pixels of the step k-1 of the object to be measured a And standard deviation σ a ;
A distance calculation module from the pixel I (I, j) to the background and the object to be measured: for two directions, the following formula is respectively adopted to calculate the distance D from the pixel I (I, j) to the background b And a distance D to the object to be measured a :
Wherein u is b 、σ b Local mean and standard deviation of the background, u, respectively a 、σ a Local mean and standard deviation of the background, respectively; when D is present b (i,j)≥D a (i, j), if the marked pixel (i, j) is 1;
and the classification module of the current pixel belongs to the two directions, and judges the class of the current pixel according to the following conditions according to the statistical characteristics of the measurement residual error and the distances between the current pixel and the local area background and the object to be measured respectively:
wherein, the 0 label represents a background pixel, and the 1 label represents a target pixel;
the marking result fusion module fuses the marking results in the two directions by adopting the following functions to obtain a calibrated binary image,
if the pixel (i, j) is a background pixel, it is marked as f (i, j) =0, and if the pixel (i, j) is a target pixel of the object to be measured, it is marked as f (i, j) =1.
As a further improvement of the present invention, the system further comprises:
and performing median filtering on the calibrated binary image based on a median filtering module in the n-order neighborhood structure, and removing noise points to obtain a final defect segmentation image of the object to be detected.
As a further improvement of the present invention, the kalman filter setting module is set by using the following method:
let I (I, j) denote the pixel values of j row of I column of the image, where I =1,2, 3.., M, j =1,2, 3.., N;
the pixels of each line of the image being taken as a measurement sequence Y k K =1, 2.., M, one kalman filter per line;
the pixels of each column of the image being taken as a measurement sequence Y k K =1, 2.., N, one kalman filter per column;
the initial state values of each kalman filter are set according to the area of the sub-image to which they belong.
As a further improvement of the present invention, the kalman filter initialization module is initialized by the following method:
the filter of the ith row of the image, when filtering from left to right, the initial state value is set toWherein u b0 =u k ,ifI(i,0)∈A k ,
The filter of the jth column of the image is set to the initial state value when filtering from top to bottomWherein u b0 =u k ,ifI(0,j)∈A k 。
As a further improvement of the invention, there is a measurement residual ζ at each position for each filter filtering k When it comes toWhen the image is abnormal at the position, the image is marked as a defective pixel, and the residual value zeta of the two directions k Respectively stored in matrix sigma up Sum Σ left In matrix sigma up 、∑ left Is the same size as the image.
As a further improvement of the present invention, the local statistic module performs local statistics on the region marked by the kalman filter by the following steps:
for filtering in the left-to-right direction, the current point (i, j), the region Σ marked by the kalman filter col Updating the mean value and variance of the last background and the pixel of the object to be detected by calculating the mean value and standard deviation of the background and the pixel of the object to be detected in L x (2R + 1) pixels on the left part of the pixel, and if no one type of pixels exist under the template, continuing the last statistical value for the type;
for filtering in the top-down direction, the current point (i, j), the region Σ marked by the kalman filter row And updating the mean value and the variance of the last background and the pixels of the object to be detected by calculating the mean value and the standard deviation of the background and the pixels of the object to be detected in (2L + 1) multiplied by R pixels at the upper part, and if no one type of pixels exist under the template, continuing the last statistical value.
The invention also discloses an electronic device comprising a display screen, wherein the electronic device comprises a processor and a memory which are connected, and the processor is used for executing a computer program stored in the memory so as to execute the image segmentation method based on the Kalman filtering and the Markov random field.
The invention also discloses a computer-readable storage medium comprising a computer program configured to implement, when invoked by a processor, the steps of the kalman filtering and markov random field based image segmentation method according to any one of the preceding claims.
Compared with the prior art, the invention has the following beneficial effects:
by adopting the technical scheme of the invention, the gray scale and the spatial information of the image pixel are fully utilized, the influence of the non-uniformity of the image background on the image segmentation is solved, the hole phenomenon generated by some methods is avoided, the segmentation precision of the image of the object to be detected is improved, the optimal segmentation result graph is provided for the subsequent particle statistics and state identification of the object to be detected, and an important application basis is further provided for the automatic detection and monitoring system of the alum blossom state of the waterworks.
Drawings
Fig. 1 is a schematic diagram of an n-th order neighborhood system of the present invention.
FIG. 2 is a flow chart of the Kalman filtering and Markov random field based image segmentation method of the present invention.
Fig. 3 shows an original image before alumen ustum treatment according to an embodiment of the present invention.
FIG. 4 shows alum flocs after its division treatment in an embodiment of the present invention, in which white is alum flocs.
Detailed Description
Preferred embodiments of the present invention are described in further detail below.
Firstly, carrying out regional background sampling on an acquired image, taking the image of each region as a subgraph, and establishing a background region statistical table as initialization information of a Kalman filter. The background gray scale of each sub-graph changes slowly and can be regarded as following Gaussian distribution, and a Constant (CV) model is used for describing the gray scale change characteristic. Then filtering the image from different directions through a Kalman filter of a Constant Velocity (CV) model, finding out heterogeneous regions according to the statistical distribution characteristics of residual errors from Kalman filtering, and judging out the abnormal points. This filter may also be referred to as a Kalman filter based residual detector. Because the detector has a hysteresis phenomenon, a hole phenomenon easily occurs, namely, a hole appears in the defect. The reason is that the whole inside of part of defect targets is stable, and the gray level change is not abnormal, so that abnormal points are not marked inside the defects, and the hole phenomenon is presented. Therefore, a Markov Random Field (MRF) is introduced, a neighborhood system is established, local statistics is carried out on two types of pixels, namely alum blossom and water background, in a region which is filtered by a Kalman filter, and current pixel defect (alum blossom) target marking is carried out by combining the spatial correlation of the pixels with a Kalman filtering residual detector. The MRF model can effectively describe the local statistical characteristics of the image by considering the condition distribution of each pixel point about a group of adjacent pixel points. The introduction of the historical local pixel distribution with the spatial correlation can avoid the hole phenomenon and eliminate the hysteresis phenomenon to a certain extent. The historical spatial correlation pixels are determined by a matrix template representing a neighborhood system, by which the statistics of the background and target alum flocs are updated in real time. And judging the category of the current point according to the distance between the current pixel and the background and the alumen ustum. And finally, eliminating noise points by using median filtering under an adjacent domain system to obtain a final alum blossom segmentation result.
The specific embodiment is implemented as follows:
1. collecting prior information of the image, namely processing the image in blocks, namely dividing the image into n rectangular areas, wherein each area is marked as A 1 ,A 2 ,...,A n Each block is seen as a sub-picture. Since the sub-image non-uniformity is much weaker than across the large map, it can be approximately considered uniform. And collecting background pixels in each sub-image, performing statistical analysis, and establishing a regional background statistical table. Assuming that water and alum blossom approximate to follow Gaussian distribution, their parameters, mean and standard deviation were calculated. The mean and standard deviation of the background water in each area are recorded as u b1 ,u b2 ...,u bn ,σ b1 ,σ b2 ,...,σ bn (ii) a The mean and standard deviation of alum flocs in each area are recorded as u a1 ,u a2 ...,u an ,σ a1 ,σ a2 ,...,σ an (ii) a These statistics are used to initialize the initialization of the Kalman filter. For one alum blossom detection system, the work can be only done once.
2. Let M × N be the size of a certain sub-image, M be a row, N be a column, (I, j) denote the spatial position, and I (I, j) denote the gray-level value of the image at spatial position (I, j). Defining the distance of the pixel to the background class asThe distance from the alum blossomWherein u b And σ b Mean and variance of background class, and u a And σ a Mean and standard deviation of alum blossom defect target.
3. When the image is modeled by a Constant (CV) model, the state equation and the measurement equation are
Z k =F k-1 Z k-1 +G k-1 U k-1 +W k-1
Y k =H k Z k +V k (1.1)
Where k denotes the kth step, here the kth pixel, the state vectorIn (1)An estimate of the pixel value representing a position in the image, z having no specific meaning here, a system control matrix G k =0, measurement vector Y is the pixel value in the image, measurement matrix H k = 01, state transition matrix:
t is the sampling time, where T =1.W k And V k The noise is system process noise and measurement noise, and is zero mean gaussian noise, and state noise covariance:
δ k-j is Kronecker-delta, i.e., delta if k ≠ j k-j =0, δ if k = j k-j =1。
The Kalman filtering method based on the CV model is described as follows:
wherein, K k Is a Kalman gain matrix, ζ k To measure the residual error (innovation),is an innovation covariance matrix. The residual (innovation) ζ from kalman filtering can be deduced under linear gaussian conditions k Obey a mean of zero mean with a variance of S k Gaussian distribution of (i.e.. Zeta.) k ~N(0,S k ) To do soComplianceDistribution, n Y And (= dim (ζ)) is the dimensionality of ζ. Generally, whenWhen the system state is very likely to change (abnormal), the image is marked as a defective pixel when an abnormal value appears at the position, wherein alpha is the significance level, 1-alpha is the confidence probability,the value of (c) can be looked up from the chi-square table. The detection performance can be improved by properly selecting alpha.
4. And (4) filtering the whole graph in two mutually perpendicular directions by using a CV model Kalman filter, namely performing multichannel simultaneous synchronous equidirectional filtering in the transverse direction and the longitudinal direction respectively.
Let the image size be M × N, M be a row, N be a column, (I, j) denote the spatial position, and I (I, j) denote the gray value of the first sub-image at spatial position (I, j). The transverse direction can be from left to right, or from right to left, and the longitudinal direction can be from top to bottom, or from bottom to top. The horizontal direction and the vertical direction are respectively taken as a detection direction, and the images are filtered from left to right and from top to bottom without selection.
The pixels of each line of the image being taken as a measurement sequence Y k K =1,2.., M. And each line is provided with a Kalman filter of a CV model, M lines are provided with M filters, and the M filters are synchronously in the same direction from left to right and respectively filter corresponding line pixels. Defective pixels in each row are detected based on statistical characteristics of the filter residue values of each step of the Kalman filter system. This residual follows a chi-squared distribution and is therefore detected by its chi-squared value. Setting a matrix sigma of the same size as the image MXN row To store the detected defects of alum blossom. When a pixel at image (i, j) is determined to be defective, the element at position (i, j) in the matrix is marked as 1, indicating a defective pixel; otherwise, the label is 0, representing a background pixel.
Similarly, each column of the image is taken as a measurement vector, each column is provided with a CV model Kalman filter and N filters, the filters are synchronously in the same direction, and chi-square detection based on Kalman filtering residual errors is carried out from top to bottom to obtain a defect matrix sigma col The size of which is the same as the image. When a pixel at image (i, j) is determined to be defective, the element at position (i, j) in the matrix is marked as 1, indicating a defective pixel; otherwise, the flag is 0, indicating a background pixel.
There are 1 filter in each column and each row, so there are M + N kalman filters, and the parameter settings of each filter are the same as above, and their initial state values are set as follows according to the sub-graph region they belong to:
let I (I, j) denote the pixel values of the j row in the ith column of the image, where I =1,2, 3.., M, j =1,2, 3.., N, the M + N kalman filters are initialized as follows:
(1) Filter of ith row of image, when filtering from left to right, initial state value is set toWherein
u b0 =u k ,if I(i,0)∈A k (1.8)
(2) The filter of the jth column of the image is set to the initial state value when filtering from top to bottomWherein
u b0 =u k ,if I(0,j)∈A k (1.9)
After initialization, the Kalman filter carries out filtering in two directions on the image. There is a measurement residual ζ at each position for each filter filtering k When is coming into contact withWhen the image is abnormal at the position, the image is marked as a defective pixel, and the residual value zeta k Stored in matrix sigma up In the matrix sigma up Is the same size as the image. Also for the left-to-right direction, zeta is calculated during the filtering process k Is saved to matrix sigma left In (1).
5. Because the residual error detector has hysteresis, the detection is only carried out from one direction, the detected defect position lags behind and deviates from the real position, and the hole phenomenon occurs. Historical category data is introduced, a Markov random field neighborhood system is set, the size of a matrix representing a neighborhood is (2L + 1) x (2R + 1), 2L +1 is the transverse length, and 2R +1 is the longitudinal length. Transverse 2R +1 CV detectors related to the region are synchronously operated; likewise, longitudinal 2L +1 CV detectors are also synchronized. The values of the integers L and R may take an empirical value depending on the actual image situation.
6. Under the Markov random field neighborhood system, the neighborhood size is a matrix of (2L + 1) × (2R + 1). Local statistics are performed on the regions that the Kalman filter has marked.
For filtering in the left-to-right direction, the current point (i, j), the region Σ already marked by the Kalman filter col And updating the mean value and the variance of the background and the alum blossom in the last time by calculating the mean value and the standard deviation of the background and the alum blossom in L x (2R + 1) pixels on the left part of the pixel, and if no one type of pixels exist under the template, the type continues to use the last statistical value. Then respectively calculating the distance from the current pixel to the center of the background asThe distance to the alum target isWherein u is b ,σ b Local mean and standard deviation of the background, u, respectively a ,σ a Local mean and standard deviation of the background, respectively, as D b (i,j)≥D a (i, j), the flag pixel (i, j) is 1.
Similarly, for filtering in the top-down direction, the Kalman filter has marked the region ∑ row And updating the mean value and the variance of the last background and alumen ustum by calculating the mean value and the standard deviation of the background and alumen ustum in (2L + 1) multiplied by R pixels on the upper part of the pixel, and if no one type of pixel exists under the template, the type continues to use the last statistical value. Then respectively calculating the distance from the current pixel to the center of the background asThe distance to the alum target isWherein u is b ,σ b Local mean and standard deviation of the background, u, respectively a ,σ a Local mean and standard deviation of the background, respectively, as D b (i,j)≥D a (i, j), the flag pixel (i, j) is 1. Based on the result of the Kalman filter detector, fusing the spatial information of the image, and marking the image, wherein the marking function is as follows:
wherein, the 0 label represents the background image element, and the 1 label represents the target image element. K =1 denotes Kalman filtering in the left-to-right direction, and K =2 denotes filtering in the top-to-bottom direction.
7. The result of the two-direction processing is fused by f (i, j), and if the pixel (i, j) is a background pixel, it is recorded as f (i, j) =0, and if the pixel (i, j) is a target, it is recorded as f (i, j) =1. The fusion function is:
8. and finally, performing median filtering on the calibrated binary image by adopting the following n-order neighborhood system, and removing noise points to obtain a final alum blossom defect segmentation image. Fig. 1 shows a 5 th-order neighborhood system with n = 5.
The flow chart for segmenting the alum blossom image of the water works by adopting the method is shown in figure 2, the alum blossom image is extracted by adopting the method of the flow chart, the graphs before and after the alum blossom are respectively shown in figures 3 and 4, and the comparison shows that the method of the embodiment realizes the optimal extraction of the alum blossom, has high segmentation precision of the alum blossom image, does not find a hole phenomenon, is convenient for subsequent alum blossom particle statistics and alum blossom state identification, and can be used for automatic detection and automatic control of the alum addition amount in the water works.
The foregoing is a more detailed description of the invention in connection with specific preferred embodiments and it is not intended that the invention be limited to these specific details. For those skilled in the art to which the invention pertains, several simple deductions or substitutions can be made without departing from the spirit of the invention, and all shall be considered as belonging to the protection scope of the invention.
Claims (8)
1. An image segmentation method based on Kalman filtering and Markov random fields is characterized in that: which comprises the following steps:
the method comprises the following steps that S1, prior processing is carried out on an image, the image containing an object to be detected is partitioned into a plurality of sub-images; let the image size be M × N, M be a row, N be a column, (I, j) denote the pixel spatial position, and I (I, j) denote the grayscale value of the sub-image at the pixel spatial position (I, j); collecting background pixels and object pixels to be detected in each sub-image, and calculating the mean value u of the background pixels b1 ,u b2 ...,u bn And standard deviation σ b1 ,σ b2 ,...,σ bn And calculating the mean value u of the pixels of the object to be measured a1 ,u a2 ...,u an And standard deviation σ a1 ,σ a2 ,...,σ an Establishing a statistical table;
step S2, a Markov random field is set, and the neighborhood size of the Markov random field is (2L + 1) × (2R + 1), wherein 2L +1 is the transverse length and 2R +1 is the longitudinal length; the values of the integers L and R are empirical values according to the actual image condition;
s3, setting a Kalman filter, and setting parameters of the Kalman filter, including a measurement matrix H k State transition matrix F k State noise covariance Q k Measuring variance R k (ii) a Wherein the content of the first and second substances,
H k =[0 1],
k represents the kth pixel, and T is the sampling time;
the algorithm of the Kalman filter is as follows:
wherein Z is k Is a state variable at time k of the system, U k For the control input of the system at time k, U k-1 Control input to the system at time k-1; system control matrix G k =0;Y k A measurement vector of pixel values for a kth pixel in the image;
for Z estimated using measurements at and before time k k The value is a posteriori estimate obtained by calculating its conditional expectation value, i.e.
For predicting Z by measured values not including k times before k times k The value being an a priori estimate, i.e.
Z predicted by a measurement value not including the time k-1 before the time k-1 k-1 The value being an a priori estimate, i.e.
P denotes the error covariance of the estimate, P k-1 For state estimationEstimation error covariance of (2):
K k is a Kalman gain matrix, ζ k In order to measure the residual error,is an innovation covariance matrix;
s4, initializing the Kalman filter by adopting the initial background and the mean value and the standard deviation of the image of the object to be detected obtained in the step S1;
s5, filtering each row and each column of the whole image in two mutually perpendicular directions by adopting a Kalman filter to obtain a residual error value of the kth pixel in each row and each column, and storing the residual error values in a matrix;
s6, under a Markov random field neighborhood system, carrying out local statistics on the marked region of the Kalman filter, and updating the mean value and the standard deviation of the pixels at the k-1 step of the background and the mean value and the standard deviation of the pixels at the k-1 step of the object to be detected according to the filtering results before the k-1 step in two directions;
step S7, aiming at two directions, respectively adopting the following formula to calculate the distance D between the pixel space position (i, j) and the background b And a distance D to the object to be measured a :
Wherein u is b 、σ b Local mean and standard deviation of the background, u, respectively a 、σ a Respectively is the local mean value and standard deviation of the object to be measured; when D is b (i,j)≥D a (i, j), the spatial position (i, j) of the marking pixel is marked as 1;
step S8, aiming at two directions, judging the category of the current pixel according to the following conditions according to the statistical characteristics of the measurement residual error and the distances between the current pixel and the local area background and the object to be measured respectively:
wherein, the 0 label represents a background pixel, and the 1 label represents a target pixel;
there is a measurement residual ζ at each position for each filter filtering k ,ζ k Obey a zero mean with a variance of S k Of a Gaussian distribution, i.e.. Zeta k ~N(0,S k ) To do soComplianceDistribution, n Y = dim (ζ) is the dimensionality of ζ; when in useWhen the abnormal value appears at the position of the image, the image is marked as a defective pixel, and the residual value in the direction from top to bottom is stored in the matrix sigma up In the left-to-right direction, the residual error value is stored in sigma left In matrix sigma up 、∑ left Is the same as the size of the image, where α is the level of saliency;
s9, fusing the marking results in the two directions by adopting the following functions to obtain a calibrated binary image,
if the pixel spatial position (i, j) is a background pixel, it is marked as f (i, j) =0, and if the pixel spatial position (i, j) is a target pixel of the object to be measured, it is marked as f (i, j) =1.
2. The kalman filtering and markov random field based image segmentation method according to claim 1, further comprising:
and S10, performing median filtering on the calibrated binary image by adopting an n-order neighborhood system, and removing isolated points or noise points to obtain a final defect segmentation image of the object to be detected.
3. The method of image segmentation based on kalman filtering and markov random fields according to claim 1, wherein the step S3 of setting the kalman filter comprises:
let I (I, j) denote the pixel values of j row of I column of the image, where I =1,2, 3.., M, j =1,2, 3.., N;
the pixels of each line of the image being taken as a measurement sequence Y k K =1,2,., M, one kalman filter per row;
the pixels of each column of the image are taken as a measurement sequence Y' k K =1,2,., N, one kalman filter per column; the initial state values of each kalman filter are set according to the area of the sub-image to which they belong.
4. The kalman filtering and markov random field based image segmentation method of claim 1, wherein the step S6 of locally counting the regions marked by the kalman filter under the markov random field neighborhood system comprises:
for filtering in the left-to-right direction, the current pixel spatial position (i, j), the region Σ marked by the kalman filter col Updating the mean value and variance of the last background and the pixel of the object to be detected by calculating the mean value and standard deviation of the background and the pixel of the object to be detected in L x (2R + 1) pixels on the left part of the pixel, and if the template does not contain one type of pixels, continuing to use the last statistical value;
for filtering in the top-down direction, the current pixel spatial position (i, j), the region Σ that the kalman filter has marked row And updating the mean value and the variance of the last background and the pixel of the object to be detected by calculating the mean value and the standard deviation of the background and the pixel of the object to be detected in (2L + 1) multiplied by R pixels at the upper part, and if the template does not contain one of the pixels, the last statistical value is adopted for the class.
5. An image segmentation system based on kalman filtering and markov random fields, characterized in that it comprises:
the image priori information processing module is used for partitioning an image containing an object to be detected into a plurality of sub-images, and setting the size of the image as M multiplied by N, M is a row, N is a column, (I, j) represents a pixel space position, and I (I, j) represents a gray value of the sub-image at the pixel space position (I, j); collecting background pixels and object pixels to be detected in each sub-image, and calculating the mean value u of the background pixels b1 ,u b2 ...,u bn And standard deviation σ b1 ,σ b2 ,...,σ bn And calculating the mean value u of the pixels of the object to be measured a1 ,u a2 ...,u an And standard deviation σ a1 ,σ a2 ,...,σ an Establishing a statistical table;
a Markov random field establishing module, wherein the neighborhood size of the Markov random field is (2L + 1) x (2R + 1), wherein 2L +1 is the transverse length, and 2R +1 is the longitudinal length; the values of the integers L and R are empirical values according to the actual image condition;
a Kalman filter module, wherein the algorithm of the Kalman filter is as follows:
wherein the content of the first and second substances,for the initial estimation of the error covariance, Y k A measurement vector of pixel values for the kth pixel in the image; system control matrix G k =0;
K k Is a Kalman gain matrix, ζ k In order to measure the residual error,is an innovation covariance matrix;
wherein, Z k Is a state variable at time k of the system, U k For the control input of the system at time k, U k-1 Control input to the system at time k-1; system control matrix G k =0;Y k A measurement vector of pixel values for the kth pixel in the image;
for Z estimated using measurements at and before time k k The value being an a posteriori estimate obtained by calculating its conditional expectation, i.e.
For predicting Z by measured values not including k times before k times k The value being an a priori estimate, i.e.
Z predicted by a measurement value not including the time k-1 before the time k-1 k-1 The value being an a priori estimate, i.e.
P denotes the error covariance of the estimate, P k-1 For state estimationEstimation error covariance of (2):
K k is a Kalman gain matrix, ζ k In order to measure the residual error,is an innovation covariance matrix; a Kalman filter parameter setting module for setting parameters of the Kalman filter, including a measurement matrix H k State transition matrix F k State noise covariance Q k Measuring the variance R k (ii) a Wherein the content of the first and second substances,
H k =[0 1],
k represents the kth pixel, and T is the sampling time;
the Kalman filter initialization module is used for initializing the Kalman filter by adopting the initial background obtained by the image prior information processing module and the mean value and standard deviation of the image of the object to be detected;
the filtering module is used for filtering the whole image in each row and each column in two mutually perpendicular directions by adopting a Kalman filter to obtain a residual error value of the kth pixel in each row and each column, and storing the residual error value in a matrix;
the local statistical module is used for carrying out local statistics on the marked region of the Kalman filter under a Markov random field neighborhood system, and updating the mean value and the standard deviation of the pixel of the k-1 step of the background and the mean value and the standard deviation of the pixel of the k-1 step of the object to be detected according to the filtering results before the k-1 step in two directions respectively;
the distance calculation module from the pixel space position (i, j) to the background and the object to be measured: for two directions, the following formula is respectively adopted to calculate the distance D from the pixel space position (i, j) to the background b And a distance D to the object to be measured a :
Wherein u is b 、σ b Local mean and standard deviation of the background, u, respectively a 、σ a Respectively the local mean value and the standard deviation of the object to be detected; when D is present b (i,j)≥D a (i, j), the spatial position (i, j) of the marking pixel is marked as 1;
and the category classification module of the current pixel judges the category of the current pixel according to the following conditions according to the statistical characteristics of the measurement residual error and the distances between the current pixel and the local area background and the object to be measured respectively in two directions:
wherein, the label 0 represents the background pixel, and the label 1 represents the target pixel;
with one measurement residual ζ at each position for each filter filtering k ,ζ k Obeying a mean value of zero and a variance of S k Gaussian distribution of (i.e.. Zeta.) k ~N(0,S k ) To is thatComplianceDistribution, n Y = dim (ζ) is the dimensionality of ζ; when in useWhen the abnormal value appears at the position of the image, the image is marked as a defective pixel, and the residual value in the direction from top to bottom is stored in the matrix sigma up In, the residual value in the left-to-right direction is saved as ∑ left In the matrix sigma up 、∑ left Size of (2) and size of imageWherein a is a significance level;
the marking result fusion module fuses the marking results in the two directions by adopting the following functions to obtain a calibrated binary image,
if the pixel spatial position (i, j) is a background pixel, it is marked as f (i, j) =0, and if the pixel spatial position (i, j) is a target pixel of the object to be measured, it is marked as f (i, j) =1.
6. The Kalman filtering and Markov random field based image segmentation system of claim 5 further comprising:
and performing median filtering on the calibrated binary image based on a median filtering module in the n-order neighborhood structure, and removing noise points to obtain a final defect segmentation image of the object to be detected.
7. An electronic device comprising a display screen, wherein the electronic device comprises a processor and a memory coupled to each other, wherein the processor is configured to execute a computer program stored in the memory to perform the kalman filtering and markov random field based image segmentation method according to any one of claims 1 to 4.
8. A computer-readable storage medium, characterized in that it comprises a computer program configured to implement, when invoked by a processor, the steps of the Kalman filtering and Markov random field based image segmentation method according to any one of claims 1 to 4.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010026633.5A CN111242918B (en) | 2020-01-10 | 2020-01-10 | Image segmentation method and system based on Kalman filtering and Markov random field |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010026633.5A CN111242918B (en) | 2020-01-10 | 2020-01-10 | Image segmentation method and system based on Kalman filtering and Markov random field |
Publications (2)
Publication Number | Publication Date |
---|---|
CN111242918A CN111242918A (en) | 2020-06-05 |
CN111242918B true CN111242918B (en) | 2023-01-24 |
Family
ID=70878377
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202010026633.5A Active CN111242918B (en) | 2020-01-10 | 2020-01-10 | Image segmentation method and system based on Kalman filtering and Markov random field |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN111242918B (en) |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113688751A (en) * | 2021-08-30 | 2021-11-23 | 上海城投水务(集团)有限公司制水分公司 | Method and device for analyzing alum blossom characteristics by using image recognition technology |
Family Cites Families (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN103606164B (en) * | 2013-12-03 | 2016-11-02 | 西安电子科技大学 | SAR image segmentation method based on high-dimensional triple Markov field |
US10037610B1 (en) * | 2017-10-03 | 2018-07-31 | StradVision, Inc. | Method for tracking and segmenting a target object in an image using Markov Chain, and device using the same |
-
2020
- 2020-01-10 CN CN202010026633.5A patent/CN111242918B/en active Active
Non-Patent Citations (1)
Title |
---|
针对动态场景的快速视差图像预测方法;陈宝文等;《科学技术与工程》;20140628;第40卷(第18期);第94-98页 * |
Also Published As
Publication number | Publication date |
---|---|
CN111242918A (en) | 2020-06-05 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN109978839B (en) | Method for detecting wafer low-texture defects | |
CN104574393B (en) | A kind of three-dimensional pavement crack pattern picture generates system and method | |
CN107657606B (en) | Method and device for detecting brightness defect of display device | |
CN115861291B (en) | Chip circuit board production defect detection method based on machine vision | |
CN108229475B (en) | Vehicle tracking method, system, computer device and readable storage medium | |
CN111210477A (en) | Method and system for positioning moving target | |
CN110660072B (en) | Method and device for identifying straight line edge, storage medium and electronic equipment | |
CN105447512A (en) | Coarse-fine optical surface defect detection method and coarse-fine optical surface defect detection device | |
CN105930852B (en) | A kind of bubble image-recognizing method | |
CN106023153B (en) | A kind of method of bubble in measurement water body | |
CN110736747A (en) | cell liquid based smear under-mirror positioning method and system | |
Zhao et al. | Recognition of flooding and sinking conditions in flotation process using soft measurement of froth surface level and QTA | |
CN108399614B (en) | Fabric defect detection method based on non-sampling wavelet and Gumbel distribution | |
CN111242918B (en) | Image segmentation method and system based on Kalman filtering and Markov random field | |
CN112396618B (en) | Grain boundary extraction and grain size measurement method based on image processing | |
CN113705564B (en) | Pointer type instrument identification reading method | |
CN112950594B (en) | Method, device and storage medium for detecting surface defects of product | |
CN117870659A (en) | Visual inertial integrated navigation algorithm based on dotted line characteristics | |
CN109784229B (en) | Composite identification method for ground building data fusion | |
WO2017220868A1 (en) | Visual cardiomyocyte analysis | |
CN109741302B (en) | SD card form recognition system and method based on machine vision | |
CN116596899A (en) | Method, device, terminal and medium for identifying circulating tumor cells based on fluorescence image | |
WO2023134251A1 (en) | Clustering-based light bar extraction method and apparatus | |
CN115471537A (en) | Monocular camera-based moving target distance and height measuring method | |
Sun et al. | Research on image segmentation and extraction algorithm for bicolor water level gauge |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |