CN111583293B - Self-adaptive image segmentation method for multicolor double-photon image sequence - Google Patents
Self-adaptive image segmentation method for multicolor double-photon image sequence Download PDFInfo
- Publication number
- CN111583293B CN111583293B CN202010393183.3A CN202010393183A CN111583293B CN 111583293 B CN111583293 B CN 111583293B CN 202010393183 A CN202010393183 A CN 202010393183A CN 111583293 B CN111583293 B CN 111583293B
- Authority
- CN
- China
- Prior art keywords
- image
- channel
- background model
- bimodal
- value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 94
- 238000003709 image segmentation Methods 0.000 title claims abstract description 29
- 230000002902 bimodal effect Effects 0.000 claims abstract description 95
- 238000012549 training Methods 0.000 claims abstract description 52
- 238000001514 detection method Methods 0.000 claims abstract description 10
- 230000011218 segmentation Effects 0.000 claims abstract description 4
- 230000007704 transition Effects 0.000 claims description 10
- 238000007781 pre-processing Methods 0.000 claims description 8
- 230000003044 adaptive effect Effects 0.000 claims description 7
- 230000008569 process Effects 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000003708 edge detection Methods 0.000 claims description 3
- 238000013507 mapping Methods 0.000 claims description 3
- 230000002093 peripheral effect Effects 0.000 claims description 3
- 238000012545 processing Methods 0.000 abstract description 5
- 210000002569 neuron Anatomy 0.000 description 11
- 239000011159 matrix material Substances 0.000 description 7
- 238000007418 data mining Methods 0.000 description 5
- 238000003384 imaging method Methods 0.000 description 5
- 238000012544 monitoring process Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000005065 mining Methods 0.000 description 4
- 230000003068 static effect Effects 0.000 description 4
- 210000004556 brain Anatomy 0.000 description 3
- 210000004027 cell Anatomy 0.000 description 3
- 238000009825 accumulation Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 241000252212 Danio rerio Species 0.000 description 1
- 208000037273 Pathologic Processes Diseases 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000036772 blood pressure Effects 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 210000002257 embryonic structure Anatomy 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000002073 fluorescence micrograph Methods 0.000 description 1
- 238000000799 fluorescence microscopy Methods 0.000 description 1
- 238000007429 general method Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000009054 pathological process Effects 0.000 description 1
- 230000035790 physiological processes and functions Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10064—Fluorescence image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a self-adaptive image segmentation method for a multicolor two-photon image sequence. Selecting the k-th image to the n-th image from the multicolor double-photon image sequence as a training sample; generating an initialized multi-channel bimodal background model by a training sample; continuously updating the multichannel bimodal background model in real time; and (3) carrying out segmentation detection on the image input in real time by using the real-time updated multi-channel bimodal background model. The method solves the problem that a background modeling and image segmentation method specially designed for sequence data characteristics of the multicolor two-photon image is lacked, overcomes the problem that the existing methods cannot adapt to and utilize the sequence data characteristics of the multicolor two-photon image, and effectively ensures the processing accuracy and the operation efficiency.
Description
Technical Field
The invention relates to an image processing method in the technical field of image data mining, in particular to a self-adaptive image segmentation method for a multicolor two-photon image sequence.
Background
Because the multicolor two-photon imaging technology can simultaneously acquire a high-contrast two-photon fluorescence image of a sample to be detected containing multiple fluorophores, and the generated data is usually a multi-channel high-resolution image sequence, the multicolor two-photon imaging data theoretically has higher information density and complexity than the monochromatic two-photon imaging data. However, due to the above characteristics, analysis, processing, and mining of polychromatic two-photon imaging data is difficult. With the rapid generation and accumulation of data, the traditional data analysis mode dominated by manpower can obviously not be continued, so that a high-efficiency automatic data mining scheme aiming at the characteristics of multicolor two-photon imaging data needs to be developed urgently, the utilization rate of the data is effectively improved, and the data value is better and deeply mined.
Adaptive image segmentation is a technique that can be used for automated data mining of sequences of polychromatic two-photon images. By learning data samples of the multicolor two-photon image sequence, a background model of the image sequence is constructed, and then the difference between the specified image frame and the background model is compared, so that target components to be monitored in the specified image frame can be rapidly and accurately segmented, and further, full-automatic monitoring and detection of various biochemical indexes and dynamic tracking of physiological or pathological processes are realized.
However, there are currently fewer adaptive image segmentation methods designed specifically for the data characteristics of multi-color two-photon image sequences. Existing methods can be largely divided into two main categories.
One class of methods is derived from the conventional image segmentation method for a single static image, and the problems of the method are as follows: the image sequence with time consistency is treated as an isolated and irrelevant single image, only the internal space information of the single image is utilized, and the time dimension information of the image sequence is completely lost, so that the implicit dynamic information of the target to be monitored in the multicolor two-photon image sequence cannot be fully mined and utilized.
Another type of methods is image segmentation methods derived from the traditional intelligent video monitoring field, and the problems of the methods are that: there is a lack of adaptation and utilization of sequence data characteristics of polychromatic two-photon images. For example, these methods may reduce the color image into a gray image in a preprocessing stage to improve the operation performance. This processing strategy is reasonable in a daily monitoring scenario. However, such dimension reduction of simplified data complexity necessarily results in loss of important color information, which obviously necessarily greatly reduces data value and even affects accuracy of analysis result for sequence data of multicolor two-photon images with color information as a dominant feature.
In summary, if the unmatched adaptive image segmentation method is transplanted blindly, not only the data mining of the multicolor two-photon image sequence cannot be really and effectively realized, but also the misjudgment of the experimental result can be seriously caused.
Therefore, in the field of automated data mining for multi-color two-photon image sequences, an effective and efficient adaptive image segmentation method specially designed for the characteristics of multi-color two-photon image sequence data is urgently needed.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides a self-adaptive image segmentation method facing to a multicolor two-photon image sequence. The method is specially designed according to the sequence data characteristics of the multicolor two-photon image, not only is background model framework optimization carried out according to the background characteristics of the image, but also the image is accurately segmented, and the designed online updating mode also completely meets various performance requirements of processing the sequence data of the multi-channel high-resolution image, such as accuracy, instantaneity, precision and the like.
The technical scheme of the method comprises the following steps:
s1: selecting the k-th image to the n-th image from the multicolor double-photon image sequence as a training sample;
s2: generating an initialized multi-channel bimodal background model by a training sample;
s3: continuously updating the multi-channel bimodal background model in real time;
s4: and carrying out segmentation detection on the image input in real time by using the real-time updated multi-channel bimodal background model.
The multicolor two-photon image sequence can be obtained by brain neuron through two-photon fluorescence microscopic imaging collection.
The step S1 includes the steps of:
s11: selecting continuous images from the k th to the n th from the multicolor double-photon image sequence as training samples;
s12: if the value range of the original pixel value of the image in the training sample is not [0,255], then:
preprocessing a training sample, and mapping the value range of pixel points on each color channel of each image in the training sample into the value range of [0,255], wherein the specific method is as follows:
wherein, U represents the upper limit value of the value range of the original pixel value of the image in the training sample, I (x, y) is the pixel value of the pixel point (x, y) in the image before the preprocessing, and J (x, y) is the pixel value of the pixel point (x, y) in the image after the preprocessing;
if the value range of the original pixel value of the image in the training sample is [0,255], the training sample is not preprocessed.
In S12, the pixel value of the image is limited to a range of [0,255 ].
The step S2 includes the steps of:
s21: constructing a multichannel bimodal background model of an image sequence initialized on an R channel of RGB:
s211, calculating two central values of the initialized multi-channel bimodal background model at the position of each pixel point (x, y) in the image on the R channel, wherein the method comprises the following steps:
(1) if the pixel point (x, y) is located at the edge of the periphery of the image, calculating all pixel values J (x, y) of the pixel point (x, y) in all images of the training sample k ，J(x,y) k+1 ，...，J(x,y) n The mode is the most frequent number, J (x, y) k Representing pixel values of pixel points (x, y) of the kth image, and respectively taking the median and the mode as a first central value and a second central value of the initialized multi-channel bimodal background model at the positions of the pixel points (x, y);
(2) if the pixel point (x, y) is not located on the peripheral edge of the image, calculating the median and the mode of all pixel values in a 3 x 3 neighborhood of all images of a training sample by taking the pixel point as the center, wherein the 3 x 3 neighborhood of each image has nine pixel points, the training sample has n-k +1 images, the training sample has 9 x (n-k + 1) pixel values, and the median and the mode are respectively used as a first central value and a second central value of the initialized multichannel bimodal background model at the position of the pixel point (x, y);
thereby obtaining a first central value and a second central value of the multi-channel bimodal background model at the position of the pixel point (x, y)
And &>
S212, on an R channel, calculating radius values of the initialized multi-channel bimodal background model shared by pixel points in all k-n images of the training sample, wherein the sharing means that the radius values of the multi-channel bimodal background model of all the pixel points in the same image are the same, and the calculating method comprises the following steps:
(1) for each image in the training sample, finding out non-edge pixel points in the image by using an image edge detection algorithm, forming a set by all the non-edge pixel points in each image, and recording the set of the non-edge pixel points in the z-th image as the set
(2) Calculating the radius value of an initialized multi-channel bimodal background model shared by pixel points in a set of non-edge pixel points of each image in all k-n images of a training sample according to the following formula:
and is provided with
Wherein,
represents the total number of non-edge pixels in the z-th image on the R channel, z representing the ordinal number z = k>
Is the radius value of the initialized multi-channel bimodal background model at the position of each pixel point on the R channel, and is greater than or equal to>
Independent of pixel location;
The pixel value of a pixel point (x, y) in the z-th image on the R channel is represented, and V is the upper limit value of the pixel value in the image, namely 255;
S213, an initialized multi-channel bimodal background model of an R channel at the position of a pixel point (x, y) in an image is formed as follows: the initialized multi-channel bimodal background model is formed by combining two value range ranges, and the central values of the two value range ranges are respectively
And &>
The radius of each value range is half
S22: on an R channel, calculating the learning rate of the initialized multi-channel bimodal background model, wherein the method comprises the following steps:
within all images of the training sample, on the R channelThe pixel values of all pixel points in the image are from theta 1 Gray scale transition to theta 2 Calculating the probability of gray scale to generate the learning rate of the multichannel bimodal background model of the nth image moment shared by pixel points in the image
Wherein theta is 1 Representing the gray level before the pixel value transition, theta 2 Representing the gray level after the transition of the pixel value, theta 1 ,θ 2 ∈[0,255]；
S23: calculating the initialized background model of the image sequence on the RGB G channel and the learning rate thereof according to the same method as the steps S21 to S22, namely obtaining the initialized multi-channel dual-mode background model of the G channel and the central value of the two value range of the initialized multi-channel dual-mode background model of the G channel
And &>
Equal radius in two value ranges>
A first value range>
θ 1 ,θ 2 ∈[0,255]；
S24: calculating the initialization background model of the image sequence on the B channel of RGB and the learning rate thereof according to the same method as the steps S21-S22, namely obtaining the initialization multichannel bimodal background model of the B channel and the central value of two value range of the initialization multichannel bimodal background model of the B channel
And &>
The same radius of the value ranges of both value ranges->
First range of values
θ 1 ,θ 2 ∈[0,255]。
The step S3 includes the steps of:
s31: continuously updating the central value of the multi-channel bimodal background model on an R channel, wherein the method comprises the following steps:
when the (n + 1) th image of the training sample is newly read in, for each pixel point (x, y) in the image, respectively updating a first central value and a second central value of the multi-channel bimodal background model at the position of the pixel point by adopting the following formulas:
wherein,
and &>
Is two central values of the multi-channel bimodal background model when the pixel point (x, y) is in the (n + 1) th image, and is/are selected>
And &>
Two central values of the multi-channel bimodal background model and the background model learning rate of the pixel point (x, y) in the nth image are respectively selected as the pixel point (x, y)>
Is the pixel value of the pixel point (x, y) in the (n + 1) th image; in the formula (1) < theta > 1 Is taken on a value of->
In the formula (2) < theta > 1 Is taken as>
And theta 2 Then all take on a value of->
S32: continuously updating the radius value of the multi-channel bimodal background model on the R channel, wherein the method comprises the following steps:
when the (n + 1) th image is read in, updating the radius value of the single-mode background model at the position of each pixel point (x, y) in the video field:
and is provided with
Wherein,
is the multi-channel bimodal background model radius value at n +1 frames on any pixel point; />
S33: on the R channel, when an n +1 frame is newly read in, updating a multichannel bimodal background model at the position of each pixel point (x, y) in the video field:
that is, the background model is composed of two value range ranges, the central values of the two value range ranges are
S34: on the R channel, the learning rate of the multi-channel bimodal background model is continuously updated, and the method comprises the following steps:
when the n +1 th image is read in, calculating the theta of the pixel values of all the pixel points positioned in the odd rows and the odd columns in the images from k +1 to n +1 images on the R channel 1 Gray scale transition to theta 2 Probability of gray scale, learning rate of multi-channel bimodal background model when generating n +1 th frame shared by pixel points in image
In this way, when the (n + 1) th image is newly read in, the same method as that in the above steps S31 to S34 is adopted to continuously update the multi-channel bimodal background model at the time of the (n + i) frame, and the background model can be represented as two background models
S34: according to the method in the above steps S31 to S34, continuously updating the multi-channel bimodal background model and the background model learning rate of the image sequence on the G channel, respectively obtaining:
and
s35: according to the method in the above steps S31 to S34, continuously updating the multi-channel bimodal background model of the image sequence on the B channel and the background model learning rate, respectively obtaining:
and
and repeating the steps to continuously update the multichannel bimodal background model of each image of the image sequence on the RGB three channels continuously.
The step S4 is specifically to process and judge each pixel point of the image by using the value range of the multi-channel bimodal background model: if the pixel value of the pixel point is in the two value range ranges of the multi-channel bimodal background model, the pixel point is taken as the background; and if the pixel value of the pixel point is not in the two value range ranges of the multi-channel bimodal background model, the pixel point is taken as the foreground.
In specific implementation, a two-photon image of a brain neuron is detected in real time, active neurons and inactive neurons can be judged, and if the pixel value of a pixel point is within two value range ranges of a multi-channel bimodal background model, the pixel point is used as an inactive neuron; and if the pixel value of the pixel point is not in the two value range ranges of the multi-channel bimodal background model, the pixel point is taken as an active neuron.
The invention has the following substantial beneficial effects:
the method of the invention alleviates the problem that the field lacks special design of self-adaptive image segmentation aiming at sequence data characteristics of multicolor two-photon images. Meanwhile, the method provided by the invention overcomes the problems that some existing methods cannot adapt to and utilize sequence data characteristics of multicolor two-photon images:
(1) The method is specially used for mining the sequence data of the multicolor two-photon image, and can fully utilize the time dimension information of the image sequence, thereby effectively mining the implicit dynamic information of the target to be monitored in the image sequence;
(2) The method is specially used for mining sequence data of the multicolor two-photon image, can effectively mine implicit characteristic information of different color channels, and does not cause the problems of data derviation or result accuracy reduction and the like caused by color information discarding;
(3) The method designs a bimodal background model frame and an online updating mechanism aiming at the inherent characteristics of the background in sequence data of the multicolor two-photon image, and effectively ensures the calculation accuracy and the operation efficiency of the background model, thereby improving the image segmentation accuracy.
Drawings
FIG. 1 is a schematic flow diagram of the process of the present invention.
FIG. 2 is a schematic flow diagram of the process of the present invention.
FIG. 3 is a schematic flow diagram of the method of the present invention.
Fig. 4 is an example of a training sample used in the method of the present invention.
Fig. 5 is an example of the results achieved by the method according to an embodiment of the invention.
FIG. 6 is an example of results obtained by an image segmentation method in the field of general intelligent video surveillance according to an embodiment.
FIG. 7 is an example of the results achieved by a general single-image-oriented static image segmentation method according to an embodiment.
Fig. 8 is a schematic diagram of a background model learning rate obtaining method in the method of the present invention.
Table 1 is a qualitative comparison of the image segmentation results of the method of the present invention with other general methods.
Detailed Description
The technical scheme of the invention is further specifically described by the following embodiments and the accompanying drawings.
As shown in fig. 1, an embodiment of the present invention is as follows:
example a sequence of multicolour two-photon images of zebra fish live embryos is taken as an example, the images have three RGB colour channels, each channel has a data depth of 16 bits, each channel has a pixel value in the range of 0 to 65535, and the image resolution is 794 x 624. Three images of the sample image sequence at minutes 0, 68 and 168 are shown in the example of fig. 4, respectively.
The specific process of this embodiment is shown in fig. 1, fig. 2, and fig. 3, and includes the following steps:
s1: selecting images from the k =1 st to the n =100 th from the multicolor two-photon image sequence as training samples:
s11: selecting continuous images from the 1 st to the 100 th images from the multicolor double-photon image sequence as training samples;
s12: and (3) preprocessing the training sample if the value range of the original pixel value of the image in the training sample is not [0,255], and mapping the value range of the pixel point on each color channel of each image in the training sample into the value range of [0,255 ].
S2: generating an initialized multi-channel bimodal background model from the training samples:
s21: constructing a multi-channel bimodal background model of an image sequence initialized on an R channel of RGB:
s211, calculating two central values of the initialized multi-channel bimodal background model at the position of each pixel point (x, y) in the image on the R channel, wherein the method comprises the following steps:
(1) if the pixel point (x, y) is located on the edge of the periphery of the image, all 100 pixel values J (x, y) of the pixel point (x, y) in all 100 images of the training sample are calculated 1 ，J(x,y) 2 ，...，J(x,y) 100 The mode is the number with the highest frequency of occurrence, and the median and the mode are respectively used as the first central value of the initialized multi-channel bimodal background model at the position of the pixel point (x, y)
And a second central value->
(2) If the pixel point (x, y) is not located on the peripheral edge of the image, calculating the median and the mode of all pixel values in the 3 × 3 neighborhood of all images of the training sample by taking the pixel point as the center, wherein the 3 × 3 neighborhood of each image has nine pixel points, the training sample has 100 images in total and 900 pixel values in total, and the median and the mode are respectively taken as the first central value and the second central value of the initialized multi-channel bimodal background model at the position of the pixel point (x, y);
thereby obtaining a first central value of the multi-channel bimodal background model at the position of the pixel point (x, y)
And a second central value +>
S212, on the R channel, calculating radius values of the initialized multi-channel bimodal background model shared by the pixel points in all 1-100 images of the training sample, wherein the calculation method comprises the following steps:
(1) for each image in the training sample, finding out non-edge pixel points in the image by using an image edge detection algorithm, and recording a set formed by all the non-edge pixel points in each image as a set
(2) And calculating the radius value of the initialized multichannel bimodal background model shared by the pixel points in the set of the non-edge pixel points of each image in all 1-100 images of the training sample.
S213, an initialized multi-channel bimodal background model of an R channel at the position of a pixel point (x, y) in an image is formed as follows: the initialized multi-channel bimodal background model is formed by combining two value range ranges, and the central values of the two value range ranges are respectively
And &>
The radius of each value range is a radius value>
The first value range is->
The second value range is
S22: on the R channel, calculating the learning rate of the initialized multi-channel bimodal background model, wherein the method comprises the following steps:
in all images of the training sample, the pixel values of all pixel points in the images are measured from theta on an R channel 1 Gray scale transition to theta 2 Calculating the probability of gray scale to generate the learning rate of the multichannel bimodal background model of the nth image moment shared by pixel points in the image
Background model learning rate in the method of the invention
Is shown in fig. 8.
Preferably, the background model learning rate
The following iterative algorithm may be used for the calculation of (c):
E(θ 1 →θ 2 )＝1；
wherein,
and &>
Are respectively provided withRepresents the pixel value of any pixel point (x, y) in the image in the k frame and the k +1 frame, and is respectively abbreviated as theta 1 And theta 2 Since the three channels of RGB of the example image are 8 bits deep, i.e. the pixel values in each channel have 256 levels of gray scale, there are: theta.theta. 1 ∈[0，255]，θ 2 ∈[0，255]；E(θ 1 →θ 2 ) =1 represents the detection of the following event 1 time: (x, y) pixel values from θ in k frame 1 Gradation jump to theta in k +1 frame 2 A gray scale; sigma E (theta) 1 →θ 2 ) Is to count theta of pixel values of all pixel points in the image from k frame 1 Gradation jump to theta in k +1 frame 2 The number of gradations, Σ E (θ) 1 →θ 2 ) Is recorded in the corresponding cell ^ of the square matrix H>
Performing the following steps; square matrix based on the status of the blood pressure>
Is compared with the range of 1-100 images of the training sample>
Accumulation of values, <' > based on>
In which the detected pixel values from theta within the training sample image are recorded 1 Gradation jump to θ 2 The total number of gray levels; will->
Is normalized to [0,1 ]]The probability value between, namely the learning rate of the background model>
Is a square matrix of size 256 × 256.
S23: calculating the initialized background model of the image sequence on the G channel of RGB and the learning rate thereof according to the same method as the steps S21 to S22, namely obtaining the initialized multi-channel of the G channelDual-modality background model, and center value of two value range of G-channel initialized multi-channel dual-modality background model
And &>
Equal radius in two value ranges>
First range of values
θ 1 ,θ 2 ∈[0,255]；
S24: calculating the initialized background model of the image sequence on the B channel of RGB and the learning rate thereof according to the same method as the steps S21 to S22, namely obtaining the initialized multi-channel dual-mode background model of the B channel and the central values of the two value range of the initialized multi-channel dual-mode background model of the B channel
And &>
The same radius of the value ranges of both value ranges->
First range of value
θ 1 ,θ 2 ∈[0,255]。
S3: continuously updating the multichannel bimodal background model in real time:
s31: on the R channel, continuously updating the central value of the multi-channel bimodal background model, wherein the method comprises the following steps:
when a 101 frame is newly read in, respectively updating a first central value and a second central value of the multichannel bimodal background model at the position of each pixel point (x, y) in the video field according to the following formulas:
s32: continuously updating the radius value of the multi-channel bimodal background model on the R channel, wherein the method comprises the following steps:
when a 101 frame is newly read in, for each pixel point (x, y) in the video field, the radius value of the monomodal background model at the position of each pixel point is updated according to the following formula:
and is
On the R channel, when a 101 frame is newly read in, the multi-channel bimodal background model at the position of each pixel point (x, y) in the video field of view is updated as follows: the background model is formed by combining two value range ranges, the central values of the two value range ranges are respectively
And &>
The radius of each value range
S33: on an R channel, continuously updating the learning rate of the multi-channel bimodal background model, wherein the method comprises the following steps:
when a 101 frame is newly read in, calculating the pixel values of all pixel points positioned in odd rows and odd columns in the image from theta in 2-101 images on an R channel 1 Gray scale transition to theta 2 Probability of gray scale, multichannel bimodal background model learning rate when generating 101 st frame shared by pixel points in image
By analogy, when a frame of 100+ i is newly read in, the same method as that in the steps S31 to S34 is adopted, and the multi-channel bimodal background model at the moment of 100+ i frame is continuously updated, wherein the background model can be represented as two values
S34: according to the method in the above steps S31 to S33, the multi-channel bimodal background model and the background model learning rate of the image sequence on the G channel are continuously updated, which are:
and
s35: according to the method in the above steps S31 to S33, the multi-channel bimodal background model and the background model learning rate of the image sequence on the B channel are continuously updated, which are respectively:
and/or>
As has been described in the foregoing, the present invention,
is a square matrix of 255 x 255 in size, since theta 1 、θ 2 Are the row and column coordinates of the matrix, respectively, and so will θ 1 、θ 2 Is substituted into->
That is, the theta in the square matrix can be obtained 1 Line, theta 2 The corresponding background model learning rate at the cell position of the column; according to the example of fig. 3, a decision is made as to whether a decision is to be taken or not>
The value of (b) is the background model learning rate corresponding to the cell position of the 160 th row and the 200 th column in the square matrix, i.e., 0.5.
S4: and (3) carrying out segmentation detection on the image input in real time by using the real-time updated multi-channel bimodal background model.
In specific implementation, a two-photon image of a brain neuron is detected in real time, active neurons and inactive neurons can be judged, and if the pixel value of a pixel point is within two value range ranges of a multi-channel bimodal background model, the pixel point is used as the inactive neuron; and if the pixel value of the pixel point is not in the two value range ranges of the multi-channel bimodal background model, the pixel point is taken as an active neuron.
The results obtained according to the example of the method of the invention are shown in fig. 5. It can be seen that, because the method is designed for the data characteristics of the multi-color two-photon image sequence and performs special optimization processing, the overall segmented foreground (i.e., the white pixel point region) is basically consistent with the target object to be detected, and the situations of missed detection (i.e., the foreground pixel points which should be marked as white are marked as black representing the background) and false detection (i.e., the pixel points which should be marked as black background are marked as white representing the foreground) are less.
Meanwhile, a general image segmentation method in the field of intelligent video monitoring is selected for comparison, and the result obtained according to the embodiment is shown in fig. 6. It can be seen that, because the method is not designed for the data characteristics of the multicolor two-photon image sequence, the divided foreground is not consistent with the target object to be detected, and more missed detection areas appear.
In addition, some general single-image-oriented static image segmentation method was chosen for comparison, and the results obtained according to the embodiment are shown in fig. 7. It can be seen that the foreground segmented by the method is poorer in consistency with the target object to be detected, and more missed detection areas appear.
In summary, the qualitative comparison results between the method of the present invention and the two general image segmentation methods are shown in table 1.
TABLE 1
| Comparison of different image segmentation methods | Qualitative comparison of image segmentation results |
| The invention provides a method | Is very good |
| Image segmentation method for certain general intelligent video monitoring field as comparison | Is poor |
| Some general single image-oriented static image segmentation method as contrast | Is poor |
The result shows that the method can solve the problem that self-adaptive image segmentation is not specially designed for the sequence data characteristic of the multicolor two-photon image, overcomes the problem that the existing methods cannot adapt to and utilize the sequence data characteristic of the multicolor two-photon image, improves the accuracy of image segmentation, and obtains remarkable technical effect.
Claims (3)
1. A method for adaptive image segmentation for a sequence of polychromatic bi-photon images, the method comprising the steps of:
s1: selecting the k-th image to the n-th image from the multicolor double-photon image sequence as a training sample;
s2: generating an initialized multi-channel bimodal background model by a training sample;
s3: continuously updating the multichannel bimodal background model in real time;
s4: carrying out segmentation detection on the image input in real time by using the real-time updated multi-channel bimodal background model;
the step S2 includes the steps of:
s21: constructing a multi-channel bimodal background model of an image sequence initialized on an R channel:
s211, calculating two central values of the initialized multi-channel bimodal background model at the position of each pixel point (x, y) in the image on the R channel, wherein the method comprises the following steps:
(1) if the pixel point (x, y) is located on the edge of the periphery of the image, calculating all pixel values J (x, y) of the pixel point (x, y) in all images of the training sample k ，J(x,y) k+1 ，...，J(x,y) n Median and mode of (2), J (x, y) k Representing pixel values of pixel points (x, y) of the kth image, and respectively taking the median and the mode as a first central value and a second central value of the initialized multi-channel bimodal background model at the positions of the pixel points (x, y);
(2) if the pixel point (x, y) is not located on the peripheral edge of the image, calculating the median and the mode of all pixel values in a 3 x 3 neighborhood of all images of the training sample by taking the pixel point as the center, and respectively taking the median and the mode as a first central value and a second central value of the initialized multichannel bimodal background model at the position of the pixel point (x, y); thereby obtaining a first central value and a second central value of the multi-channel bimodal background model at the position of the pixel point (x, y)
And
s212, on the R channel, calculating the radius value of the initialized multi-channel bimodal background model shared by the pixel points in all k-n images of the training sample, wherein the calculation method comprises the following steps:
(1) for each image in the training sample, finding out non-edge pixel points in the image by using an image edge detection algorithm, forming a set by all the non-edge pixel points in each image, and recording the set of the non-edge pixel points in the z-th image as a set
(2) Calculating the radius value of an initialized multi-channel bimodal background model shared by pixel points in a set of non-edge pixel points of each image in all k-n images of a training sample according to the following formula:
and is
s213, initializing R channel at position of pixel point (x, y) in imageThe modal background model is constructed as follows: the initialized multi-channel bimodal background model is formed by combining two value range ranges, and the central values of the two value range ranges are respectively
And &>
The radius of each value range is a radius value->
The first value range is
A second value range which is +>
S22: on the R channel, calculating the learning rate of the initialized multi-channel bimodal background model, wherein the method comprises the following steps:
in all images of the training sample, the pixel values of all pixel points in the images are measured from theta on an R channel 1 Gray scale transition to theta 2 Calculating the probability of gray scale to generate the learning rate of the multichannel bimodal background model of the nth image moment shared by pixel points in the image
Wherein theta is 1 Representing the gray level before the pixel value transition, theta 2 Indicating the gray level after the pixel value transition, theta 1 ,θ 2 ∈[0,255]；
S23: calculating the initialized background model of the image sequence on the G channel and the learning rate thereof according to the same method as the steps S21 to S22, namely obtaining the initialized multi-channel bimodal background model of the G channel and the central values of two value range ranges of the initialized multi-channel bimodal background model of the G channel
And &>
The same radius of the value ranges of both value ranges->
The first value range->
A second value range->
And a learning rate of->
Wherein theta is 1 ,θ 2 ∈[0,255]；
S24: calculating the initialized background model of the image sequence on the B channel and the learning rate thereof according to the same method as the steps S21 to S22, namely obtaining the initialized multi-channel dual-mode background model of the B channel and the central values of the two value range ranges of the initialized multi-channel dual-mode background model of the B channel
And &>
The same radius of the value ranges of both value ranges->
The first value range->
A second value range->
And a learning rate of->
Wherein theta is 1 ,θ 2 ∈[0,255]；
The step S3 includes the steps of:
s31: on the R channel, continuously updating the central value of the multi-channel bimodal background model, wherein the method comprises the following steps:
when the (n + 1) th image of the training sample is newly read in, for each pixel point (x, y) in the image, respectively updating a first central value and a second central value of the multi-channel bimodal background model at the position of the pixel point by adopting the following formulas:
wherein,
and &>
Is two central values of the multi-channel bimodal background model when the pixel point (x, y) is in the (n + 1) th image, and is/are selected>
And &>
Respectively, a multi-channel bimodal background model of a pixel point (x, y) in the nth imageAnd the background model learning rate,/'>
Is the pixel value of the pixel point (x, y) in the (n + 1) th image; in the formula (1) < theta > 1 Is taken as>
In the formula (2) < theta > 1 Is taken as>
And theta 2 Then all take on values of->
S32: continuously updating the radius value of the multi-channel bimodal background model on the R channel, wherein the method comprises the following steps:
when the (n + 1) th image is read in, updating the radius value of the single-mode background model at the position of each pixel point (x, y) in the video field:
and is
Wherein,
is the radius value of the multi-channel bimodal background model at n +1 frames on any pixel point;
s33: on the R channel, when an n +1 frame is newly read in, updating a multichannel bimodal background model at the position of each pixel point (x, y) in the video field:
s34: on R channel, multi-channel bimodal backgroundThe learning rate of the model is continuously updated, and the method comprises the following steps: when the n +1 th image is newly read in, calculating the pixel values of all the pixels positioned in the odd rows and the odd columns in the image on the R channel from theta in k +1 to n +1 images 1 Gray scale transition is theta 2 Probability of gray scale, learning rate of multi-channel bimodal background model when generating n +1 th frame shared by pixel points in image
S34: continuously updating the multi-channel bimodal background model of the image sequence on the G channel and the learning rate of the background model according to the method in the steps S31 to S34;
s35: continuously updating the multichannel bimodal background model of the image sequence on the channel B and the learning rate of the background model according to the method in the steps S31 to S34;
and repeating the steps to continuously update the multichannel bimodal background model of each image of the image sequence on the RGB three channels continuously.
2. The adaptive image segmentation method for a sequence of polychromatic two-photon images according to claim 1, wherein: the step S1 includes the steps of:
s11: selecting continuous images from the k th to the n th from the multicolor double-photon image sequence as training samples;
s12: if the value range of the original pixel value of the image in the training sample is not [0,255], then:
preprocessing a training sample, and mapping the value range of pixel points on each color channel of each image in the training sample into the value range of [0,255], wherein the specific method is as follows:
wherein, U represents the upper limit value of the value range of the original pixel value of the image in the training sample, I (x, y) is the pixel value of the pixel point (x, y) in the image before the preprocessing, and J (x, y) is the pixel value of the pixel point (x, y) in the image after the preprocessing;
if the value range of the original pixel value of the image in the training sample is [0,255], the training sample is not preprocessed.
3. The adaptive image segmentation method for a sequence of polychromatic two-photon images according to claim 1, wherein: step S4 is specifically to utilize the value range of the multi-channel bimodal background model to process and judge each pixel point of the image: if the pixel value of the pixel point is in the two value range ranges of the multi-channel bimodal background model, the pixel point is taken as the background; and if the pixel value of the pixel point is not in the two value range ranges of the multi-channel bimodal background model, the pixel point is taken as the foreground.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010393183.3A CN111583293B (en) | 2020-05-11 | 2020-05-11 | Self-adaptive image segmentation method for multicolor double-photon image sequence |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010393183.3A CN111583293B (en) | 2020-05-11 | 2020-05-11 | Self-adaptive image segmentation method for multicolor double-photon image sequence |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111583293A CN111583293A (en) | 2020-08-25 |
| CN111583293B true CN111583293B (en) | 2023-04-11 |
Family
ID=72126475
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010393183.3A Active CN111583293B (en) | 2020-05-11 | 2020-05-11 | Self-adaptive image segmentation method for multicolor double-photon image sequence |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111583293B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018112137A1 (en) * | 2016-12-15 | 2018-06-21 | General Electric Company | System and method for image segmentation using a joint deep learning model |
| KR101913952B1 (en) * | 2017-07-11 | 2018-10-31 | 경북대학교 산학협력단 | Automatic Recognition Method of iPSC Colony through V-CNN Approach |
| CN108765463A (en) * | 2018-05-30 | 2018-11-06 | 河海大学常州校区 | A kind of moving target detecting method calmodulin binding domain CaM extraction and improve textural characteristics |
| KR20190134933A (en) * | 2018-05-18 | 2019-12-05 | 오드컨셉 주식회사 | Method, apparatus and computer program for extracting representative feature of object in image |
| AU2019201787A1 (en) * | 2018-05-22 | 2019-12-12 | Adobe Inc. | Compositing aware image search |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10380741B2 (en) * | 2016-12-07 | 2019-08-13 | Samsung Electronics Co., Ltd | System and method for a deep learning machine for object detection |
-
2020
- 2020-05-11 CN CN202010393183.3A patent/CN111583293B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018112137A1 (en) * | 2016-12-15 | 2018-06-21 | General Electric Company | System and method for image segmentation using a joint deep learning model |
| KR101913952B1 (en) * | 2017-07-11 | 2018-10-31 | 경북대학교 산학협력단 | Automatic Recognition Method of iPSC Colony through V-CNN Approach |
| KR20190134933A (en) * | 2018-05-18 | 2019-12-05 | 오드컨셉 주식회사 | Method, apparatus and computer program for extracting representative feature of object in image |
| AU2019201787A1 (en) * | 2018-05-22 | 2019-12-12 | Adobe Inc. | Compositing aware image search |
| CN108765463A (en) * | 2018-05-30 | 2018-11-06 | 河海大学常州校区 | A kind of moving target detecting method calmodulin binding domain CaM extraction and improve textural characteristics |
Non-Patent Citations (1)
| Title |
|---|
| 何亮时.动态场景中的改进混合高斯背景模型.计算机工程.2012,第38卷(第8期),全文. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111583293A (en) | 2020-08-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111612763B (en) | Mobile phone screen defect detection method, device and system, computer equipment and medium | |
| CN111524137B (en) | Cell identification counting method and device based on image identification and computer equipment | |
| CN106778687B (en) | Fixation point detection method based on local evaluation and global optimization | |
| CN110705639B (en) | Medical sperm image recognition system based on deep learning | |
| CN116416166B (en) | Liver biopsy data analysis method and system | |
| CN107657619B (en) | A kind of low-light (level) Forest fire image dividing method | |
| CN110889448A (en) | Electrocardiogram classification method based on convolutional neural network | |
| CN115641335B (en) | Embryo abnormity multi-cascade intelligent comprehensive analysis system based on time difference incubator | |
| CN113052859A (en) | Super-pixel segmentation method based on self-adaptive seed point density clustering | |
| CN111986126A (en) | Multi-target detection method based on improved VGG16 network | |
| CN111242026A (en) | Remote sensing image target detection method based on spatial hierarchy perception module and metric learning | |
| Rachmad et al. | Classification of mycobacterium tuberculosis based on color feature extraction using adaptive boosting method | |
| CN108510483A (en) | A kind of calculating using VLAD codings and SVM generates color image tamper detection method | |
| CN111583293B (en) | Self-adaptive image segmentation method for multicolor double-photon image sequence | |
| CN111047654A (en) | High-definition high-speed video background modeling method based on color information | |
| CN116433528A (en) | Image detail enhancement display method and system for target area detection | |
| CN116563301A (en) | Medical image segmentation system based on federal learning | |
| CN116543414A (en) | Tongue color classification and tongue redness and purple quantification method based on multi-model fusion | |
| CN110991361B (en) | Multi-channel multi-modal background modeling method for high-definition high-speed video | |
| CN116309914A (en) | Image signal analysis system and method using layout detection | |
| CN116452526A (en) | Rice seed identification and counting method based on image detection | |
| CN115564647A (en) | Novel super-division module and up-sampling method for image semantic segmentation | |
| CN112950479B (en) | Image gray level region stretching algorithm | |
| CN111028245A (en) | Multi-mode composite high-definition high-speed video background modeling method | |
| CN111008995B (en) | Single-channel multi-mode background modeling method for high-definition high-speed video |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |