CN111583293B - Self-adaptive image segmentation method for multicolor double-photon image sequence - Google Patents

Abstract

The invention discloses a self-adaptive image segmentation method for a multicolor two-photon image sequence. Selecting the k-th image to the n-th image from the multicolor double-photon image sequence as a training sample; generating an initialized multi-channel bimodal background model by a training sample; continuously updating the multichannel bimodal background model in real time; and (3) carrying out segmentation detection on the image input in real time by using the real-time updated multi-channel bimodal background model. The method solves the problem that a background modeling and image segmentation method specially designed for sequence data characteristics of the multicolor two-photon image is lacked, overcomes the problem that the existing methods cannot adapt to and utilize the sequence data characteristics of the multicolor two-photon image, and effectively ensures the processing accuracy and the operation efficiency.

Description

An adaptive image segmentation method for multi-color two-photon image sequences

Technical Field

The invention relates to an image processing method in the technical field of image data mining, and in particular to an adaptive image segmentation method for multi-color two-photon image sequences.

Background Art

Since multicolor two-photon imaging technology can simultaneously obtain high-contrast two-photon fluorescence images of samples containing multiple fluorophores, the data generated is usually a multi-channel high-resolution image sequence. Therefore, multicolor two-photon imaging data theoretically has higher information density and complexity than monochrome two-photon imaging data. However, it is precisely because of the above characteristics that the analysis, processing, and mining of multicolor two-photon imaging data are difficult. Moreover, with the rapid generation and accumulation of data, the traditional data analysis method dominated by humans is obviously no longer sustainable. Therefore, it is urgent to develop an efficient and automated data mining solution for the characteristics of multicolor two-photon imaging data, effectively improve the utilization rate of data, and better and more deeply mine the value of data.

Adaptive image segmentation is a technology that can be used for automated data mining of multi-color two-photon image sequences. By learning the data samples of multi-color two-photon image sequences, the background model of the image sequence is first constructed, and then by comparing the difference between the specified image frame and the background model, the target component to be monitored in the specified image frame can be quickly and accurately segmented, thereby realizing fully automatic monitoring and detection of various biochemical indicators, as well as dynamic tracking of physiological or pathological processes.

However, there are few adaptive image segmentation methods designed specifically for the data characteristics of multi-color two-photon image sequences. Existing methods can be mainly divided into two categories.

One type of method is derived from the traditional image segmentation method for a single static image. The problem with this method is that it treats a temporally coherent image sequence as an isolated and unrelated single image and only utilizes the spatial information within the single image, while completely losing the temporal dimension information of the image sequence. Therefore, it is impossible to fully explore and utilize the implicit dynamic information of the target to be monitored in the multi-color two-photon image sequence.

Another type of method is the image segmentation method from the traditional intelligent video surveillance field. The problem with these methods is that they lack the adaptation and utilization of the characteristics of multi-color two-photon image sequence data. For example, these methods will reduce the dimension of color images into grayscale images in the preprocessing stage to improve the computing performance. This processing strategy is reasonable in daily monitoring scenarios. However, this dimensionality reduction that simplifies the complexity of the data will inevitably lead to the loss of important color information. For multi-color two-photon image sequence data that uses color information as an advantageous feature, it will obviously greatly reduce the data value and even affect the accuracy of the analysis results.

In summary, if an incompatible adaptive image segmentation method is blindly transplanted, not only will it be impossible to truly and effectively realize data mining of multi-color two-photon image sequences, but it will also seriously lead to misjudgment of experimental results.

Therefore, in the field of automated data mining for multi-color two-photon image sequences, there is an urgent need to design effective and efficient adaptive image segmentation methods specifically for the data characteristics of multi-color two-photon image sequences.

Summary of the invention

In view of the problems existing in the prior art, the present invention provides an adaptive image segmentation method for multi-color two-photon image sequences. The method is specially designed according to the characteristics of multi-color two-photon image sequence data. Not only does it optimize the background model framework according to the background characteristics of such images, thereby accurately segmenting the images, but the designed online update mode also fully meets the various performance requirements of accuracy, real-time performance and precision in processing such multi-channel high-resolution image sequence data.

The technical solution of the method of the present invention comprises the following steps:

S1: Select the kth to nth images from the multi-color two-photon image sequence as training samples;

S2: A multi-channel bimodal background model initialized by training samples;

S3: continuous real-time update of the above multi-channel bimodal background model;

S4: Use the multi-channel bimodal background model updated in real time to perform segmentation detection on the real-time input image.

The multi-color two-photon image sequence can be obtained by collecting brain neurons through two-photon fluorescence microscopy imaging.

The step S1 comprises the following steps:

S11: Select continuous images from the kth to the nth images in the multi-color two-photon image sequence as training samples;

S12: If the value range of the original pixel value of the image in the training sample is not [0,255], then:

The training samples are preprocessed to map the value range of the pixels on each color channel of each image in the training samples to the value range of [0,255]. The specific method is as follows:

Among them, U represents the upper limit of the value range of the original pixel value of the image in the training sample, I(x,y) is the pixel value of the pixel point (x,y) in the image before preprocessing, and J(x,y) is the pixel value of the pixel point (x,y) in the image after preprocessing;

If the value range of the original pixel values in the training sample is [0,255], no preprocessing is performed on the training sample.

In S12, the pixel values of the image are limited to a value range of [0, 255].

The step S2 comprises the following steps:

S21: Construct a multi-channel bimodal background model initialized on the R channel of RGB for the image sequence:

S211, on the R channel, for each pixel point (x, y) in the image, calculate the two center values of the initialized multi-channel bimodal background model at the position of the pixel point (x, y), the method is as follows:

① If the pixel point (x, y) is located on the edges of the image, the median and mode of all pixel values J(x, y) _k , J(x, y) _k+1 , ..., J(x, y) _n of the pixel point (x, y) in all images of the training sample are calculated. The mode is the number with the highest frequency. J(x, y) _k represents the pixel value of the pixel point (x, y) in the kth image. The median and mode are used as the first center value and the second center value of the initialized multi-channel dual-modal background model at the position of the pixel point (x, y), respectively.

② If the pixel point (x, y) is not located on the edge of the image, the median and mode of all pixel values in the 3×3 neighborhood of all images in the training sample with the pixel point as the center are calculated. There are nine pixels in the 3×3 neighborhood of each image. There are n-k+1 images in the training sample, with a total of 9×(n-k+1) pixel values. The median and mode are used as the first and second center values of the initialized multi-channel dual-modal background model at the pixel point (x, y) position respectively;

和

Thus, the first center value and the second center value of the multi-channel dual-modal background model at the pixel point (x, y) are obtained respectively.

and

S212. On the R channel, the radius value of the initialized multi-channel bimodal background model shared by the pixels in all k to n images of the training sample is calculated. Sharing means that the radius values of the multi-channel bimodal background model of all pixels in the same image are the same. The calculation method is as follows:

① For each image in the training sample, use the image edge detection algorithm to find the non-edge pixels in the image, and form a set of all non-edge pixels in each image. The set of non-edge pixels in the zth image is recorded as

② In all k to n images of the training sample, in the set of non-edge pixels of each image, calculate the radius value of the initialized multi-channel bimodal background model shared by the pixels according to the following formula:

and

代表在R通道上第z幅图像内非边沿像素点的总数，z表示图像的序数z＝k,...,n-1，

是R通道上每个像素点位置上的初始化多通道双模态背景模型的半径值，

与像素点位置无关；

表示R通道上第z幅图像中像素点(x,y)的像素值，V是图像中像素值的上限值，即255；in,

Represents the total number of non-edge pixels in the zth image on the R channel, where z represents the ordinal number of the image z=k,...,n-1.

is the radius value of the initialized multi-channel bimodal background model at each pixel position on the R channel,

It has nothing to do with the pixel position;

Represents the pixel value of the pixel point (x, y) in the z-th image on the R channel, and V is the upper limit of the pixel value in the image, which is 255;

的下标n是代表第n幅图像时的半径值，这个值是由k～n幅图像数据累积计算所得。初始化模型的半径就是看(累积到)第n幅图像时的

求出了

后，在n+1帧时用

去迭代更新背景模型半径

也就是说，k～n帧内，背景模型半径不需要用迭代的方法计算获得。从n+1帧开始，才使用迭代的方法计算背景模型半径。

The subscript n represents the radius value when viewing the nth image. This value is calculated by accumulating the data of k to n images. The radius of the initialization model is the radius when viewing (accumulating) the nth image.

Asked for

Then, in frame n+1,

Iterate and update the background model radius

That is to say, within the k~n frames, the background model radius does not need to be calculated using the iterative method. Starting from the n+1 frame, the background model radius is calculated using the iterative method.

和

每个值域取值范围的半径均为半

S213, the initialized multi-channel bimodal background model of the R channel at the pixel point (x, y) position in the image is constructed as follows: The initialized multi-channel bimodal background model is composed of two value ranges, and the center values of the two value ranges are respectively

and

The radius of each value range is half

S22: On the R channel, the learning rate of the initialized multi-channel bimodal background model is calculated as follows:

其中θ₁表示像素值跃迁前的灰阶等级，θ₂表示像素值跃迁后的灰阶等级，θ₁,θ₂∈[0,255]；In all images of the training sample, the probability of the pixel values of all pixels in the image transitioning from grayscale θ ₁ to grayscale θ ₂ is calculated on the R channel to generate the learning rate of the multi-channel bimodal background model shared by the pixels in the image at the nth image moment

Wherein θ ₁ represents the grayscale level before the pixel value transition, θ ₂ represents the grayscale level after the pixel value transition, θ ₁ ,θ ₂ ∈[0,255];

和

两个值域取值范围的相同半径

第一个值域范

θ₁,θ₂∈[0,255]；S23: According to the same method as steps S21 to S22 above, the initialization background model of the image sequence on the G channel of RGB and its learning rate are calculated, that is, the initialization multi-channel bimodal background model of the G channel and the center value of the two value ranges of the initialization multi-channel bimodal background model of the G channel are obtained.

and

The same radius of the two ranges

The first range

θ ₁ ,θ ₂ ∈[0,255];

和

两个值域取值范围的相同半径

第一个值域范围

θ₁,θ₂∈[0,255]。S24: According to the same method as steps S21 to S22 above, the initialization background model of the image sequence on the B channel of RGB and its learning rate are calculated, that is, the initialization multi-channel bimodal background model of the B channel and the center value of the two value ranges of the initialization multi-channel bimodal background model of the B channel are obtained.

and

The same radius of the two ranges

The first value range

θ ₁ ,θ ₂ ∈[0,255].

The step S3 comprises the following steps:

S31: On the R channel, the center value of the multi-channel bimodal background model is continuously updated as follows:

When the n+1th image of the training sample is newly read, for each pixel point (x, y) in the image, the first center value and the second center value of the multi-channel bimodal background model at the pixel point position are updated respectively using the following formulas:

和

是像素点(x,y)在第n+1幅图像时的多通道双模态背景模型的两个中心值，

和

分别是像素点(x,y)在第n幅图像时的多通道双模态背景模型的两个中心值和背景模型学习率，

是像素点(x,y)在第n+1幅图像时的像素值；在公式(1)中θ₁的取值为

在公式(2)中θ₁的取值为

而θ₂则都取值为

in,

and

are the two center values of the multi-channel bimodal background model of the pixel point (x, y) in the n+1th image,

and

are the two central values of the multi-channel bimodal background model and the background model learning rate of the pixel point (x, y) in the nth image,

is the pixel value of the pixel point (x, y) in the n+1th image; in formula (1), the value of θ ₁ is

In formula (2), the value of _θ1 is

And θ ₂ is always

S32: On the R channel, continuously update the radius value of the multi-channel dual-modal background model, as follows:

When the n+1th image is newly read, for each pixel point (x, y) in the video field of view, the radius value of the unimodal background model at the pixel point position is updated:

and

是任意像素点上在n+1帧时的多通道双模态背景模型半径值；in,

is the radius value of the multi-channel bimodal background model at any pixel point at frame n+1;

S33: On the R channel, when the n+1 frame is newly read, the multi-channel bimodal background model at each pixel point (x, y) in the video field of view is updated:

That is, the background model is composed of two value ranges, and the center values of the two value ranges are

S34: On the R channel, the learning rate of the multi-channel bimodal background model is continuously updated as follows:

When the n+1th image is newly read, the probability that the pixel values of all pixels in odd rows and columns in the image transition from θ ₁ grayscale to θ ₂ grayscale in images k+1 to n+1 is calculated on the R channel, and the learning rate of the multi-channel bimodal background model shared by the pixels in the image at the n+1th frame is generated.

Similarly, when the n+1th image is newly read, the multi-channel dual-modal background model at the n+i frame time is continuously updated using the same method as in the above steps S31 to S34. The background model can be represented by two

与

S34: According to the method in the above steps S31 to S34, the multi-channel bimodal background model and the background model learning rate of the image sequence on the G channel are continuously updated, and the obtained values are respectively:

and

与

S35: According to the method in the above steps S31 to S34, the multi-channel bimodal background model and the background model learning rate of the image sequence on the B channel are continuously updated, and the obtained values are respectively:

and

Repeat the above steps to iterate and continuously update the multi-channel bimodal background model on the three RGB channels of each image in the image sequence.

The step S4 specifically uses the value range of the multi-channel bimodal background model to process and judge each pixel of the image: if the pixel value of the pixel is within the two value ranges of the multi-channel bimodal background model, the pixel is used as the background; if the pixel value of the pixel is not within the two value ranges of the multi-channel bimodal background model, the pixel is used as the foreground.

In a specific implementation, real-time detection of two-photon images of brain neurons can determine active neurons and inactive neurons. If the pixel value of a pixel is within the two value ranges of the multi-channel bimodal background model, the pixel is regarded as an inactive neuron; if the pixel value of a pixel is not within the two value ranges of the multi-channel bimodal background model, the pixel is regarded as an active neuron.

The substantial beneficial effects of the present invention are:

The method of the present invention alleviates the problem of lack of adaptive image segmentation specifically designed for the characteristics of multi-color two-photon image sequence data in the field. At the same time, the method provided by the present invention overcomes the problem that some existing methods cannot adapt to and utilize the characteristics of multi-color two-photon image sequence data:

(1) This method is specifically used to mine multi-color two-photon image sequence data. It can make full use of the time dimension information of the image sequence, thereby effectively mining the implicit dynamic information of the target to be monitored in the image sequence;

(2) This method is specifically used to mine multi-color two-photon image sequence data, and can effectively mine the implicit feature information of different color channels without discarding color information, resulting in data depreciation or reduced result accuracy;

(3) This method specifically targets the inherent characteristics of the background in multi-color two-photon image sequence data and designs a dual-modal background model framework and online update mechanism, which effectively ensures the accuracy and operational efficiency of the background model calculation, thereby improving the accuracy of image segmentation.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a schematic flow diagram of the method of the present invention.

FIG. 2 is a schematic flow diagram of the method of the present invention.

FIG3 is a schematic flow chart of the method of the present invention.

FIG. 4 is an example of training samples used in the method of the present invention.

FIG. 5 is an example of the results obtained by the method of the present invention according to an embodiment.

FIG. 6 is an example of the results obtained by a certain general image segmentation method in the field of intelligent video surveillance according to an embodiment.

FIG. 7 is an example of the results obtained by a certain general static image segmentation method for a single image according to an embodiment.

FIG8 is a schematic diagram of a method for obtaining a background model learning rate in the method of the present invention.

Table 1 is a qualitative comparison of the image segmentation results of the method of the present invention and other general methods.

DETAILED DESCRIPTION

The technical solution of the present invention is further specifically described below through embodiments and in conjunction with the accompanying drawings.

As shown in FIG1 , the embodiments of the present invention are as follows:

In the embodiment, a multi-color two-photon image sequence of a living zebrafish embryo is used as an example. The image has three color channels, RGB, and the data depth of each channel is 16 bits. The range of the pixel value of each channel is 0 to 65535, and the image resolution is 794 × 624. The example in FIG4 shows three images corresponding to the sample image sequence at the 0th minute, the 68th minute, and the 168th minute, respectively.

The specific process of this embodiment is shown in Figures 1, 2 and 3, and includes the following steps:

S1: Select k=1 to n=100 images from the multi-color two-photon image sequence as training samples:

S11: Select continuous images from the 1st to the 100th images in the multi-color two-photon image sequence as training samples;

S12: If the value range of the original pixel values of the images in the training samples is not [0, 255], the training samples are preprocessed to map the value range of the pixel points on each color channel of each image in the training samples to the value range of [0, 255].

S2: Multi-channel bimodal background model initialized by training samples:

S21: Construct a multi-channel bimodal background model initialized on the R channel of RGB for the image sequence:

S211, on the R channel, for each pixel point (x, y) in the image, calculate the two center values of the initialized multi-channel bimodal background model at the position of the pixel point (x, y), the method is as follows:

和第二中心值

① If the pixel point (x, y) is located on the edge of the image, calculate the median and mode of all 100 pixel values J(x, y) ₁ , J(x, y) ₂ , ..., J(x, y) ₁₀₀ of the pixel point (x, y) in all 100 images of the training sample. The mode is the number with the highest frequency. The median and mode are used as the first center value of the initialized multi-channel bimodal background model at the pixel point (x, y) position.

and the second center value

② If the pixel point (x, y) is not located on the edge of the image, the median and mode of all pixel values in the 3×3 neighborhood of all images in the training sample with the pixel point as the center are calculated. There are nine pixels in the 3×3 neighborhood of each image. There are 100 images in the training sample, with a total of 900 pixel values. The median and mode are used as the first and second center values of the initialized multi-channel dual-modal background model at the pixel point (x, y) position respectively;

和第二中心值

Thus, the first center value of the multi-channel bimodal background model at the pixel point (x, y) is obtained

and the second center value

S212, on the R channel, calculate the radius value of the initialized multi-channel bimodal background model shared by the pixels in all 1-100 images of the training sample, and the calculation method is as follows:

① For each image in the training sample, use the image edge detection algorithm to find the non-edge pixels in the image, and record the set of all non-edge pixels in each image as

② In all 1-100 images of the training sample, in the set of non-edge pixels of each image, calculate the radius value of the initialized multi-channel bimodal background model shared by the pixels.

和

每个值域取值范围的半径均为半径值

第一个值域范围为

第二个值域范围为

S213, the initialized multi-channel bimodal background model of the R channel at the pixel point (x, y) position in the image is constructed as follows: The initialized multi-channel bimodal background model is composed of two value ranges, and the center values of the two value ranges are respectively

and

The radius of each value range is the radius value

The first range is

The second range is

S22: On the R channel, the learning rate of the initialized multi-channel bimodal background model is calculated as follows:

本发明方法中背景模型学习率

的示意图如图8所示。In all images of the training sample, the probability of the pixel values of all pixels in the image transitioning from grayscale θ ₁ to grayscale θ ₂ is calculated on the R channel to generate the learning rate of the multi-channel bimodal background model shared by the pixels in the image at the nth image moment

Background model learning rate in the method of the present invention

The schematic diagram is shown in Figure 8.

的计算可采用如下的迭代算法：As a preference, the background model learning rate

The calculation of can be done using the following iterative algorithm:

E(θ ₁ →θ ₂ )＝1;

和

分别代表图像内的任意像素点(x,y)在第k帧和第k+1帧中的像素值，并分别简记为θ₁和θ₂，由于示例图像的RGB三个通道均为8位深度，即每个通道中像素值具有256级灰阶，所以有：θ₁∈[0，255]，θ₂∈[0，255]；E(θ₁→θ₂)＝1表示检测到以下的事件1次：(x,y)的像素值从k帧中的θ₁灰阶跳变为k+1帧中的θ₂灰阶；∑E(θ₁→θ₂)是统计图像内所有像素点的像素值从k帧中的θ₁灰阶跳变为k+1帧中的θ₂灰阶的次数，将∑E(θ₁→θ₂)的值记录在方阵H的对应单元

中；方阵

是对训练样本的1～100幅图像内

值的累加，

中记录了训练样本图像内检测到的像素值从θ₁灰阶跳变为θ₂灰阶的总次数；将

的值归一化为[0,1]之间的概率值，即得到背景模型学习率

是大小为256×256的方阵。in,

and

Represent the pixel values of any pixel point (x, y) in the image in the kth frame and the k+1th frame, and are abbreviated as θ ₁ and θ ₂ respectively. Since the RGB channels of the example image are all 8-bit deep, that is, the pixel values in each channel have 256 grayscale levels, so: θ ₁ ∈ [0, 255], θ ₂ ∈ [0, 255]; E(θ ₁ →θ ₂ ) = 1 means that the following event is detected once: the pixel value of (x, y) jumps from the grayscale θ ₁ in the k frame to the grayscale θ ₂ in the k+1 frame; ∑E(θ ₁ →θ ₂ ) is the number of times the pixel values of all pixels in the image jump from the grayscale θ ₁ in the k frame to the grayscale θ ₂ in the k+1 frame, and the value of ∑E(θ ₁ →θ ₂ ) is recorded in the corresponding unit of the square matrix H.

Middle; Square

It is the number of images in the 1 to 100 training samples.

The accumulation of values,

The total number of times the pixel value detected in the training sample image jumps from grayscale θ ₁ to grayscale θ ₂ is recorded in

The value of is normalized to a probability value between [0,1], that is, the background model learning rate is obtained

is a square matrix of size 256×256.

和

两个值域取值范围的相同半径

第一个值域范

θ₁,θ₂∈[0,255]；S23: According to the same method as steps S21 to S22 above, the initialization background model of the image sequence on the G channel of RGB and its learning rate are calculated, that is, the initialization multi-channel bimodal background model of the G channel and the center value of the two value ranges of the initialization multi-channel bimodal background model of the G channel are obtained.

and

The same radius of the two ranges

The first range

θ ₁ ,θ ₂ ∈[0,255];

和

两个值域取值范围的相同半径

第一个值域范围

θ₁,θ₂∈[0,255]。S24: According to the same method as steps S21 to S22 above, the initialization background model of the image sequence on the B channel of RGB and its learning rate are calculated, that is, the initialization multi-channel bimodal background model of the B channel and the center value of the two value ranges of the initialization multi-channel bimodal background model of the B channel are obtained.

and

The same radius of the two ranges

The first value range

θ ₁ ,θ ₂ ∈[0,255].

S3: Continuous real-time update of the above multi-channel bimodal background model:

S31: On the R channel, the center value of the multi-channel bimodal background model is continuously updated as follows:

When the 101st frame is newly read, for each pixel point (x, y) in the video field of view, the first center value and the second center value of the multi-channel bimodal background model at its position are updated according to the following formulas:

S32: On the R channel, continuously update the radius value of the multi-channel dual-modal background model, as follows:

When the 101st frame is newly read, for each pixel point (x, y) in the video field of view, the radius value of the unimodal background model at its position is updated according to the following formula:

and

和

每个值域取值范围的半径均

On the R channel, when the 101st frame is newly read, the multi-channel bimodal background model at each pixel point (x, y) in the video field of view is updated as follows: the background model is composed of two value ranges, and the center values of the two value ranges are

and

The radius of each value range is

S33: On the R channel, the learning rate of the multi-channel bimodal background model is continuously updated as follows:

When the 101st frame is newly read, the probability that the pixel values of all pixels in odd rows and columns in the image transition from grayscale θ ₁ to grayscale θ ₂ in images 2 to 101 is calculated on the R channel to generate the learning rate of the multi-channel bimodal background model at the 101st frame shared by the pixels in the image

Similarly, when the 100+i frame is newly read, the multi-channel bimodal background model at the 100+i frame time is continuously updated by the same method as in the above steps S31 to S34. The background model can be represented by two values:

与

S34: According to the method in the above steps S31 to S33, the multi-channel bimodal background model and the background model learning rate of the image sequence on the G channel are continuously updated, which are respectively:

and

与

S35: According to the method in steps S31 to S33 above, the multi-channel bimodal background model and the background model learning rate of the image sequence on the B channel are continuously updated, which are respectively:

and

是大小为255×255的方阵，由于θ₁、θ₂分别是该方阵的行坐标和列坐标，因此将θ₁、θ₂的具体值代入

即可获取方阵中第θ₁行、第θ₂列的单元位置上对应的背景模型学习率；根据图3的示例，

的值就是该方阵中第160行、第200列的单元位置上对应的背景模型学习率，即0.5。As mentioned earlier,

is a square matrix of size 255×255. Since θ ₁ and θ ₂ are the row coordinates and column coordinates of the square matrix respectively, the specific values of θ ₁ and θ ₂ are substituted into

The background model learning rate corresponding to the unit position at the θ _1th row and the θ _2th column in the square matrix can be obtained; according to the example in Figure 3,

The value of is the background model learning rate corresponding to the unit position of the 160th row and the 200th column in the matrix, which is 0.5.

S4: Use the multi-channel bimodal background model updated in real time to perform segmentation detection on the real-time input image.

In a specific implementation, real-time detection of two-photon images of brain neurons can determine active neurons and inactive neurons. If the pixel value of a pixel is within the two value ranges of the multi-channel bimodal background model, the pixel is regarded as an inactive neuron; if the pixel value of a pixel is not within the two value ranges of the multi-channel bimodal background model, the pixel is regarded as an active neuron.

The results obtained by the method of the present invention according to the embodiment are shown in Figure 5. It can be seen that since the method is designed for the data characteristics of the multi-color two-photon image sequence and has been specially optimized, the foreground (i.e., the white pixel area) segmented is generally consistent with the target object to be detected, and there are fewer cases of missed detection (i.e., the foreground pixel that should be marked as white is marked as black representing the background) and false detection (i.e., the pixel that should be marked as black background is marked as white representing the foreground).

At the same time, a certain general image segmentation method in the field of intelligent video surveillance is selected for comparison, and the results obtained according to the embodiment are shown in Figure 6. It can be seen that since this method is not designed for the data characteristics of multi-color two-photon image sequences, the foreground segmented by it is not consistent with the target object to be detected, and there are many missed detection areas.

In addition, a general static image segmentation method for a single image is selected for comparison, and the results obtained according to the embodiment are shown in Figure 7. It can be seen that the foreground segmented by this method is less consistent with the target object to be detected, and more missed detection areas appear.

In summary, the qualitative comparison results between the method of the present invention and the above two general image segmentation methods are shown in Table 1.

Table 1

Comparison of different image segmentation methods Qualitative comparison of image segmentation results The present invention proposes a method very good A general image segmentation method in the field of intelligent video surveillance as a comparison Poor A general static image segmentation method for single image as a comparison Poor

From the results, it can be seen that the present invention can solve the problem of lack of adaptive image segmentation specifically designed for the characteristics of multi-color two-photon image sequence data, overcome the problem that some existing methods are unable to adapt to and utilize the characteristics of multi-color two-photon image sequence data, improve the accuracy of image segmentation, and achieve outstanding and significant technical effects.

Claims (3)

1. An adaptive image segmentation method for multi-color two-photon image sequences, characterized in that the method comprises the following steps: S1: Select the kth to nth images from the multi-color two-photon image sequence as training samples; S2: A multi-channel bimodal background model initialized by training samples; S3: continuous real-time update of the above multi-channel bimodal background model; S4: Use the multi-channel bimodal background model updated in real time to perform segmentation detection on the real-time input image; The step S2 comprises the following steps: S21: Construct a multi-channel bimodal background model initialized on the R channel of the image sequence: S211, on the R channel, for each pixel point (x, y) in the image, calculate the two center values of the initialized multi-channel bimodal background model at the position of the pixel point (x, y), the method is as follows: ① If the pixel point (x, y) is located on the edges of the image, the median and mode of all pixel values J(x, y) _k , J(x, y) _k+1 , ..., J(x, y) _n of the pixel point (x, y) in all images of the training sample are calculated, where J(x, y) _k represents the pixel value of the pixel point (x, y) in the kth image, and the median and mode are used as the first center value and the second center value of the initialized multi-channel bimodal background model at the position of the pixel point (x, y), respectively; ② If the pixel point (x, y) is not located on the edge of the image, the median and mode of all pixel values in the 3×3 neighborhood of all training sample images with the pixel point as the center are calculated, and the median and mode are used as the first center value and the second center value of the initialized multi-channel bimodal background model at the pixel point (x, y) position; thus, the first center value and the second center value of the multi-channel bimodal background model at the pixel point (x, y) position are obtained.

and

S212, on the R channel, calculate the radius value of the initialized multi-channel bimodal background model shared by the pixels in all k to n images of the training sample, and the calculation method is as follows: ① For each image in the training sample, use the image edge detection algorithm to find the non-edge pixels in the image, and form a set of all non-edge pixels in each image. The set of non-edge pixels in the zth image is recorded as

② In all k to n images of the training sample, in the set of non-edge pixels of each image, calculate the radius value of the initialized multi-channel bimodal background model shared by the pixels according to the following formula:

and

Represents the pixel value of the pixel point (x, y) in the z-th image, and V is the upper limit of the pixel value in the image; S213, the initialized multi-channel bimodal background model of the R channel at the pixel point (x, y) position in the image is constructed as follows: The initialized multi-channel bimodal background model is composed of two value ranges, and the center values of the two value ranges are respectively

and

The radius of each value range is the radius value

The first range is

The second range is

S22: On the R channel, the learning rate of the initialized multi-channel bimodal background model is calculated as follows: In all images of the training sample, the probability of the pixel values of all pixels in the image transitioning from grayscale θ ₁ to grayscale θ ₂ is calculated on the R channel to generate the learning rate of the multi-channel bimodal background model shared by the pixels in the image at the nth image moment

Wherein θ ₁ represents the grayscale level before the pixel value transition, θ ₂ represents the grayscale level after the pixel value transition, θ ₁ ,θ ₂ ∈[0,255]; S23: According to the same method as steps S21 to S22 above, the initialization background model of the image sequence on the G channel and its learning rate are calculated, that is, the initialization multi-channel bimodal background model of the G channel and the center value of the two value ranges of the initialization multi-channel bimodal background model of the G channel are obtained.

and

The same radius of the two ranges

The first value range

The second range

And the learning rate is

Where θ ₁ ,θ ₂ ∈[0,255]; S24: According to the same method as steps S21 to S22 above, the initialization background model of the image sequence on the B channel and its learning rate are calculated, that is, the initialization multi-channel bimodal background model of the B channel and the center value of the two value ranges of the initialization multi-channel bimodal background model of the B channel are obtained.

and

The same radius of the two ranges

The first value range

The second range

And the learning rate is

Where θ ₁ ,θ ₂ ∈[0,255]; The step S3 comprises the following steps: S31: On the R channel, the center value of the multi-channel bimodal background model is continuously updated as follows: When the n+1th image of the training sample is newly read, for each pixel point (x, y) in the image, the first center value and the second center value of the multi-channel bimodal background model at the pixel point position are updated respectively using the following formulas:

in,

and

are the two center values of the multi-channel bimodal background model of the pixel point (x, y) in the n+1th image,

and

are the two central values of the multi-channel bimodal background model and the background model learning rate of the pixel point (x, y) in the nth image,

is the pixel value of the pixel point (x, y) in the n+1th image; in formula (1), the value of θ ₁ is

In formula (2), the value of _θ1 is

And θ ₂ is always

S32: On the R channel, continuously update the radius value of the multi-channel dual-modal background model, as follows: When the n+1th image is newly read, for each pixel point (x, y) in the video field of view, the radius value of the unimodal background model at the pixel point position is updated:

and

in,

is the radius value of the multi-channel bimodal background model at any pixel point at frame n+1; S33: On the R channel, when the n+1 frame is newly read, the multi-channel bimodal background model at each pixel point (x, y) in the video field of view is updated: S34: On the R channel, the learning rate of the multi-channel bimodal background model is continuously updated, and the method is as follows: when the n+1th image is newly read, the probability that the pixel values of all the pixels located in odd rows and odd columns in the image transition from the grayscale of θ 1 to the grayscale of θ ₂ in the k+1 to n+ ₁ images is calculated on the R channel, and the learning rate of the multi-channel bimodal background model at the n+1th frame shared by the pixels in the image is generated.

S34: According to the method in steps S31 to S34 above, continuously update the multi-channel bimodal background model and the background model learning rate of the image sequence on the G channel; S35: According to the method in steps S31 to S34 above, continuously update the multi-channel bimodal background model and the background model learning rate of the image sequence on the B channel; Repeat the above steps to iterate and continuously update the multi-channel bimodal background model on the three RGB channels of each image in the image sequence. 2. The adaptive image segmentation method for multi-color two-photon image sequences according to claim 1, characterized in that: the step S1 comprises the following steps: S11: Select continuous images from the kth to the nth images in the multi-color two-photon image sequence as training samples; S12: If the value range of the original pixel value of the image in the training sample is not [0,255], then: The training samples are preprocessed to map the value range of the pixels on each color channel of each image in the training samples to the value range of [0,255]. The specific method is as follows:

Among them, U represents the upper limit of the value range of the original pixel value of the image in the training sample, I(x,y) is the pixel value of the pixel point (x,y) in the image before preprocessing, and J(x,y) is the pixel value of the pixel point (x,y) in the image after preprocessing; If the value range of the original pixel values in the training sample is [0,255], no preprocessing is performed on the training sample. 3. According to claim 1, an adaptive image segmentation method for multi-color two-photon image sequences is characterized in that: step S4 specifically uses the value range of the multi-channel bimodal background model to process and judge each pixel point of the image: if the pixel value of the pixel point is within the two value ranges of the multi-channel bimodal background model, then the pixel point is used as the background; if the pixel value of the pixel point is not within the two value ranges of the multi-channel bimodal background model, then the pixel point is used as the foreground.

Images