CN101042771A - Medicine image segmentation method based on intensification learning - Google Patents

Abstract

The invention discloses a medical image segmentation method based on reinforcement learning. Through different image samples, according to the defined state, action and reward, learning is carried out through the interaction between reinforcement learning and the environment, and the optimal behavior strategy is learned by trial and error. . Finally, a new image segmentation method based on reinforcement learning is formed, which uses the learned knowledge to segment similar medical images. The invention has the advantages of being able to effectively distinguish the nucleus and the cytoplasm, carry out self-adaptive and incremental learning, and accurately segment images.

Description

Medical Image Segmentation Method Based on Reinforcement Learning

technical field

The invention relates to an incremental medical image segmentation method based on reinforcement learning.

Background technique

The precise segmentation of image objects is one of the important foundations of medical image pattern recognition. In the application of lung cancer cell recognition, in most lung cancer cell images, the contrast between the nucleus and the cytoplasm is low, the boundary between the nucleus edge and the background is blurred, and the influence of background impurities and noise makes it difficult to identify Lung cancer cell images are more accurately segmented.

Traditional image segmentation methods are mainly divided into threshold-based segmentation and gradient-based segmentation. The former cannot accurately segment the target area for images with multi-peak grayscale histograms. The latter also cannot segment the target area very well for the case where the target and background gray levels are close. In addition, due to the great difference in image acquisition of lung cancer cytopathological images, it is difficult for ordinary image segmentation methods to adapt to such a complex environment.

At present, reinforcement learning has been widely used in prediction, intelligent control, image processing and many other fields. Compared with traditional image segmentation methods, medical image segmentation methods based on reinforcement learning have incremental learning capabilities, can adapt to complex environments, and make correct segmentation of medical lung cancer cell images.

Contents of the invention

Purpose of the invention: The purpose of the present invention is to address the deficiencies of the prior art and provide a medical image segmentation method based on reinforcement learning that can make correct segmentation.

Technical solution: The present invention uses different image samples, according to the defined state, action and reward, learns through the interaction between reinforcement learning and the environment, and learns the optimal behavior strategy by trial and error. Finally, a new image segmentation method based on reinforcement learning is formed, which uses the learned knowledge to segment similar medical images. The method includes the following steps: (1) Initialize the Q matrix, and the Q matrix is recorded in the form of a two-dimensional array in the current state and all subsequent states in the current state and all subsequent states to select actions to obtain cumulative rewards; (2) Sobel the new sample image (A commonly used edge detection operator) edge detection to obtain the edge image; (3) Segment the new sample image using the method of maximum variance between classes to obtain a binary image containing the nucleus and cytoplasm; (4) Define the state as The ratio of the overlap between the target contour edge of the current threshold segmentation result and the edge obtained by sobel edge detection and the overlap ratio of the target area area of the current threshold segmentation result and the target area segmented by the method of maximum variance between classes; define the action as the current threshold increase Or reduce the gray level represented by the action A _i , A=[-30-10-5-1 0 1 5 10 30]; define the reward as the degree of conformity between the currently segmented target area and the actual optimal segmentation of the image; (5 ) Calculate the state and reward corresponding to each segmentation threshold of 0-255, so that each threshold corresponds to the state and reward (6) Repeat step (7) until the mean square error of the latest 10 average Q matrix updates is less than 0.005; ( 7) Given an initial threshold, repeat steps (8) to (10) until the threshold changes to the optimal segmentation threshold; (8) Get the current state according to the threshold; (9) Adopt ε-greedy strategy (ε-greedy strategy with The probability of 1-ε selects the action with the largest reward in the Q-matrix, and selects other actions with the probability of ε) Select an action and change the threshold; (10) Get the corresponding feedback reward to update the Q matrix according to the new threshold after changing the threshold; (11 ) Repeat (2)-(10) for the new sample image.

Beneficial effect: the remarkable advantage of the present invention is that it can effectively distinguish the nucleus and the cytoplasm, learn adaptively and incrementally, and accurately segment the image.

Description of drawings

Fig. 1 is the framework model of the method of the present invention.

Fig. 2 is a structural diagram of modules in the method of the present invention.

Fig. 3 is a flowchart of the method of the present invention.

Figure 4 is the edge image obtained by the sobel operator for edge detection.

Figure 5 is the resulting image of the between-class variance maximum segmentation.

Figure 6 is the optimal segmentation result

Detailed ways

As shown in Figure 1, the framework model of the method of the present invention.

As shown in Figure 2, the method of the present invention includes a state perception module, an action selection module, a policy update module, a reward perception module and an image segmentation module.

The process flow of the present invention is as shown in Figure 3, described in detail below:

Step 1. Initialize the Q matrix. The Q matrix records in the form of a two-dimensional array the cumulative rewards obtained in the current state and all subsequent states to select actions with strategy π.

Step 2, use the sobel operator (a commonly used edge detection operator) to perform edge detection on the new sample image to obtain an edge image. The image of the edge detection result is shown in Figure 4.

Step 3: Carry out maximum variance segmentation between classes on the new sample image to obtain a binary image including nucleus and cytoplasm. The resulting image of the between-class variance maximum segmentation is shown in Fig. 5.

Step 4, define the state S as the ratio E of the overlap between the target contour edge of the current threshold segmentation result and the edge detected by sobel edge detection, and the overlap of the target area area of the current threshold segmentation result and the target area segmented by the maximum variance method between classes Ratio F, that is, S=(E×F); define the action as the gray level represented by the current threshold increase or decrease action A _i , A=[-30-10-5-1 0 1 5 10 30]; define the reward R is the degree of conformity between the currently segmented target area and the actual optimal segmentation of the image.

$\mathrm{<span\; class="notranslate">\; E.</span>}<span\; class="notranslate">\; =</span>\frac{<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; Edge</span>}}_{\mathrm{<span\; class="notranslate">\; T</span>}}\mathrm{<span\; class="notranslate">\; I</span>}{\mathrm{<span\; class="notranslate">\; Edge</span>}}_{\mathrm{<span\; class="notranslate">\; S</span>}}<span\; class="notranslate">\; |</span>}{<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; Edge</span>}}_{\mathrm{<span\; class="notranslate">\; S</span>}}<span\; class="notranslate">\; |</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>$

Edge _T is the edge of the current segmentation, and Edge _s is the edge extracted by edge detection.

$\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; =</span>\frac{<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; Front</span>}}_{\mathrm{<span\; class="notranslate">\; T</span>}}\mathrm{<span\; class="notranslate">\; I</span>}{\mathrm{<span\; class="notranslate">\; Front</span>}}_{\mathrm{<span\; class="notranslate">\; OSTU</span>}}<span\; class="notranslate">\; |</span>}{<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; Front</span>}}_{\mathrm{<span\; class="notranslate">\; OSTU</span>}}<span\; class="notranslate">\; |</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>$

Front _T is the target area currently segmented, and Front _OSTU is the target area segmented by the method of maximal variance between classes (OSTU).

$\mathrm{<span\; class="notranslate">\; R</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 100</span>}<span\; class="notranslate">\; \&times;</span>\frac{<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; o</span>}}\mathrm{<span\; class="notranslate">\; I</span>}{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; T</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; +</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; o</span>}}\mathrm{<span\; class="notranslate">\; I</span>}{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; T</span>}}<span\; class="notranslate">\; |</span>}{<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; o</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; o</span>}}<span\; class="notranslate">\; |</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span>$

B _O is the optimally segmented background, and F _O is the optimally segmented foreground object. _BT is the background of the current segmentation, and _FT is the foreground target of the current segmentation.

Step 5. Calculate the state and reward corresponding to each segmentation threshold of 0-255 according to formulas (1), (2), and (3), so that each threshold corresponds to the state and reward.

Step 6, repeat step (7), until the mean square error of the last 10 average Q matrix updates before and after updating is less than 0.005.

Step 7. Given an initial threshold, repeat steps (8) to (10) until the threshold changes to the optimal segmentation threshold.

Step 8, get the current state according to the current threshold.

Step 9: Use the ε-greedy strategy (the ε-greedy strategy selects the action with the largest reward in the Q-matrix with the probability of 1-ε, and selects other actions with the probability of ε) to select actions and change the segmentation threshold.

Step 10, according to the new threshold after changing the threshold, the corresponding feedback reward is obtained to update the Q matrix. The update formula is as in formula (4): s is the current state, a is the action corresponding to s, s' is the next state after executing action a, and a' is the action corresponding to s'.

$\mathrm{<span\; class="notranslate">\; Q</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&LeftArrow;</span>\mathrm{<span\; class="notranslate">\; Q</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; \&alpha;</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; \&gamma;</span>}\underset{{\mathrm{<span\; class="notranslate">\; a</span>}}^{<span\; class="notranslate">\; \&prime;</span>}}{\mathrm{<span\; class="notranslate">\; max</span>}}\mathrm{<span\; class="notranslate">\; Q</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; the\; s</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; a</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; Q</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span>$

Step 11, repeat steps 2 to 10 for the new sample image.

Claims (1)

1, a kind of medical image cutting method based on intensified learning is characterized in that this method may further comprise the steps:

(1) initialization Q matrix, Q matrix are recorded under current state and all the follow-up states with the two-dimensional array form and go to select to move the accumulation award of doing acquisition with tactful π;

(2) adopt the sobel operator to carry out rim detection to the new samples image, obtain edge image;

(3) the new samples image is carried out the inter-class variance maximum fractionation, obtain comprising the bianry image of nucleus and endochylema;

(4) definition status S is the ratio F that current Threshold Segmentation result's objective contour edge and equitant ratio E in edge that the sobel rim detection obtains and current Threshold Segmentation result's target area area overlaps with target area area that the maximum method of inter-class variance splits, i.e. S=(E * F); The definition action increases or reduces the gray level of action Ai representative, A=[-30-10-5-1 015 10 30 for current threshold value]; Definition award R is the matching degree that current target area that is partitioned into and image actual optimum are cut apart;

(5) state and the award of each segmentation threshold correspondence of calculating 0-255, so each threshold value is corresponding with state and award;

(6) repeating step (7), the mean square deviation before and after 10 times the average Q matrix update is less than 0.005 up to date;

(7) a given initial threshold, the optimum segmentation threshold value is arrived up to changes of threshold in repeating step (8)～(10);

(8) obtain current state according to current threshold value;

(9) adopt ε-greedy policy selection action, change segmentation threshold;

(10) obtain corresponding feedback award according to the new threshold value after the change threshold value and upgrade the Q matrix;

(11) to new samples image repeating step 2 to 10.

Images