Data acquisition method/system based on doctor behaviors and medical image processing system
Technical Field
The invention relates to the field of medical image processing, in particular to a data acquisition method/system based on doctor behaviors and a medical image processing system.
Background
The AI can be trained according to a large number of samples to obtain a prediction model when automatically detecting the non-calcified plaque characteristics, but the accuracy of prediction cannot be effectively improved by the method, and the main reasons are as follows: identification or degree judgment of non-calcified plaques is extremely complex, interference factors are many, higher requirements are put forward for people who annotate samples, and effective information is not extracted when the AI extracts features due to limited marking information of the samples.
Therefore, how to obtain enough samples and how to have more annotation information on the samples are beneficial to improving the accuracy of model prediction.
Disclosure of Invention
The invention aims to provide a data acquisition method/system and a medical image processing system based on doctor behaviors.
In order to achieve the purpose, the invention adopts the following technical scheme:
the data acquisition method based on doctor behaviors comprises the following steps:
tracking an eye movement track of a doctor when the doctor views each medical image based on an eye movement tracking technology, and acquiring a focus coordinate of the doctor on each medical image;
matching each attention coordinate with interface image data thereof to obtain an attention area of each medical image;
calculating candidate target areas of the medical images;
taking intersection of the attention area and the candidate target area of each medical image to obtain the reliable area of each medical image and finish the labeling of the reliable area in the corresponding medical image;
and obtaining a valuable region near the reliable region based on the neighborhood coordinates of the reliable region, and completing the labeling of the valuable region on the corresponding medical image.
Preferably, when tracking the eye movement track of a doctor viewing each medical image based on the eye movement tracking technology, acquiring the observation duration and sequence of the doctor on each medical image; the ease rating of the case is determined and annotated based on the doctor's observation duration or sequence or a combination of observation duration and sequence for each medical image.
Preferably, when tracking the eye movement track of a doctor viewing each medical image based on the eye movement tracking technology, acquiring the observation duration of the doctor on each medical image; and carrying out weight assignment on the reliability of the detection result of each medical image, wherein the weight is the product of the time weight and the initial weight.
Preferably, the initial weight is obtained by:
giving a test set, wherein each case of data in the test set comprises N kinds of medical images, and N is a natural number greater than 1;
each medical image is given a weight combination [ W1, WN ]]W1+ … … + WN equals 1, and W1, WN are combined for the weight]W in (1) are combined and tried according to set step length respectively to obtain an optimal weight combination so that Sum (R)i-Ri') minimum, RiBelongs to R, R is a prediction result set of a test set, Ri' belongs to R ', and R ' is an answer set of the test;
and dividing the optimal weight by the time weight of the corresponding image to obtain the initial weight of each image.
The invention also discloses a data acquisition system based on doctor behaviors, which comprises:
the eye tracking module tracks eye movement tracks when a doctor views each medical image and acquires the attention coordinates of the doctor to each medical image; matching each attention coordinate with interface image data thereof to obtain an attention area of each medical image;
a candidate region calculation module that calculates a candidate target region of each medical image;
the reliable region labeling module is used for taking intersection of the attention region and the candidate target region of each medical image to obtain the reliable region of each medical image and finish labeling of the reliable region in the corresponding medical image;
the valuable region labeling module acquires a valuable region near the reliable region based on the neighborhood coordinates of the reliable region and finishes labeling of the valuable region on the corresponding medical image;
and the sample output by the sample output module comprises the reliable region label and the valuable region label.
Preferably, the system further comprises a difficulty and ease labeling module, wherein the eye movement tracking module is used for acquiring the observation duration and sequence of each medical image of a doctor when tracking the eye movement track of the doctor when viewing each medical image; the difficulty marking module determines and marks the difficulty rating of the case based on the observation duration or sequence of each medical image or the combination of the observation duration and the sequence of each medical image; the samples output by the sample output module comprise the reliable region labels, the valuable region labels and the difficulty rating labels.
Preferably, the system further comprises a weight labeling module, and the eye movement tracking module is used for acquiring the observation duration of each medical image by a doctor when tracking the eye movement track of the doctor viewing each medical image; the weight labeling module endows each medical image with a weight of reliability, wherein the weight is the product of time weight and initial weight; and the samples output by the sample output module comprise the reliable region labels, the valuable region labels and the weight labels of all medical images.
The invention also discloses a medical image processing system, which comprises an input module of images to be predicted and a non-calcified plaque detection model;
the non-calcified plaque prediction model is trained by using the data acquired as before as a training sample;
and inputting the image to be predicted into the image input module to be predicted, and predicting the image to be predicted by the non-calcified plaque prediction model to give a prediction result.
After adopting the technical scheme, compared with the background technology, the invention has the following advantages:
1. the invention captures the behaviors of doctors and specialists based on the eye tracking technology, automatically generates time, frequency, sequence, focus points and nearby points of the focus points for the doctors to observe various images, extracts reliable regions and valuable regions according to the acquired information, and can automatically acquire a large number of high-value samples.
2. The method has the advantages that the obtained sample information is rich, meanwhile, the difficulty rating is included, and the difficult sample can be trained in a targeted manner according to the difficulty rating, so that the prediction result is more accurate.
3. The invention gives weight information, and finally applies the weight information to model training, and can give the same weight to the fusion of the prediction results of all images for similar data, so that the output result is closer to the real evaluation result of the doctor expert.
Drawings
FIG. 1 is a block diagram of the data acquisition system based on doctor's behavior according to the present invention.
Fig. 2 is a block diagram of the medical image processing system of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
In the present invention, it should be noted that the terms "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. are all based on the orientation or positional relationship shown in the drawings, and are only for convenience of describing the present invention and simplifying the description, but do not indicate or imply that the apparatus or element of the present invention must have a specific orientation, and thus, should not be construed as limiting the present invention.
Example 1
The invention discloses a data acquisition method based on doctor behaviors, which comprises the following steps:
tracking an eye movement track of a doctor when the doctor views each medical image based on an eye movement tracking technology, and acquiring a focus coordinate of the doctor on each medical image;
matching each attention coordinate with interface image data thereof to obtain an attention area of each medical image;
calculating candidate target areas of the medical images;
taking intersection of the attention area and the candidate target area of each medical image to obtain the reliable area of each medical image and finish the labeling of the reliable area in the corresponding medical image;
and obtaining a valuable region near the reliable region based on the neighborhood coordinates of the reliable region, and completing the labeling of the valuable region on the corresponding medical image.
The eye tracking technology belongs to the prior art, and is not described herein.
The calculation of candidate target regions for each medical image is also known in the art, and can be understood as the detection of non-calcified plaque of a blood vessel, which can be calculated as follows:
1. the deep learning network marks 1 on the non-calcified area, and marks 0 on the negative sample which is not the non-calcified area; and then, classifying and annotating the non-calcified areas by using a classification net.
2. The traditional algorithm is used for segmenting the blood vessel region, and the segmentation can be a deep learning or traditional segmentation method. After the segmentation result is obtained, it is checked whether or not there is a morphological "dip" in the vertical direction of the blood vessel. The definition of such a recess is: the width of the blood vessel at the location is lower than the width of the blood vessel on both sides of the location.
The neighborhood coordinates of the reliable region are obtained by presetting a threshold, and the embodiment is preferably 30 pixels of the edge of the reliable region. The significance of obtaining the region of value is: the available judgment information is introduced. Because a sample is prepared for the neural network, the neural network needs more information to judge the focus, not only the image information of the focus itself, but also the image information beside the focus needs to be used, and the image beside the focus has more information to assist in judgment, so that the value is more valuable, and the accuracy of model judgment can be improved.
Preferably, when tracking the eye movement track of a doctor viewing each medical image based on the eye movement tracking technology, acquiring the observation duration and sequence of the doctor on each medical image; the ease rating of the case is determined and annotated based on the doctor's observation duration or sequence or a combination of observation duration and sequence for each medical image.
Wherein the difficulty rating may be determined from the observation duration. Firstly, presetting a plurality of different difficulty grades, such as simplicity, generality and complexity, setting different time thresholds for each difficulty grade, and obtaining corresponding difficulty grades when the total observation time of a doctor on a medical image exceeds the corresponding time threshold.
The setting may be performed according to the sequence. For example, for a single sequence (i.e., for each medical image, the physician only views once), it can be defined as simple; a dual sequence (i.e. for each medical image, the doctor views all the medical images and then views one or more medical images again) can be defined as general; for multiple sequences (i.e., for each medical image, the physician repeatedly views between some of the medical images after having viewed all of the medical images), a complexity may be defined.
The determination can also be made by combining the observation duration and the sequence, and the final conclusion can be obtained if the observation duration and the sequence respectively account for half of the weight.
Preferably, when tracking the eye movement track of a doctor viewing each medical image based on the eye movement tracking technology, acquiring the observation duration of the doctor on each medical image; and giving a weight to the reliability of the detection result of each medical image, wherein the weight is the product of the time weight and the initial weight.
For example, a medical image typically includes original, CPR, short axis and straightening, substituted with 1, 2, 3, 4, respectively, and the sequence and time of interest may be such that: 1(20s)2(30s)3(10s)4(10s)2(10s)1(5s)2(5 s); the 1, 2, 3 and 4 can set an initial weight, the duration can also be the weight, and the weight can be obtained by multiplying the two.
The initial weight is obtained by:
given a test set, N medical images exist in each case of data in the test set, and N is a natural number greater than 1. Each medical image is given a weight combination [ W1, WN ]]W1+ … … + WN equals 1, and W1, WN are combined for the weight]In (1)W combines the trial according to the set step length respectively to obtain an optimal weight combination so that Sum (R)i-Ri') minimum, RiBelongs to R, R is a prediction result set of the test set (the prediction result is a fusion result of detection results of all medical images), and R isi' belongs to R ', and R ' is an answer set of the test; and dividing the optimal weight by the time weight of the corresponding image to obtain the initial weight of each image.
In this embodiment, N is 4, i.e., the original image, CPR, short axis, and straightened image are included. When A, B, C and D represent the detection results of the medical images and W represents the confidence weight, the fusion results of the detection results of the medical images include:
R1=W1*A1+W2*B1+W3*C1+W4*D1;
R2=W1*A2+W2*B2+W3*C2+W4*D2;
……;
finding the optimal combination of W1, W2, W3 and W4, dividing the W1, W2, W3 and W4 by the time weights Wt1, Wt2, Wt3 and Wt4 to obtain the initial weight setting: wi1, Wi2, Wi3 and Wi 4.
Example 2
The invention also discloses a data acquisition system based on doctor behaviors, which comprises:
the eye tracking module tracks eye movement tracks when a doctor views each medical image and acquires the attention coordinates of the doctor to each medical image; matching each attention coordinate with interface image data thereof to obtain an attention area of each medical image;
a candidate region calculation module that calculates a candidate target region of each medical image;
the reliable region labeling module is used for taking intersection of the attention region and the candidate target region of each medical image to obtain the reliable region of each medical image and finish labeling of the reliable region in the corresponding medical image;
and the valuable region labeling module acquires a valuable region near the reliable region based on the neighborhood coordinates of the reliable region and finishes labeling of the valuable region on the corresponding medical image.
The difficulty labeling module is used for tracking the eye movement track of the doctor when viewing each medical image, and acquiring the observation time length and sequence of the doctor on each medical image; and the difficulty marking module determines the difficulty rating of the case and marks the case based on the observation time length or the sequence of each medical image or the combination of the observation time length and the sequence of each medical image.
The eye tracking module is used for tracking the eye movement track of a doctor viewing each medical image and acquiring the observation duration of the doctor on each medical image; the weight labeling module endows each medical image with a weight of reliability, wherein the weight is the product of time weight and initial weight; and the samples output by the sample output module comprise the reliable region labels, the valuable region labels and the weight labels of all medical images.
And the sample output module is used for outputting the generated samples, and the output samples comprise the labeling information generated by each labeling module.
Example 3
The invention also discloses a medical image processing system which comprises an image input module to be predicted and a non-calcified plaque detection model.
The non-calcified plaque prediction model is trained by using the data acquired as before as a training sample; the trained model can classify the input images, give the detection results and the weights of the medical images in the type of images, and finally output the final prediction results.
And inputting the image to be predicted into the image input module to be predicted, and predicting the image to be predicted by the non-calcified plaque prediction model to give a prediction result.
During training, the prediction model performs different training according to the difficulty of the sample and the combination of a preset strategy.
For example, for a sample defined as simple, it is trained only once per training period, for a sample defined as general, it is randomly increased by a times of training per training period, and for a sample defined as complex, it is randomly increased by 2a times of training per training period. The value of a is preset, and can be 1 or 2 generally.
The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.