CN111861999A

CN111861999A - Detection method and device for artery and vein cross compression sign, electronic equipment and readable storage medium

Info

Publication number: CN111861999A
Application number: CN202010590521.2A
Authority: CN
Inventors: 刘佳; 杨叶辉; 孙旭; 王磊
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2020-06-24
Filing date: 2020-06-24
Publication date: 2020-10-30

Abstract

The application discloses a method and a device for detecting arteriovenous crossing compression sign, electronic equipment and a readable storage medium, and relates to the technical field of deep learning, the technical field of computer vision and the field of AI medical treatment. The specific implementation scheme is as follows: performing blood vessel segmentation on an image to be detected to obtain a blood vessel segmentation result of whether each pixel in the image to be detected is a blood vessel; determining a blood vessel intersection point according to the blood vessel segmentation result; obtaining a region of interest ROI image of the blood vessel intersection point from the image to be detected; classifying the ROI image by using a classification model to obtain a classification result of whether the ROI image is subjected to artery and vein cross compression; the classification model is obtained by training in advance based on a supervised training mode. The accuracy of the detection result of the arteriovenous cross compression can be improved.

Description

Detection method and device for artery and vein cross compression sign, electronic equipment and readable storage medium

Technical Field

The invention relates to the technical field of image processing, in particular to the technical field of deep learning, the technical field of computer vision and the field of AI medical treatment, and particularly relates to a method and a device for detecting arteriovenous crossing compression sign, an electronic device and a readable storage medium.

Background

The AV nicking is a phenomenon in which a vein is compressed by a hardened artery at the intersection of the vein and the artery due to an increase in blood pressure (i.e., high blood pressure). In the retinal color photograph, the arteriovenous cross-compression is characterized by a decrease in venous calibre on both sides of an Arteriovenous (AV) intersection. As shown in fig. 1(a), a schematic diagram of a normal arteriovenous crossing condition, fig. 1(b) a schematic diagram of an arteriovenous crossing compression feature, and fig. 1(c) a retinal image.

The arteriovenous cross compression sign of the fundus of the human body is not only related to the current blood pressure of the human body, but also related to the past blood pressure, and is a durable and long-term marker of hypertension. Meanwhile, the cross-compression of arteries and veins is very important for the early and timely discovery of high risk groups of cardiovascular diseases.

In the prior art, when the artery and vein cross compression characterization is detected on an fundus image, an unsupervised K mean (Kmeans) algorithm is used for carrying out artery and vein vessel segmentation, a vessel topological graph is established for all vessels within a certain radius range of the center of a video disc, and artery and vein cross compression characterization nodes are identified according to the vessel topological graph and the artery and vein vessel classification.

However, the fundus image quality difference is large due to the influence of the shooting equipment and the shooting operation, meanwhile, the robustness of the arteriovenous vessel segmentation performed by adopting an unsupervised Kmeans algorithm is poor due to the large difference of the fundus structure of each human eye and possible fundus diseases, and the accuracy of the detection result of the arteriovenous cross stress is directly influenced due to the inaccuracy of arteriovenous vessel information; in addition, the gray values of the arteriovenous vessels in the fundus image are relatively close, so that the arteriovenous vessels have low display difference, the accuracy of the arteriovenous vessel segmentation result is reduced, and the accuracy of the arteriovenous vessel cross-compression feature detection result is influenced.

Disclosure of Invention

Aspects of the present application provide a method and an apparatus for detecting arteriovenous cross constriction sign, an electronic device, and a readable storage medium, so as to improve accuracy of an arteriovenous cross constriction sign detection result.

According to a first aspect, a method for detecting an arteriovenous crossing compression sign is provided, which comprises the following steps:

performing blood vessel segmentation on an image to be detected to obtain a blood vessel segmentation result of whether each pixel in the image to be detected is a blood vessel;

determining a blood vessel intersection point according to the blood vessel segmentation result;

Obtaining a region of interest ROI image of the blood vessel intersection point from the image to be detected;

classifying the ROI image by using a classification model to obtain a classification result of whether the ROI image is subjected to artery and vein cross compression; the classification model is obtained by training in advance based on a supervised training mode.

According to a second aspect, there is provided a device for detecting an arteriovenous crossing compression sign, comprising:

the segmentation unit is used for performing blood vessel segmentation on the image to be detected to obtain a blood vessel segmentation result of whether each pixel in the image to be detected is a blood vessel;

a determining unit for determining a vessel intersection point according to the vessel segmentation result;

an acquisition unit, configured to acquire a region of interest ROI image of the blood vessel intersection from the image to be detected;

the classification model is used for classifying the ROI image to obtain a classification result of whether the ROI image is subjected to artery and vein cross compression; the classification model is obtained by training in advance based on a supervised training mode.

According to a third aspect, there is provided an electronic device comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

The memory stores instructions executable by the at least one processor to cause the at least one processor to perform the method of the aspects and any possible implementation described above.

According to a fourth aspect, there is provided a non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of the aspects and any possible implementation as described above.

According to the technical scheme, the vessel segmentation is carried out on the image to be detected to obtain the vessel segmentation result of whether each pixel in the image to be detected is a vessel, then the vessel intersection point is determined according to the vessel segmentation result, and then the ROI image of the region of interest of the vessel intersection point is obtained from the image to be detected, and then the ROI image is classified by utilizing a classification model obtained based on supervised training mode training to obtain the classification result of whether the ROI image is a vessel cross compression sign, so that the detection of the vessel cross compression sign node in the image to be detected is realized. In the scheme, the artery and vein vessel segmentation is not needed, only whether the vessel is segmented or not is needed, the relative difficulty is small, the segmentation is easy, the segmentation robustness is good, the accuracy of the classification result of the artery and vein cross compression sign is improved, and the accuracy of the artery and vein cross compression sign detection result is improved.

In addition, by adopting the technical scheme provided by the application, the classification model is obtained by adopting a deep learning technology in advance and training in a supervision mode, and the ROI image of the blood vessel intersection is classified into the artery and vein cross stress sign, so that the accuracy of the classification result is improved, and the accuracy of the artery and vein cross stress sign detection result is improved.

In addition, by adopting the technical scheme provided by the application, the blood vessel intersection point is determined according to the blood vessel segmentation result, the ROI image of the blood vessel intersection point is used as the source data of the classification of the arteriovenous crossing compression sign, the influence of the wrong judgment result of the blood vessel intersection point on the classification result of the arteriovenous crossing compression sign is small, and the actual judgment logic of the arteriovenous crossing compression sign is better met.

It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present application, nor do they limit the scope of the present application. Other features of the present application will become apparent from the following description.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to the drawings without inventive labor. The drawings are only for the purpose of illustrating the present invention and are not to be construed as limiting the present application. Wherein:

FIG. 1(a) is a schematic view of a normal arteriovenous crossing condition;

FIG. 1(b) is a schematic view of an arteriovenous crossing compression feature;

FIG. 1(c) is a retinal image;

FIG. 2 is a schematic diagram according to a first embodiment of the present application;

FIG. 3 is a schematic illustration of the determination of the intersection point in an embodiment of the present application;

FIG. 4 is a schematic illustration according to a second embodiment of the present application;

FIG. 5 is a schematic illustration according to a third embodiment of the present application;

FIG. 6 is a schematic illustration according to a fourth embodiment of the present application;

fig. 7 is a schematic view of an electronic device for implementing the method for detecting an arteriovenous crossing compression characteristic according to the embodiment of the present application.

Detailed Description

The following description of the exemplary embodiments of the present application, taken in conjunction with the accompanying drawings, includes various details of the embodiments of the application for the understanding of the same, which are to be considered exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

It is to be understood that the embodiments described are only a few embodiments of the present application and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

It should be noted that the terminal according to the embodiment of the present application may include, but is not limited to, a mobile phone, a Personal Digital Assistant (PDA), a wireless handheld device, a Tablet Computer (Tablet Computer), a Personal Computer (PC), an MP3 player, an MP4 player, a wearable device (e.g., smart glasses, smart watches, smart bracelets, etc.), a smart home device, and other smart devices.

In addition, the term "and/or" herein is only one kind of association relationship describing an associated object, and means that there may be three kinds of relationships, for example, a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter related objects are in an "or" relationship.

In the prior art, an unsupervised Kmeans algorithm is used for arteriovenous vessel segmentation, a vessel topological graph is established for all vessels in a certain radius range of the center of a video disc, arteriovenous cross stress characteristic nodes are identified according to the vessel topological graph and arteriovenous vessel types, the quality difference of fundus images is large due to the influence of shooting equipment and shooting operation, meanwhile, the robustness of the arteriovenous vessel segmentation performed by the unsupervised Kmeans algorithm is poor due to the fact that the structure difference of the fundus of each human eye is large and fundus diseases possibly exist, the difference of arteriovenous vessel appearance is low due to the fact that the gray values of arteriovenous vessels in fundus images are close, the accuracy of arteriovenous vessel segmentation results is also reduced, and the accuracy of arteriovenous cross stress characteristic detection results is influenced.

The application aims at the problems and provides a method and a device for detecting arteriovenous crossing compression sign, electronic equipment and a readable storage medium, which can automatically identify normal arteriovenous crossing and arteriovenous crossing compression sign in an eyeground color image without depending on manual assistance, can position the position of the arteriovenous crossing compression sign, and can improve the accuracy of the detection result of the arteriovenous crossing compression sign.

Fig. 2 is a schematic diagram according to a first embodiment of the present application, as shown in fig. 2.

101. And carrying out blood vessel segmentation on the image to be detected to obtain a blood vessel segmentation result of whether each pixel in the image to be detected is a blood vessel.

The image to be detected in the embodiment of the present application may be a fundus image or a retina image, or an image of another part, which is not limited in the embodiment of the present application.

102. And determining a blood vessel intersection point according to the blood vessel segmentation result.

103. Acquiring a region of interest (ROI) image of the blood vessel intersection from the image to be detected.

104. And classifying the ROI image by using a classification model to obtain a classification result of whether the ROI image is subjected to artery and vein cross compression.

The classification model is obtained by training in advance based on a supervised training mode.

Optionally, the classification model in the embodiment of the present application may be implemented based on a deep learning neural network (e.g., a convolutional neural network), or may be implemented by using other classification models, for example, a gradient-boosting iterative decision tree (GDBT) model, a Support Vector Machine (SVM), a feed-forward neural network, a Long Short Term Memory (LSTM) model, and the like, and the present application does not limit the specific implementation of the classification model.

It should be noted that part or all of the execution subjects of 101 to 104 may be an application located at the local terminal, or may also be a functional unit such as a plug-in or Software Development Kit (SDK) set in the application located at the local terminal, or may also be a processing engine located in a network side server, which is not particularly limited in this embodiment.

It is to be understood that the application may be a native app (native app) installed on the terminal, or may also be a web page program (webApp) of a browser on the terminal, which is not limited in this embodiment.

In the embodiment, a blood vessel segmentation result of whether each pixel in an image to be detected is a blood vessel is obtained by performing blood vessel segmentation on the image to be detected, then, a blood vessel intersection point is determined according to the blood vessel segmentation result, an ROI (region of interest) image of the blood vessel intersection point is obtained from the image to be detected, and then, the ROI image is classified by using a classification model obtained by training based on a supervised training mode to obtain a classification result of whether the ROI image is a artery and vein cross compression sign, so that the detection of the artery and vein cross compression sign node in the image to be detected is realized. Because the artery and vein vessel segmentation is not needed, only whether the blood vessel is segmented or not is needed, the relative difficulty is small, the segmentation is easy, the segmentation robustness is good, the accuracy of the classification result of the artery and vein cross compression sign is improved, and the accuracy of the detection result of the artery and vein cross compression sign is improved.

Optionally, in a possible implementation manner of this embodiment, the classification model may be obtained by training in advance based on a supervised training manner as follows:

respectively inputting a plurality of first sample images marked with classification reference information into a neural network, and respectively outputting a first prediction probability value of whether each first sample image is an arteriovenous crossing compression sign or not through the neural network, wherein the classification reference information comprises a category 1 of the arteriovenous crossing compression sign or a category 0 of the arteriovenous crossing compression sign (namely normal arteriovenous crossing); and training the neural network based on the difference between the first prediction probability value and the classification reference information, namely adjusting the parameter values of the network parameters in the neural network. The operation may be an iterative execution process, and the training of the neural network is realized by iteratively executing the operation until a first preset training completion condition is met, so that the training of the neural network is completed.

In a specific implementation process, the first preset training completion condition may be set according to an actual requirement, and may be, for example: the training times of the neural network reach a preset number (for example, 100 ten thousand times), and/or the difference between the first prediction probability value and the classification reference information is smaller than a preset difference.

Optionally, in a possible implementation manner of this embodiment, in 101, a deep learning segmentation model may be adopted to perform feature extraction on the image to be detected, and perform blood vessel segmentation based on the features extracted from the image to be detected, so as to obtain a score map (Scores _ map) of whether each pixel in the image to be detected is a blood vessel, where the score map includes a score (i.e., a probability value) of whether each pixel in the image to be detected is a blood vessel; then, segmenting the score map based on a first preset threshold value to obtain a binary segmentation result (Segmented _ vessel _ mask) of whether each pixel in the image to be detected is a blood vessel or not, wherein the binary segmentation result can also be called a blood vessel mask; the blood vessel segmentation result comprises the binarization segmentation result.

For example, if the first preset threshold is 0.6, in the score map, if the score of a pixel is greater than 0.6, the category of the pixel is considered to be a blood vessel, and the score is modified to 1, otherwise, if the score of the pixel is not greater than 0.6, the category of the pixel is considered not to be a blood vessel, and the score is modified to 0, and the binary segmentation result is obtained.

The value of the first preset threshold in this embodiment may be set according to actual requirements and may be adjusted according to different situations.

The deep learning segmentation model may be any deep learning segmentation algorithm framework model, such as a U-shaped full convolution neural network (uet). The Unet comprises two parts, wherein the first part is a feature extraction Network, and can be realized by adopting the design mode of feature extraction layers of a Residual Neural Network (Resnet) Network, a Visual Geometry Group (VGG) Network and other networks, and has the advantages that the training of the Unet can be accelerated by utilizing a pre-trained mature model; the second part is an up-sampling network and can be realized by adopting a deconvolution processing mode. Since the network structure of the Unet is U-shaped, the Unet is formed.

Adopting Unet to carry out feature extraction on the image to be detected, and carrying out blood vessel segmentation based on the features extracted from the image to be detected: firstly, inputting an image to be detected to Unet; extracting the features of the image to be detected through a feature extraction network (Resnet-50 network), and forward propagating the extracted features to an upsampling network and a convolution layer shared by the feature extraction network and the upsampling network in the Unet, so that on one hand, a higher-dimensional feature map is generated, and on the other hand, the part of features are fused into the upsampling layer, so that the information finally used for pixel classification is richer and important information in the image to be detected can be reserved; and classifying whether each pixel in the image to be detected is blood vessel or not by the convolution layer in the Unet based on the feature extracted by the feature extraction network and the fusion feature of the feature extraction network and the up-sampling network, and outputting the score of whether each pixel is blood vessel or not, so as to obtain the score map of whether each pixel in the image to be detected is blood vessel or not.

In a specific implementation, the deep learning segmentation model may be trained in advance, the second sample image is input into the deep learning segmentation model, and a second predicted probability value of whether each pixel in the second sample image is a blood vessel is output through the deep learning segmentation model, wherein each pixel in the second sample image is marked with labeling information of whether each pixel is a blood vessel; and obtaining a binary weighted cross entropy loss function value used for expressing the difference between the second prediction probability value and the labeling information of each pixel and a Dice loss function value used for expressing the contact degree between the blood vessel region in the second sample image determined based on the second prediction probability value and the blood vessel region determined based on the labeling information, and performing optimization training on the depth learning segmentation model by combining the binary weighted cross entropy loss function value and the Dice loss function value, namely adjusting the parameter value of the network parameter in the depth learning segmentation model until a second preset training completion condition is met, and completing the training on the depth learning segmentation model.

In a specific implementation process, the second preset training completion condition may be set according to an actual requirement, and may be, for example: the training times of the deep learning segmentation model reach preset times (for example, 150 ten thousand times), and/or the sum of the binary weighted cross entropy loss function value and the Dice loss function value is smaller than a preset value.

In the embodiment, a deep learning segmentation model is adopted to perform feature extraction and blood vessel segmentation on an image to be detected, and the score map is segmented based on the first preset threshold value to obtain a binary segmentation result of whether each pixel in the image to be detected is a blood vessel, so that the blood vessel segmentation result is more visual, and the efficiency and the accuracy of a subsequent blood vessel intersection determination result are improved.

Optionally, in a possible implementation manner of this embodiment, in 102, a preset expansion algorithm may be used to expand a pixel, which is a blood vessel, in the image to be detected based on the blood vessel segmentation result, that is, the size of the pixel, which belongs to the blood vessel, in the image to be detected is enlarged and thickened to obtain an expanded blood vessel segmentation result, so as to avoid the problems of incomplete blood vessel segmentation or fracture, discontinuity and the like of the segmented blood vessel caused by image quality factors, and a specific expansion scale may be set according to an actual situation, so as to avoid blood vessel fracture and confusion of two different blood vessels; then, extracting a blood vessel skeleton from the expanded blood vessel segmentation result; further, the extracted blood vessel skeleton is subjected to blood vessel intersection recognition.

In this embodiment, the expansion of the pixel of the blood vessel in the image to be detected can avoid the problems of the blood vessel incomplete segmentation or the breakage, discontinuity and the like of the segmented blood vessel caused by image quality factors, and then the blood vessel skeleton is extracted and the blood vessel intersection point is identified according to the blood vessel skeleton, so that the blood vessel intersection point identification efficiency can be improved, the problems of the breakage, discontinuity and the like of the segmented blood vessel caused by the blood vessel incomplete segmentation or the image quality factors can be avoided, and the number of the intersection points is unnecessarily increased, thereby affecting the intersection point identification efficiency.

In an alternative example, a refinement algorithm may be used to extract the vessel skeleton from the dilated vessel segmentation result. The key of the refinement algorithm is to maintain topology invariance, namely the skeleton of the blood vessel obtained by refinement is consistent with the topology structure of the expanded blood vessel segmentation result. Therefore, refinement and extraction of the vascular skeleton keep the topological structure unchanged. When the thinning algorithm is adopted to extract the blood vessel skeleton from the expanded blood vessel segmentation result, points at the edge of the blood vessel in the blood vessel segmentation result can be deleted, so that the Euler number before and after each point is deleted is kept unchanged, and the number of connecting components, annular holes and holes in the blood vessel segmentation result is kept unchanged. In the refinement algorithm, a point on an edge that does not change the image topology after deletion is referred to as a simple point. Before and after the point is deleted, the Euler number is kept unchanged at the point adjacent to the Euler number, and the number of the components is kept unchanged. The refinement algorithm can be iteratively performed to search and delete simple points from the vessel boundary until there are no movable points, resulting in the vessel skeleton.

In an optional example, when performing blood vessel intersection point identification on the extracted blood vessel skeleton, each pair of bifurcation structures in the blood vessel skeleton can be acquired respectively; then, based on the distance between two bifurcation points in each pair of bifurcation structures and the included angle in each pair of bifurcation structures, respectively, the intersection point in each pair of bifurcation structures is determined as the blood vessel intersection point.

In this embodiment, each pair of bifurcation structures in the blood vessel framework may be obtained first, and based on a distance between two bifurcation points in the bifurcation structures and an included angle in the bifurcation structures, a crossing point in the bifurcation structure is determined as the blood vessel crossing point, thereby realizing accurate determination of the blood vessel crossing point based on a geometric manner.

In a specific implementation, determining the intersection point in each pair of the bifurcation structures based on the distance between two bifurcation points in each pair of the bifurcation structures and the included angle in each pair of the bifurcation structures respectively can be implemented by:

identifying whether the distance between the two bifurcation points is less than a second preset threshold (e.g., 0.5 mm);

identifying whether an included angle between two bifurcation lines in the bifurcation structure is smaller than a third preset threshold (for example, 70 degrees);

Identifying whether an included angle between any of the bifurcation lines and a connecting line between the bifurcation points is larger than a fourth preset threshold (for example, 130 degrees);

if the distance between the two bifurcation points is smaller than a second preset threshold (referred to as a first condition), and/or the included angle between the two bifurcation lines in the bifurcation structure is smaller than a third preset threshold, and the included angle between any bifurcation line and a connecting line between the bifurcation points is larger than a fourth preset threshold (referred to as a second condition), that is, as long as any one or both of the first condition and the second condition are met, determining the middle point between the two bifurcation points as a crossing point in each pair of bifurcation structures. Otherwise, if any one of the first condition and the second condition is not satisfied, determining that no intersection exists in the current pair of the branched structures.

The values of the second preset threshold, the third preset threshold and the fourth preset threshold in this embodiment may be set according to actual requirements and may be adjusted according to different situations.

In this embodiment, the intersection point in the bifurcation structure may be determined based on the distance between two bifurcation points, the included angle between two bifurcation lines in the bifurcation structure, and the included angle between a connecting line between a bifurcation line and the bifurcation point, so that the accuracy of intersection point identification is improved.

Fig. 3 is a schematic diagram illustrating the determination of the intersection point in the embodiment of the present application. As shown in fig. 3, a pair of bifurcation structures is provided, which includes a bifurcation point 1 and a bifurcation point 2, a length of a connecting line between the bifurcation point 1 and the bifurcation point 2 is a distance between the two bifurcation points, an included angle between the two bifurcation lines includes α 1 and α 2, and an included angle between the bifurcation line and the connecting line between the bifurcation points includes β 1 and β 2, and as long as one of α 1 and α 2 is smaller than a third preset threshold, the included angle between the two bifurcation lines in the bifurcation structure can be considered to be smaller than the third preset threshold. Similarly, as long as one of β 1 and β 2 is greater than a fourth preset threshold, the angle between the connection line between any of the bifurcation lines and the bifurcation point is considered to be greater than the fourth preset threshold.

Optionally, in a possible implementation manner of this embodiment, in 103, for each blood vessel intersection, an ROI image with a first preset size (for example, 64 × 64) may be extracted from the image to be detected by taking the position of the blood vessel intersection as a central point, and the ROI image may be input into a classification model as a candidate image for arteriovenous crossing compression feature identification.

In the embodiment, the position of the blood vessel intersection point is taken as a central point, the ROI image with the first preset size is intercepted from the image to be detected and is taken as a candidate image for arteriovenous cross compression sign recognition, and compared with the process of performing arteriovenous cross compression sign recognition on the whole image to be detected, the blood vessel intersection point in the candidate image is clearer, so that the accuracy of an arteriovenous cross compression sign recognition result can be improved.

Optionally, in a possible implementation manner of this embodiment, in 104, the classification model may be utilized to perform feature extraction on an ROI image, and perform classification based on features extracted from the ROI image, so as to obtain a classification score of whether the ROI image is a arteriovenous cross-compression feature; then, classifying the classification scores based on a fifth preset threshold value to obtain a classification result of whether the ROI image is the artery-vein cross-compression sign or not.

In a specific implementation, an ROI image may be input into a classification model, a plurality of convolution layers in the classification model sequentially extract features of the ROI image to obtain a full connection layer in a high-level feature input classification model, the high-level features are converted into one-dimensional feature vectors through the full connection layer, then the classification layer in the classification model classifies based on the one-dimensional feature vectors, and outputs a classification score of whether the ROI image is a arteriovenous cross-compression feature, and then, the classification score is classified based on a fifth preset threshold, assuming that the fifth preset threshold is 0.7, and if the classification score is greater than 0.7, the ROI image is considered as the arteriovenous cross-compression feature; otherwise, if the classification score is not greater than 0.7, the ROI image is not considered to be an arteriovenous crossing oppression sign (namely normal arteriovenous crossing).

The value of the fifth preset threshold in this embodiment may be set according to actual requirements and may be adjusted according to different situations, and the value of the fifth preset threshold in this embodiment is not limited.

In this embodiment, a supervised training mode is used for training to obtain a classification model, the ROI image is subjected to feature extraction, classification is performed based on the extracted features, and after a classification score is obtained, a classification result of whether the ROI image is a static and arterial cross-compression sign or not is obtained based on the classification score.

Fig. 4 is a schematic diagram of a second embodiment according to the present application, as shown in fig. 4. Before the embodiment shown in fig. 2, the method may further include:

201. the original image is scaled to a second predetermined size (e.g. 1024 x 1024) resulting in a scaled image (Ori _ img).

202. And enhancing the blood vessel region in the zoomed image by using a preset blood vessel enhancement method to obtain an enhanced image (enhanced _ img).

Optionally, in a possible implementation manner of this embodiment, a blood vessel enhancement method based on a Hessian (Hessian) matrix may be used to enhance (color-highlight) a blood vessel region in the zoomed image, and the application does not limit a specific blood vessel enhancement method.

203. And performing thresholding processing on the enhanced image based on a sixth preset threshold value to obtain an enhanced blood vessel mask (enhanced _ mask).

Optionally, in a possible implementation manner of this embodiment, a pixel value of the enhanced image, which is greater than a sixth preset threshold, may be set to 1, and a pixel value of the enhanced image, which is not greater than the sixth preset threshold, may be set to 0, where a pixel with a pixel value of 1 forms the enhanced blood vessel mask.

The value of the sixth preset threshold may be set according to actual requirements, for example, may be 128, and may be adjusted according to different situations.

204. And mixing the green channel information in the enhanced blood vessel mask into a green channel in the zoomed image to obtain the image to be detected (Fusion _ ori _ img).

Specifically, the mixing process can be expressed as: fusion _ Ori _ img ═ Merge (Ori _ img _ red, enhanced _ mask _ enhanced _ img _ green + Ori _ img _ green, Ori _ img _ blue).

Where Merge () represents a blend operation, Ori _ img _ red represents image information of a red channel in the scaled image; enhanced _ mask _ enhanced _ img _ green + Ori _ img _ green represents the image information of the green channel obtained after the green channel information in the enhanced vascular mask is mixed into the green channel in the zoomed image; ori _ img _ blue represents the image information of the blue channel in the scaled image.

In the embodiment, the original image is subjected to blood vessel region enhancement, and the obtained green channel information in the enhanced blood vessel mask information is fused into the zoomed image of the original image for subsequent blood vessel segmentation, so that the integrity and the accuracy of the blood vessel segmentation can be improved, and the accuracy of the recognition result of the arteriovenous cross-constriction is improved. In addition, in the blood vessel image, because the image information of the green channel has less noise relative to the image information of the red channel and the blue channel, the blood vessel information enhancement is carried out on the zoomed image based on the green channel information in the enhanced blood vessel mask, which is beneficial to improving the accuracy of the blood vessel segmentation result.

It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present application is not limited by the order of acts described, as some steps may occur in other orders or concurrently depending on the application. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required in this application.

In the foregoing embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

Fig. 5 is a schematic view of a third embodiment according to the present application, as shown in fig. 5. The device 300 for detecting arteriovenous crossing compression features of the present embodiment may include a segmentation unit 301, a determination unit 302, an acquisition unit 303, and a classification model 304. The segmentation unit 301 is configured to perform blood vessel segmentation on an image to be detected to obtain a blood vessel segmentation result of whether each pixel in the image to be detected is a blood vessel; a determining unit 302 for determining a vessel intersection point according to the vessel segmentation result; an obtaining unit 303, configured to obtain a region of interest ROI image of the blood vessel intersection from the image to be detected; the classification model 304 is used for classifying the ROI image to obtain a classification result of whether the ROI image is subjected to artery and vein cross compression; wherein, the classification model 304 is obtained by training in advance based on a supervised training mode.

It should be noted that, part or all of the execution main body of the apparatus for detecting arteriovenous crossing compression in this embodiment may be an application located at the local terminal, or may also be a functional unit such as a plug-in or Software Development Kit (SDK) provided in the application located at the local terminal, or may also be a processing engine located in a network side server, which is not particularly limited in this embodiment.

Optionally, in a possible implementation manner of this embodiment, the dividing unit 301 is specifically configured to: extracting features of the image to be detected by adopting a deep learning segmentation model, and performing blood vessel segmentation on the basis of the features extracted from the image to be detected to obtain a score map of whether each pixel in the image to be detected is a blood vessel; segmenting the score map based on a first preset threshold value to obtain a binary segmentation result of whether each pixel in the image to be detected is a blood vessel; the blood vessel segmentation result comprises the binarization segmentation result.

Optionally, in a possible implementation manner of this embodiment, the determining unit 302 is specifically configured to: expanding the pixels of the blood vessels in the image to be detected based on a preset expansion algorithm according to the blood vessel segmentation result to obtain an expanded blood vessel segmentation result; extracting a blood vessel skeleton from the expanded blood vessel segmentation result; and carrying out blood vessel intersection identification on the extracted blood vessel skeleton.

In an optional example, when the determining unit 302 performs the blood vessel intersection point identification on the extracted blood vessel skeleton, it is specifically configured to: respectively acquiring each pair of bifurcation structures in the blood vessel skeleton; determining the intersection point in each pair of the bifurcation structures as the vessel intersection point based on the distance between two bifurcation points in each pair of the bifurcation structures and the included angle in each pair of the bifurcation structures, respectively.

In a specific implementation, the determining unit 302 is specifically configured to, when determining the intersection point in each pair of the bifurcation structures based on a distance between two bifurcation points in each pair of the bifurcation structures and an included angle in each pair of the bifurcation structures, respectively: identifying whether the distance between the two bifurcation points is smaller than a second preset threshold value; identifying whether an included angle between two bifurcation lines in the bifurcation structure is smaller than a third preset threshold value or not; identifying whether an included angle between any bifurcation line and a connecting line between bifurcation points is larger than a fourth preset threshold value or not; and if the distance between the two bifurcation points is smaller than a second preset threshold value, and/or the included angle between the two bifurcation lines in the bifurcation structure is smaller than a third preset threshold value, and the included angle between any bifurcation line and a connecting line between the bifurcation points is larger than a fourth preset threshold value, determining that the middle point between the two bifurcation points is a cross point in each pair of bifurcation structures.

Optionally, in a possible implementation manner of this embodiment, the obtaining unit 303 is specifically configured to: and respectively intercepting an ROI image with a first preset size from the image to be detected by taking the position of the blood vessel intersection point as a central point for each blood vessel intersection point.

Optionally, in a possible implementation manner of this embodiment, the classification model 304 is specifically configured to: and extracting the characteristics of the ROI image, and classifying the ROI image based on the characteristics extracted from the ROI image to obtain a classification score of whether the ROI image is subjected to artery and vein cross-compression.

Fig. 6 is a schematic view according to a fourth embodiment of the present application, and as shown in fig. 6, on the basis of the embodiment shown in fig. 5, the apparatus 300 for detecting arteriovenous crossing compression characteristics of this embodiment may further include: a dividing unit 401, configured to perform category division on the classification score based on a fifth preset threshold, so as to obtain a classification result of whether the ROI image is a sound-vein crossing compression sign.

Optionally, referring to fig. 6 again, the device 300 for detecting an arteriovenous crossing compression sign according to the above embodiment may further include: a preprocessing unit 402 for: scaling the original image to a second preset size to obtain a scaled image; enhancing the blood vessel region in the zoomed image by using a preset blood vessel enhancement method to obtain an enhanced image; thresholding the enhanced image based on a sixth preset threshold value to obtain an enhanced blood vessel mask; and mixing the green channel information in the enhanced blood vessel mask into the green channel in the zoomed image to obtain the image to be detected.

It should be noted that the method in the embodiment corresponding to fig. 2 to fig. 4 can be implemented by the detection device for arteriovenous crossing compression provided in the embodiment of fig. 5 to fig. 6. For detailed description, reference may be made to relevant contents in the embodiments corresponding to fig. 2 to fig. 4, which are not described herein again.

The present application also provides an electronic device and a non-transitory computer readable storage medium having computer instructions stored thereon, according to embodiments of the present application.

Fig. 7 is a schematic view of an electronic device for implementing the method for detecting an arteriovenous crossing compression characteristic according to the embodiment of the present application. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application that are described and/or claimed herein.

As shown in fig. 7, the electronic apparatus includes: one or more processors 501, memory 502, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions for execution within the electronic device, including instructions stored in or on the memory to display graphical information of a GUI (graphical user interface) on an external input/output apparatus, such as a display device coupled to the interface. In other embodiments, multiple processors and/or multiple buses may be used, along with multiple memories and multiple memories, as desired. Also, multiple electronic devices may be connected, with each device providing portions of the necessary operations (e.g., as a server array, a group of blade servers, or a multi-processor system). Fig. 7 illustrates an example of a processor 501.

Memory 502 is a non-transitory computer readable storage medium as provided herein. The memory stores instructions executable by the at least one processor to cause the at least one processor to perform the method for detecting arteriovenous crossing compression provided herein. A non-transitory computer readable storage medium of the present application stores computer instructions for causing a computer to perform the method of detecting an arteriovenous cross compression signature provided herein.

The memory 502, which is a non-transitory computer readable storage medium, may be used to store non-transitory software programs, non-transitory computer executable programs, and units, such as program instructions/units corresponding to the detection method of arteriovenous cross-compression feature in the embodiment of the present application (for example, the segmentation unit 301, the determination unit 302, the acquisition unit 303, and the classification model 304 shown in fig. 5). The processor 501 executes various functional applications and data processing of the server by running the non-transitory software programs, instructions and units stored in the memory 502, that is, implements the method for detecting arteriovenous crossing compression in the above method embodiment.

The memory 502 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to use of an electronic device that implements the detection method of arteriovenous crossing compression characteristics provided by the embodiment of the present application, and the like. Further, the memory 502 may include high speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory 502 may optionally include a memory remotely located from the processor 501, and these remote memories may be connected via a network to an electronic device implementing the method of detecting arteriovenous cross-compression signatures provided by embodiments of the present application. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The electronic device of the method for detecting the arteriovenous crossing compression sign can further comprise: an input device 503 and an output device 504. The processor 501, the memory 502, the input device 503 and the output device 504 may be connected by a bus or other means, and fig. 7 illustrates the connection by a bus as an example.

The input device 503 may receive input numeric or character information and generate key signal inputs related to user settings and function control of an electronic device implementing the method for detecting arteriovenous crossing compression provided by the embodiments of the present application, such as an input device like a touch screen, a keypad, a mouse, a track pad, a touch pad, a pointing stick, one or more mouse buttons, a track ball, a joystick, etc. The output devices 504 may include a display device, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibrating motors), among others. The display device may include, but is not limited to, an LCD (liquid crystal display), an LED (light emitting diode) display, and a plasma display. In some implementations, the display device can be a touch screen.

Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

These computer programs (also known as programs, software applications, or code) include machine instructions for a programmable processor, and may be implemented using high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., magnetic discs, optical disks, memory, PLDs (programmable logic devices)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, verbal, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: LAN (local area network), WAN (wide area network), internet, and blockchain networks.

The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

According to the technical scheme of this application embodiment, owing to do not need to carry out arteriovenous blood vessel and cut apart, only need cut apart whether blood vessel, relative degree of difficulty is less, easily cuts apart, and cuts apart the robustness better, helps improving the accuracy of the classification result of arteriovenous cross compression sign to promote the accuracy of arteriovenous cross compression sign testing result.

It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present application may be executed in parallel, sequentially, or in different orders, and the present invention is not limited thereto as long as the desired results of the technical solutions disclosed in the present application can be achieved.

The above-described embodiments should not be construed as limiting the scope of the present application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for detecting an arteriovenous crossing compression sign comprises the following steps:

2. The method according to claim 1, wherein the performing the blood vessel segmentation on the image to be detected to obtain a blood vessel segmentation result of whether each pixel in the image to be detected is a blood vessel comprises:

extracting features of the image to be detected by adopting a deep learning segmentation model, and performing blood vessel segmentation on the basis of the features extracted from the image to be detected to obtain a score map of whether each pixel in the image to be detected is a blood vessel;

segmenting the score map based on a first preset threshold value to obtain a binary segmentation result of whether each pixel in the image to be detected is a blood vessel; the blood vessel segmentation result comprises the binarization segmentation result.

3. The method of claim 1, wherein the determining a vessel intersection point from the vessel segmentation result comprises:

expanding the pixels of the blood vessels in the image to be detected based on a preset expansion algorithm according to the blood vessel segmentation result to obtain an expanded blood vessel segmentation result;

extracting a blood vessel skeleton from the expanded blood vessel segmentation result;

and carrying out blood vessel intersection identification on the extracted blood vessel skeleton.

4. The method of claim 3, wherein the performing vessel intersection identification on the extracted vessel skeleton comprises:

respectively acquiring each pair of bifurcation structures in the blood vessel skeleton;

determining the intersection point in each pair of the bifurcation structures as the vessel intersection point based on the distance between two bifurcation points in each pair of the bifurcation structures and the included angle in each pair of the bifurcation structures, respectively.

5. The method of claim 4, wherein determining the intersection point in each pair of the furcation structures based on a distance between two furcation points in each pair of the furcation structures and an included angle in each pair of the furcation structures, respectively, comprises:

identifying whether the distance between the two bifurcation points is smaller than a second preset threshold value;

Identifying whether an included angle between two bifurcation lines in the bifurcation structure is smaller than a third preset threshold value or not;

identifying whether an included angle between any bifurcation line and a connecting line between bifurcation points is larger than a fourth preset threshold value or not;

and if the distance between the two bifurcation points is smaller than a second preset threshold value, and/or the included angle between the two bifurcation lines in the bifurcation structure is smaller than a third preset threshold value, and the included angle between any bifurcation line and a connecting line between the bifurcation points is larger than a fourth preset threshold value, determining that the middle point between the two bifurcation points is a cross point in each pair of bifurcation structures.

6. The method according to claim 1, wherein said acquiring a region of interest, ROI, image of said vessel intersection from said image to be detected comprises:

and respectively intercepting an ROI image with a first preset size from the image to be detected by taking the position of the blood vessel intersection point as a central point for each blood vessel intersection point.

7. The method of claim 1, wherein the classifying the ROI image by using a classification model to obtain a classification result of whether the ROI image is subjected to arteriovenous cross-compression comprises the following steps:

Extracting the characteristics of the ROI image by using the classification model, and classifying the ROI image based on the characteristics extracted from the ROI image to obtain a classification score of whether the ROI image is subjected to arteriovenous cross compression;

and classifying the classification scores based on a fifth preset threshold value to obtain a classification result of whether the ROI image is subjected to artery and vein cross compression.

8. The method according to any one of claims 1-7, wherein before the vessel segmentation of the image to be detected, further comprising:

scaling the original image to a second preset size to obtain a scaled image;

enhancing the blood vessel region in the zoomed image by using a preset blood vessel enhancement method to obtain an enhanced image;

thresholding the enhanced image based on a sixth preset threshold value to obtain an enhanced blood vessel mask;

and mixing the green channel information in the enhanced blood vessel mask into the green channel in the zoomed image to obtain the image to be detected.

9. A detection device for arteriovenous crossing compression sign comprises:

10. The apparatus according to claim 9, wherein the segmentation unit is specifically configured to:

11. The apparatus according to claim 10, wherein the determining unit is specifically configured to:

12. The apparatus according to claim 11, wherein the determining unit is configured to, when performing the vessel intersection point identification on the extracted blood vessel skeleton:

13. The apparatus according to claim 12, wherein the determining unit is configured to determine the intersection point in each pair of the bifurcation structures based on a distance between two bifurcation points in each pair of the bifurcation structures and an angle in each pair of the bifurcation structures, in particular:

14. The apparatus according to claim 9, wherein the obtaining unit is specifically configured to:

15. The apparatus of claim 9, wherein the classification model is specifically configured to:

extracting the characteristics of the ROI image, and classifying the ROI image based on the characteristics extracted from the ROI image to obtain a classification score of whether the ROI image is subjected to artery and vein cross compression;

the device further comprises:

and the classification unit is used for classifying the classification scores based on a fifth preset threshold value to obtain a classification result of whether the ROI image is subjected to artery and vein cross-compression.

16. The apparatus of any of claims 9-15, further comprising:

The preprocessing unit is used for zooming the original image to a second preset size to obtain a zoomed image; enhancing the blood vessel region in the zoomed image by using a preset blood vessel enhancement method to obtain an enhanced image; thresholding the enhanced image based on a sixth preset threshold value to obtain an enhanced blood vessel mask; and mixing the green channel information in the enhanced blood vessel mask into the green channel in the zoomed image to obtain the image to be detected.

17. An electronic device, comprising:

at least one processor; and

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.

18. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-8.