CN111340800B

CN111340800B - Image detection method, computer device, and storage medium

Info

Publication number: CN111340800B
Application number: CN202010191372.2A
Authority: CN
Inventors: 刘曦; 董昢; 薛忠
Original assignee: Lianying Intelligent Medical Technology Beijing Co ltd
Current assignee: Lianying Intelligent Medical Technology Beijing Co ltd
Priority date: 2020-03-18
Filing date: 2020-03-18
Publication date: 2024-02-27
Anticipated expiration: 2040-03-18
Also published as: CN111340800A

Abstract

The application relates to an image detection method, a computer device and a storage medium. The method comprises the following steps: acquiring a medical image to be detected; the medical image to be detected comprises a region of interest; inputting the medical image to be detected into a key point detection model to obtain a key point set corresponding to the region of interest; the key point detection model is obtained by training based on a sample medical image and a gold standard image corresponding to the sample medical image, wherein the gold standard image comprises a key point mark corresponding to the sample medical image; determining related structural parameters of the region of interest based on the key point set corresponding to the region of interest; the relevant structural parameters are used to characterize the morphology of the region of interest. By adopting the method, the manpower can be saved.

Description

Image detection method, computer device, and storage medium

Technical Field

The present disclosure relates to the field of image processing technologies, and in particular, to an image detection method, a computer device, and a storage medium.

Background

The spine is an important support structure for the human body, and includes 7 cervical vertebrae, 12 thoracic vertebrae, and 5 lumbar vertebrae. In modern society, physical activities of mental workers are reduced, office time required for sitting for a long time is greatly increased, so that people with problems of the spine are more and more, and treatment means for spinal problems of different types and different degrees are often different, so that some parameters of the spine need to be measured to determine which type or degree of the spine belongs to based on the parameters.

In the related art, after a subject captures a medical image, a doctor usually marks a spine related point on the medical image manually, and then measures some parameters of the spine according to the marked spine related point.

However, the above-described technique has a problem of consuming labor.

Disclosure of Invention

In view of the foregoing, it is desirable to provide an image detection method, apparatus, computer device, and storage medium that can save labor.

An image detection method, the method comprising:

acquiring a medical image to be detected; the medical image to be detected comprises a region of interest;

inputting the medical image to be detected into a key point detection model to obtain a key point set corresponding to the region of interest; the key point detection model is obtained by training based on a sample medical image and a gold standard image corresponding to the sample medical image, wherein the gold standard image comprises a key point mark corresponding to the sample medical image;

determining relevant structural parameters of the region of interest based on the key point set corresponding to the region of interest; the relevant structural parameters are used to characterize the morphology of the region of interest.

In one embodiment, the relevant structural parameters of the region of interest include curvature of the region of interest, and the set of key points includes location information of each key point; the method further comprises the following steps:

Dividing each key point in the key point set on the region of interest based on the position information of each key point in the key point set to obtain at least one key point subset on the region of interest;

determining a feature value corresponding to each key point subset based on at least one key point subset;

and determining the category of the curvature of the region of interest based on the feature value corresponding to each key point subset.

In one embodiment, determining the class of the curvature of the region of interest based on the feature value corresponding to each of the key point subsets includes:

inputting the characteristic value corresponding to each key point subset into a classification model, and determining the class of the curvature of the region of interest; the classification model is obtained by training based on a sample medical image set, wherein the sample medical image set comprises feature values corresponding to each key point subset of the region of interest in the training medical image and labeling categories of curvature of the region of interest in the training medical image.

In one embodiment, the determining, based on at least one of the keypoint subsets, a feature value corresponding to each of the keypoint subsets includes:

fitting each key point in each key point subset to determine a circle corresponding to each key point subset;

Determining a target radius of a circle corresponding to each key point subset based on the circle corresponding to each key point subset and the position information of each key point in each key point subset; the target radius is a radius with positive and negative signs;

and obtaining curvature values corresponding to the key point subsets according to the target radius of the circle corresponding to the key point subsets, and determining the curvature values corresponding to the key point subsets as the characteristic values corresponding to the key point subsets.

In one embodiment, the determining the target radius of the circle corresponding to each of the key point subsets based on the circle corresponding to each of the key point subsets and the position information of each of the key points in each of the key point subsets includes:

obtaining the circle center position of the circle corresponding to each key point subset according to the circle corresponding to each key point subset;

based on the circle center position of the circle corresponding to each key point subset and the position information of each key point in each key point subset, obtaining a relative position result of each key point subset and the circle center position of the corresponding circle;

if the circle center position of the circle corresponding to one key point subset in the relative position result is at the left side of the key point subset, determining the radius of the circle corresponding to the one key point subset as a target radius;

And if the circle center position of the circle corresponding to the other key point subset in the relative position result is at the right side of the key point subset, setting the radius of the circle corresponding to the other key point subset as a negative radius, and determining the negative radius as a target radius.

In one embodiment, the determining the relevant structural parameters of the region of interest based on the set of key points corresponding to the region of interest includes:

calculating the distance between the points in the key point set based on the position information of each key point in the key point set corresponding to the region of interest, and obtaining the related structural parameters of the region of interest;

and/or calculating the distance between lines formed by the key points in the key point set based on the position information of each key point in the key point set corresponding to the region of interest to obtain the related structural parameters of the region of interest;

and/or calculating the distance between the line formed by each key point in the key point set and the point based on the position information of each key point in the key point set corresponding to the region of interest, so as to obtain the related structural parameters of the region of interest;

and/or calculating the area of the surface formed by each key point in the key point set based on the position information of each key point in the key point set corresponding to the region of interest, and obtaining the related structural parameters of the region of interest.

In one embodiment, the inputting the medical image to be detected into the keypoint detection model to obtain the set of keypoints corresponding to the region of interest includes:

inputting the medical image to be detected into a key point detection model to obtain a probability map of at least one key point corresponding to the region of interest in the medical image to be detected;

determining at least one candidate point corresponding to each key point of the region of interest based on the probability map of the at least one key point;

processing at least one candidate point corresponding to each key point of the region of interest by adopting a dynamic programming algorithm to obtain a target point corresponding to each key point of the region of interest;

and obtaining a key point set corresponding to the region of interest based on one target point corresponding to each key point of the region of interest.

In one embodiment, the determining at least one candidate point corresponding to each key point of the region of interest based on the probability map of at least one key point includes:

performing binarization processing on the probability map of each key point according to a preset probability threshold value to obtain a binarization mask image corresponding to the probability map of each key point;

marking the connected domains in each binarized mask image, and determining candidate connected domains with areas larger than a preset area threshold according to the marked connected domains;

And calculating a weighted center and an average probability value of probability values in a probability map of the key points corresponding to each candidate connected domain, and obtaining at least one candidate point corresponding to each key point of the region of interest.

An image detection apparatus, the apparatus comprising:

the acquisition module is used for acquiring the medical image to be detected; the medical image to be detected comprises a region of interest;

the detection module is used for inputting the medical image to be detected into the key point detection model to obtain a key point set corresponding to the region of interest; the key point detection model is obtained by training based on a sample medical image and a gold standard image corresponding to the sample medical image, wherein the gold standard image comprises a key point mark corresponding to the sample medical image;

the determining module is used for determining relevant structural parameters of the region of interest based on the key point set corresponding to the region of interest; the relevant structural parameters are used to characterize the morphology of the region of interest.

A computer device comprising a memory storing a computer program and a processor which when executing the computer program performs the steps of:

A readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of:

According to the image detection method, the image detection device, the computer equipment and the storage medium, the key point detection is carried out on the medical image to be detected by adopting the key point detection model, the key point set corresponding to the region of interest in the medical image to be detected is obtained, the related structural parameters of the region of interest are determined based on the key point set corresponding to the region of interest, and the related structural parameters can represent the shape of the region of interest. In the method, the key point detection model can be adopted to detect the key points of the medical image of the testee, and a doctor is not required to manually mark the key points according to experience, so that the method can reduce the experience requirement of the doctor, and meanwhile, the doctor cannot consume too much energy and time, so that the labor can be saved; in addition, since the key point detection model is used for detecting the key points, the problem of error detection of the key points caused by fatigue of doctors is avoided, the key points detected by the method are accurate, so that when the accurate key points are used for determining the related structural parameters of the region of interest, the determined related structural parameters are accurate, and when the accurate related structural parameters are used for determining the subsequent treatment schemes, the determined treatment schemes are accurate.

Drawings

FIG. 1 is an internal block diagram of a computer device in one embodiment;

FIG. 2a is a flow chart of an image detection method according to an embodiment;

FIG. 2b is a schematic diagram of the location of identified keypoints in one embodiment;

FIG. 3a is a flowchart of an image detection method according to another embodiment;

FIG. 3b is a schematic diagram of a key point set partition in another embodiment;

FIG. 3c is a schematic view of the class of vertebral curvature in another embodiment;

FIG. 4 is a flowchart of an image detection method according to another embodiment;

FIG. 5 is a flowchart of an image detection method according to another embodiment;

fig. 6 is a block diagram showing the structure of an image detection apparatus in one embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.

The spine is an important support structure for the human body, and includes 7 cervical vertebrae, 12 thoracic vertebrae, and 5 lumbar vertebrae. In modern society, mental workers have reduced physical activities and require greatly increased office time for sedentary sitting, so that the spine problem has become a serious public health problem, and the life quality of people is greatly reduced. The treatment principles of different types and degrees of spinal diseases are quite different, prognosis after treatment is different, and accurate and consistent accurate measurement based on spinal images is very important for image diagnosis, operation planning and postoperative rehabilitation design. Currently, doctors usually use a method of manually marking the spine points to mark and measure the spine in clinical work, on one hand, the method requires that the doctors have enough experience, because the measurement accuracy of the anatomical parameters directly influences the selection of a treatment scheme; on the other hand, the manual measurement method generally needs to consume a great deal of effort of doctors, and is easy to cause fatigue and weakness of the doctors, and the working efficiency and quality are affected. In addition, for the detection of key points of the anatomical structure of the spine, besides manual calibration, many of the prior art are based on traditional algorithms and deep learning methods, however, due to the fact that the distance between adjacent vertebrae is small and the features are highly similar, the application of the traditional key point detection method in the medical image of the spine is greatly limited due to the fact that pathological conditions of patients such as scoliosis or metal implantation brackets, steel nails and the like are caused. At present, spine key point detection based on convolutional neural network is mainly used for positioning vertebral bodies and is used for assisting detection and positioning of other organs adjacent to the vertebral bodies, such as intervertebral discs, ribs and the like, but cannot be used for spine key point detection. The accurate detection of key points in the spine anatomical structure is an important precondition for accurately quantifying spine anatomical structure parameters and evaluating spine morphological changes, so that the key point detection has important clinical significance and scientific research significance. For this reason, the present application provides an image detection method, apparatus, computer device, and storage medium, so as to solve the above-mentioned technical problems.

The image detection method provided by the application can be applied to computer equipment, wherein the computer equipment can be a terminal or a server, and the computer equipment can be in wired or wireless communication with medical scanning equipment. Taking a computer device as an example, an internal structure diagram of the computer device may be shown in fig. 1. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless mode can be realized through WIFI, an operator network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement an image detection method. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, can also be keys, a track ball or a touch pad arranged on the shell of the computer equipment, and can also be an external keyboard, a touch pad or a mouse and the like.

It will be appreciated by those skilled in the art that the structure shown in fig. 1 is merely a block diagram of some of the structures associated with the present application and is not limiting of the computer device to which the present application may be applied, and that a particular computer device may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.

The execution subject of the embodiments of the present application may be a computer device or an image detection device, and the following description will refer to the computer device as the execution subject.

In one embodiment, an image detection method is provided, and this embodiment relates to a specific process of detecting a key point of a region of interest in a medical image to be detected, and determining a relevant structural parameter of the region of interest based on the detected key point. As shown in fig. 2a, the method may comprise the steps of:

s202, acquiring a medical image to be detected; the medical image to be detected comprises a region of interest.

Wherein the medical image to be detected may be a medical image of an object to be detected, which may be a human or a non-human, the medical image to be detected may be a CT (computed tomography, electronic computer tomography) image, a PET (positron emission tomography, positron emission computed tomography) image, an MR (Magnetic Resonance, nuclear magnetic resonance) image, an X-ray image, or the like; the medical image to be detected may be a one-dimensional image, a two-dimensional image, a three-dimensional image, etc. The region of interest may be the spine, or may be other organs, or may be any of the various vertebrae in the spine, such as the cervical vertebrae, the thoracic vertebrae, the lumbar vertebrae, and the like.

Specifically, the computer device may scan the object to be detected through the scanning device connected to the computer device, reconstruct and correct the scan data to obtain the medical image to be detected, or may read the medical image from a database in which the medical image to be detected is stored in advance, or may have other acquisition modes, which is not limited in this embodiment.

S204, inputting the medical image to be detected into a key point detection model to obtain a key point set corresponding to the region of interest; the key point detection model is obtained by training based on a sample medical image and a gold standard image corresponding to the sample medical image, wherein the gold standard image comprises key point marks corresponding to the sample medical image.

The key point detection model can be a convolutional neural network model, and of course can also be other neural network models, if the key point detection model is the convolutional neural network model, the key point detection model can be a deep convolutional neural network CNN, a generating type countermeasure network GAN, convolutional neural networks U-Net and V-Net or a cyclic neural network RNN and the like; alternatively, the keypoint detection model herein may be a V-Net model. In addition, each region of interest may include at least one key point, where the detected key points may be combined to obtain a set of key points, where the key points may be boundary points, center points, corner points of the artificially defined region of interest, and so on, and the key points may be, for example, a central line point of a vertebral body of the cervical spine or a corner point behind and above the vertebral body.

In this step, before using the keypoint detection model, the keypoint detection model may also be trained, with the following training procedure:

1) Generating training data sets

For a given N sample medical images, I keypoints P are marked for each sample medical image using labeling software _i (1.ltoreq.i.ltoreq.I, and I is a natural number), see FIG. 2b, which shows a schematic representation of the marking of keypoints on three sections (sagittal, tubular, horizontal) of a sample medical image, where the intersection points of the cross-hairs are the locations of the marked keypoints (it should be noted that this figure is only schematic and does not affect the essence of the scheme), after marking, the coordinates of each keypoint can be saved, and a mask image corresponding to each keypoint can be generated, where the mask image consists of I coordinates P of each keypoint _i The circular or spherical binary mask with the radius r is used as a circle center, and can be also called a gold standard image. So that I mask images can be obtained for each sample medical image,and then, matching each sample medical image with the corresponding I mask images to obtain N training data pairs, wherein the N training data pairs form a training data set. The size of N, I, r here may be as practical, and N may be 1000, i 24, r 6 pixels, and so on, for example.

2) Building convolutional neural network model

An initial convolutional neural network model is built, and super parameters of the convolutional neural network model are set, wherein an input channel of the network is 1, an output channel of the network is I+1, and the detection probability diagrams of the I key points and the detection probability diagrams of the background are respectively corresponding to each other; the network structure comprises an input module, two downsampling modules, two upsampling modules and an output module, wherein except the output module, the other modules use a batch normalization layer and a nonlinear activation function Relu, the nonlinear activation function in the output module uses softmax instead, the output value of the nonlinear activation function is in a (0, 1) interval, the last softmax is made among output channels, so that the sum of corresponding position elements in each finally output probability map is 1, and the probability values in the probability map respectively represent the probability that the current position pixel in the original map belongs to each label class. The training data set can be divided into a training set X1, a verification set X2 and a test set X3, the training set, the verification set and the test set are mutually independent, the corresponding numbers are N1, N2 and N3 respectively, the corresponding numbers are natural numbers, n1+n2+n3=N, and n1 is more than or equal to 1/2N. The sizes of N1, N2 and N3 herein may be determined according to practical situations, and N1, N2 and N3 may be 500, 200 and 300, respectively, assuming that N is 1000. In addition, the above super parameters may include network layer number, convolution kernel, learning rate, parameter initialization, training round number, and batch size.

3) Training convolutional neural networks

Training the convolutional neural network established in the step 2) by using the training data set generated in the step 1), wherein a training set X1 is used for training a convolutional neural network model, a verification set X2 is used for evaluating the current performance of the model, and a test set X3 is used for checking the generalization performance of the model; in the training process, the training set is divided into a plurality of batches and repeatedly input into a convolutional neural network model for training for a plurality of rounds, meanwhile, the difference between an output image and a mask image is calculated by using a loss function and is used as a training error to be fed back to the convolutional neural network model, and model parameters are updated through a learning algorithm; after the training of each batch is finished, performing performance test on the convolutional neural network model by using a verification set, and when the performance test index training tends to be stable, considering that the training of the convolutional neural network model is finished, and storing the trained network model. For example, assuming that N is 1000, the training batch may be 100, and the training set may be 100 rounds of training the convolutional neural network model by repeating 100 batches, where the loss function may be a Focal loss function (Focal loss), a set similarity metric function (Dice), and the learning algorithm may be an adaptive moment estimation optimization algorithm (Adaptive Moment Estimation, adam), a random gradient descent (Stochastic gradient descent, SGD), a Momentum algorithm (Momentum), and the like.

After the key point detection model is trained, the computer equipment can input the medical image to be detected into the key point detection model to obtain at least one key point on the region of interest, and then the key points are combined to obtain a corresponding key point set of the region of interest. The keypoints obtained by the keypoint detection model may include position information, category information, etc. of each keypoint, wherein the category information may include an organ type to which the keypoint belongs and a segment to which the organ belongs.

S206, determining relevant structural parameters of the region of interest based on the key point set corresponding to the region of interest; the relevant structural parameters are used to characterize the morphology of the region of interest.

In this step, the relevant structural parameters may also be referred to as eigenvalues, which may include anatomical parameters and morphological feature description parameters, wherein the anatomical parameters include: cobb angle (Cobb), intervertebral space distance, spinal canal diameter, vertebral body slip distance, morphometric characterization parameters include curvature, alignment, etc. From these relevant structural parameters, the type of lesion, the extent of the lesion, etc. of the region of interest can be determined.

Specifically, after obtaining the key point set corresponding to the region of interest, the computer device may obtain relevant structural parameters of the region of interest by calculating part of the key points in the key point set or calculating all the key points in the key point set. Of course, it is also possible to use different points in the set of key points to calculate different relevant structural parameters, etc. In addition, when the key points are marked on the sample medical image, the key points can be divided into one or more subsets, each subset is bound with a calculation method of related structural parameters, after the key point set corresponding to the region of interest is obtained, which subset the key points belong to can be determined first, and then the calculation method corresponding to the subset is used for calculating the key points to obtain the related structural parameters.

In the image detection method, the key point detection is carried out on the medical image to be detected by adopting the key point detection model, the key point set corresponding to the region of interest in the medical image to be detected is obtained, and the relevant structural parameters of the region of interest are determined based on the key point set corresponding to the region of interest, and can represent the morphology of the region of interest. In the method, the key point detection model can be adopted to detect the key points of the medical image of the testee, and a doctor is not required to manually mark the key points according to experience, so that the method can reduce the experience requirement of the doctor, and meanwhile, the doctor cannot consume too much energy and time, so that the labor can be saved; in addition, since the key point detection model is used for detecting the key points, the problem of error detection of the key points caused by fatigue of doctors is avoided, the key points detected by the method are accurate, so that when the accurate key points are used for determining the related structural parameters of the region of interest, the determined related structural parameters are accurate, and when the accurate related structural parameters are used for determining the subsequent treatment schemes, the determined treatment schemes are accurate.

In another embodiment, another image detection method is provided, and this embodiment relates to a specific process of determining a type of curvature of the region of interest based on the location information of each key point if the related structural parameter includes curvature and the set of key points includes the location information of each key point. On the basis of the above embodiment, as shown in fig. 3a, the method may further include the following steps:

s302, dividing each key point in the key point set on the region of interest based on the position information of each key point in the key point set to obtain at least one key point subset on the region of interest.

The position information of each key point may be coordinates of each key point, and may be one-dimensional coordinates, two-dimensional coordinates, three-dimensional coordinates, and the like.

Specifically, after the position information of each key point in the key point set is obtained, each key point in the key point set may be divided into a subset corresponding to the whole key point and a subset corresponding to a part of the key points according to a whole and/or a partial division concept based on the position information of each key point, that is, the positions of each key point, where the subset corresponding to the whole key point and the subset corresponding to the part of the key point may represent the shape of the region of interest, and the number of the divided subsets may be determined according to the actual situation.

In general, when evaluating the morphology of one organ using two points, there is a problem that the evaluation result is inaccurate due to a large amount of noise, and therefore, in general, each of the subsets includes at least three key points, and the position information of the key points included in each of the subsets has continuity.

For example, referring to fig. 3b, assuming that the key point set obtained by the detection model is 12 key points of the upper and lower corners behind the cervical vertebrae C2-C7, which are points 1-12, respectively, the 12 points of the C2-C7 can be divided into 5 subsets, for example, the whole subset, according to the location of each point and the concept of whole local area: c2-C7 (points 1-12), local subset: C2-C4 (point 1-point 6), C3-C5 (point 3-point 8), C4-C6 (point 5-point 10), C5-C7 (point 7-point 12). It should be noted that fig. 3b is only a schematic diagram, and does not affect the essence of the embodiments of the present application.

S304, determining the characteristic value corresponding to each key point subset based on at least one key point subset.

In this step, the feature value corresponding to each key point subset may be the shortest distance or the longest distance between the center point of the circular arc fitted by each key point in each subset and each key point, or may be the radius of the fitted circular arc, or may be the curvature of the fitted circular arc, or the like.

S306, determining the type of the curvature of the region of interest based on the feature value corresponding to each key point subset.

In this step, optionally, the feature value corresponding to each key point subset may be input into a classification model, to determine the class of the curvature of the region of interest; the classification model is obtained by training based on a sample medical image set, wherein the sample medical image set comprises feature values corresponding to each key point subset of the region of interest in the training medical image and labeling categories of curvature of the region of interest in the training medical image.

The classification model may be trained in advance before being classified by using the classification model, and the training process is as follows: obtaining sample medical image sets of different objects, wherein each training medical image in the sample medical image sets comprises a labeling category of the curvature of the region of interest, obtaining a key point set of the region of interest on each training medical image by using a key point detection model, then carrying out subset division on the key point set of each training medical image to obtain each key point subset corresponding to each training medical image, determining characteristic values corresponding to each key point subset, inputting the characteristic values corresponding to each key point subset of each training medical image into an initial classification model to obtain a prediction category of the curvature of the region of interest, and training the initial classification model according to the loss between the prediction category of the curvature and the labeling category of the curvature to obtain a trained classification model.

After training the classification model, the feature value corresponding to each key point subset obtained in the step S304 can be input into the classification model to obtain the curvature class of the region of interest, and after obtaining the curvature class, the doctor can determine the lesion type and the lesion degree of the object to be detected according to the curvature class. The classification model is utilized to obtain the class of the curvature, and compared with the class of the curvature obtained by manually analyzing the image through experience, the class of the curvature obtained by the embodiment is more accurate, and meanwhile, the labor and the detection time are saved; and simultaneously, the false detection rate caused by doctor fatigue is reduced.

In addition, the categories may be represented by numerals, letters, characters, or the like. The categories may include normal or abnormal, etc., although other categories may be included. For example, referring to fig. 3c, when the spine is in different forms, the spine types may include normal (also referred to as normal), straight, reverse arch, etc. (it should be noted that fig. 3c is only a schematic view, and does not affect the essential content of the embodiments of the present application). If the curvature class of the object to be detected is determined to be the anti-bow through the classification model, the curvature class can be recorded into a corresponding image report, and the object to be detected is reminded of cervical spondylosis.

According to the image detection method provided by the embodiment, if the relevant structural parameters of the region of interest include the curvature of the region of interest, the key point set can be divided into at least one subset according to the position information of each key point in the key point set, the characteristic value of each subset is obtained, and the category of the curvature of the region of interest is obtained based on the characteristic value of each subset. In this embodiment, since the key point set of the region of interest may be divided into a plurality of key point subsets, the obtained data volume may be more abundant, and thus, when the feature values of the plurality of key point subsets are used to determine the class of the curvature of the region of interest, the obtained class of the curvature may be more accurate.

In another embodiment, another image detection method is provided, and this embodiment relates to a specific process of how to determine the corresponding feature value based on the key point subset. On the basis of the above embodiment, as shown in fig. 4, the step S304 may include the following steps:

s402, fitting processing is carried out on each key point in each key point subset, and a circle corresponding to each key point subset is determined.

In the step, the fitting process can adopt interpolation, polishing, least square method and the like to perform fitting, and each key point in each key point subset can be subjected to iterative fitting in the fitting process to obtain a circle fitted for each key point subset. The circle fitted to each key point subset also needs to satisfy a certain condition, taking a key point subset as an example, and finally the circle fitted to the key point subset needs to make the average distance from each key point in the key point subset to the circular arc shortest.

S404, determining the target radius of the circle corresponding to each key point subset based on the circle corresponding to each key point subset and the position information of each key point in each key point subset; the target radius is a radius with positive and negative signs.

In this step, optionally, the target radius may be obtained using the following steps A1-A4, as follows:

a1, obtaining the circle center position of a circle corresponding to each key point subset according to the circle corresponding to each key point subset;

a3, obtaining a relative position result of the circle center position of each key point subset and the corresponding circle based on the circle center position of the circle corresponding to each key point subset and the position information of each key point in each key point subset;

a1, if the circle center position of a circle corresponding to a key point subset in the relative position result is at the left side of the key point subset, determining the radius of the circle corresponding to the key point subset as a target radius;

and step A4, if the circle center position of the circle corresponding to the other key point subset in the relative position result is at the right side of the key point subset, setting the radius of the circle corresponding to the other key point subset as a negative radius, and determining the negative radius as a target radius.

In S402, when a corresponding circle is fitted to each key point subset, information such as a circle center position of the fitted circle, a circle radius of the circle, and the like can be obtained, that is, a circle center and a circle center coordinate of the circle corresponding to each key point subset can be obtained, then a positive direction is defined, and a general right side is a positive direction. Continuing with the example of the cervical vertebrae C2-C7 cone divided into 5 key point subsets, 5 circles can be obtained by fitting, and 5 target radii can be obtained.

S406, obtaining curvature values corresponding to the key point subsets according to the target radius of the circle corresponding to the key point subsets, and determining the curvature values corresponding to the key point subsets as the characteristic values corresponding to the key point subsets.

Specifically, the derivative of the target radius of the circle corresponding to each key point subset may be obtained by taking the derivative of the target radius, and the derivative of the target radius is recorded as the curvature corresponding to each key point subset.

According to the image detection method provided by the embodiment, curve fitting can be performed on each key point subset to obtain circles corresponding to the key point subsets, the target radius of each circle is obtained based on the obtained circles and the key point positions of the key point subsets, and the curvature corresponding to the target radius of each circle is used as a characteristic value. In this embodiment, since the fitting circle and the radius of the fitting circle can be obtained by fitting the subset of the key points, and the radius of the fitting circle can represent the bending degree of the region of interest, the curvature of the region of interest can be quantified, so that the obtained classification result is relatively accurate when curvature classification is performed by using the curvature corresponding to the radius of the circle.

In another embodiment, another image detection method is provided, and this embodiment relates to a specific process of how to determine relevant structural parameters of a region of interest. On the basis of the above embodiment, the step S206 may include the following step B:

step B, calculating the distance between points in the key point set based on the position information of each key point in the key point set corresponding to the region of interest, and obtaining the related structural parameters of the region of interest; and/or calculating the distance between lines formed by the key points in the key point set based on the position information of each key point in the key point set corresponding to the region of interest to obtain the related structural parameters of the region of interest; and/or calculating the distance between the line formed by each key point in the key point set and the point based on the position information of each key point in the key point set corresponding to the region of interest, so as to obtain the related structural parameters of the region of interest; and/or calculating the area of the surface formed by each key point in the key point set based on the position information of each key point in the key point set corresponding to the region of interest, and obtaining the related structural parameters of the region of interest.

In this step, taking the relevant parameters given in S206 as an example, the distance between points may be represented by the distance between the intervertebral spaces, the diameter of the cone, and the like, and during calculation, an L2 norm between the spatial position coordinates of two key points in the set of key points may be calculated to obtain a distance value; the Kebujiao can represent the distance between lines, and when in calculation, a line can be formed by gathering any two key points in the key points to obtain corresponding space vectors, the space vectors corresponding to the two lines are obtained by the method, and then the dot product of the space vectors of the two lines is calculated to obtain a vector included angle; the cone slipping distance and the line sequence can represent the distance between lines and points, when in calculation, a space vector corresponding to one line can be obtained through two key points in the key point set, and then the space distance between the other key point and the space vector is calculated to obtain a distance value; the curvature can represent the area of the surface, the fitted circle can be obtained by fitting the key points in the key point set, and the area of the fitted circle can be calculated. The calculated values may then be used as relevant structural parameters of the region of interest.

The image detection method provided by the embodiment can calculate the distance between the points in the key point set, the distance between the points and the line, the distance between the line and the area of the surface formed by the points based on the position information of each key point in the key point set corresponding to the region of interest, and can serve as related structural parameters, so that a data basis can be provided for judging the type and the lesion degree of a subsequent focus, and a reference basis can be provided for the subsequent operation planning or the postoperative rehabilitation.

In another embodiment, another image detection method is provided, and this embodiment relates to a specific process of how to obtain a key point set of a region of interest. On the basis of the above embodiment, as shown in fig. 5, the step S204 may include the following steps:

s502, inputting the medical image to be detected into a key point detection model to obtain a probability map of at least one key point corresponding to the region of interest in the medical image to be detected.

The pixel value of each position on the probability map of each key point is the probability that the pixel value of the corresponding position on the medical image to be detected belongs to one key point of the region of interest. The background probability map can also be obtained through the key point detection model, and the probability value of each position on the background probability map is the probability value of the background of the corresponding position on the image to be detected.

S504, determining at least one candidate point corresponding to each key point of the region of interest based on the probability map of the at least one key point.

In this step, taking I keypoints on each medical image to be detected as an example, I keypoint probability maps can be obtained, and based on the I probability maps, the corresponding keypoint coordinates can be obtained, where multiple candidate points may appear in each keypoint coordinate. For probability maps of any keypoint, optionally, the keypoint coordinates may be obtained using steps C1-C3 as follows:

and step C1, carrying out binarization processing on the probability map of each key point according to a preset probability threshold value to obtain a binarization mask image corresponding to the probability map of each key point.

The probability threshold here may be 0.3, 0.35, 0.5, etc. For example, here, the probability smaller than the probability threshold may be set to 0, and the probability larger than the probability threshold may be set to 1, resulting in a binarized mask image.

And C2, marking the connected domains in each binarized mask image, and determining candidate connected domains with areas larger than a preset area threshold according to the marked connected domains.

Here the area threshold may be 30, 35, 40, etc. Here, the connected domain with the area smaller than the area threshold in each binarized mask image may be deleted, so as to obtain a candidate connected domain with the area greater than or equal to the area threshold.

And C3, calculating a weighted center and an average probability value of probability values in a probability map of the key points corresponding to each candidate connected domain, and obtaining at least one candidate point corresponding to each key point of the region of interest.

The probability value on the probability map is used as a weight, weighted summation is carried out in the region of the probability map corresponding to each candidate connected domain, so as to obtain a weighted center point corresponding to each candidate connected domain, and meanwhile, the probability value in the region of the probability map corresponding to each candidate connected domain can be averaged to obtain an average probability value, and the point corresponding to the weighted center point and the average probability value is used as a candidate point of the candidate connected domain.

S506, processing at least one candidate point corresponding to each key point of the region of interest by adopting a dynamic programming algorithm to obtain a target point corresponding to each key point of the region of interest.

In this step, since there may be a plurality of candidate connected regions larger than the area threshold on each binarized mask image, there may be a plurality of candidate points, and each probability map corresponds to only one key point, and then one point needs to be selected from the plurality of candidate points as the key point.

During screening, a dynamic programming algorithm can be adopted, geometric constraint processing is carried out on a plurality of candidate points corresponding to each key point probability map through preset constraint conditions, a key point chain with the largest cumulative probability is obtained, coordinates of each key point in the chain are output as detection results, and therefore a target point corresponding to each key point probability map can be obtained, and the target point is used as a key point on the probability map.

S508, obtaining a key point set corresponding to the region of interest based on one target point corresponding to each key point of the region of interest.

In the step, all the obtained target points can be spliced or combined, and then the key point set corresponding to the region of interest can be obtained.

According to the image detection method provided by the embodiment, the medical image to be detected can be input into the key point detection model to obtain the probability map corresponding to each key point, a plurality of candidate points corresponding to each key point are determined based on the probability map corresponding to each key point, the plurality of candidate points corresponding to each key point are screened by adopting a dynamic programming algorithm to obtain one target point corresponding to each key point, and a key point set is obtained based on the obtained plurality of target points. In this embodiment, since the key point detection model and the dynamic programming algorithm can be used to obtain the key point set, the whole key point detection process is very fast and takes short time, the accuracy of the key point detection result is improved, and the robustness is high.

In another embodiment, in order to facilitate a more detailed description of the technical solution of the present application, the following description is provided in connection with a more detailed embodiment, and the method may include the following steps S1-S10:

s1, acquiring a medical image to be detected; the medical image to be detected comprises a region of interest;

s2, inputting the medical image to be detected into a key point detection model to obtain a probability map of at least one key point corresponding to the region of interest;

s3, carrying out binarization processing on the probability map of each key point, marking the connected domain, and determining candidate connected domains with areas larger than a threshold value;

s4, calculating a weighted center and an average probability value of probability values in a probability map corresponding to each candidate connected domain, and obtaining a plurality of candidate points;

s5, determining a target point corresponding to each key point from a plurality of candidate points based on a dynamic programming algorithm, and obtaining a key point set corresponding to the region of interest;

s6, calculating relevant structural parameters of the region of interest based on the key point set corresponding to the region of interest;

s7, dividing each key point in the key point set on the region of interest to obtain at least one key point subset on the region of interest;

S8, fitting each key point in each key point subset to determine a circle corresponding to each key point subset;

s9, based on the circles corresponding to the key point subsets, obtaining the radius and curvature values of the circles corresponding to the key point subsets;

s10, inputting the curvature corresponding to each key point subset into the classification model to obtain the type of the curvature of the region of interest.

It should be understood that, although the steps in the flowcharts of fig. 2a, 3a, 4, 5 are shown in order as indicated by the arrows, these steps are not necessarily performed in order as indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps of fig. 2a, 3a, 4, 5 may comprise sub-steps or phases which are not necessarily performed at the same time but may be performed at different times, nor does the order of execution of the sub-steps or phases necessarily follow one another, but may be performed alternately or alternately with at least some of the other steps or phases.

In one embodiment, as shown in fig. 6, there is provided an image detection apparatus including: an acquisition module 10, a detection module 11 and a determination module 12, wherein:

an acquisition module 10 for acquiring a medical image to be detected; the medical image to be detected comprises a region of interest;

the detection module 11 is used for inputting the medical image to be detected into the key point detection model to obtain a key point set corresponding to the region of interest; the key point detection model is obtained by training based on a sample medical image and a gold standard image corresponding to the sample medical image, wherein the gold standard image comprises a key point mark corresponding to the sample medical image;

a determining module 12, configured to determine relevant structural parameters of the region of interest based on the set of key points corresponding to the region of interest; the relevant structural parameters are used to characterize the morphology of the region of interest.

For specific limitations of the image detection apparatus, reference may be made to the above limitations of the image detection method, and no further description is given here.

In another embodiment, another image detection apparatus is provided, where, based on the above embodiment, if the relevant structural parameter of the region of interest includes curvature of the region of interest, the set of key points includes location information of each key point; the apparatus may further include: the device comprises a dividing module, a characteristic value determining module and a category determining module, wherein:

The division module is used for dividing each key point in the key point set on the region of interest based on the position information of each key point in the key point set to obtain at least one key point subset on the region of interest;

the characteristic value determining module is used for determining the characteristic value corresponding to each key point subset based on at least one key point subset;

and the category determining module is used for determining the category of the curvature of the region of interest based on the characteristic value corresponding to each key point subset.

Optionally, the above-mentioned category determining module is further configured to input a feature value corresponding to each subset of the key points into the classification model, and determine a category of curvature of the region of interest; the classification model is obtained by training based on a sample medical image set, wherein the sample medical image set comprises feature values corresponding to each key point subset of the region of interest in the training medical image and labeling categories of curvature of the region of interest in the training medical image.

In another embodiment, another image detection apparatus is provided, and the feature value determining module may include a fitting unit, a first determining unit, and a second determining unit, where:

The fitting unit is used for carrying out fitting processing on each key point in each key point subset and determining a circle corresponding to each key point subset;

a first determining unit, configured to determine a target radius of a circle corresponding to each of the key point subsets based on the circle corresponding to each of the key point subsets and the position information of each of the key points in each of the key point subsets; the target radius is a radius with positive and negative signs;

and the second determining unit is used for obtaining the curvature value corresponding to each key point subset according to the target radius of the circle corresponding to each key point subset, and determining the curvature value corresponding to each key point subset as the characteristic value corresponding to each key point subset.

Optionally, the first determining unit is further configured to obtain a center position of a circle corresponding to each key point subset according to the circle corresponding to each key point subset; based on the circle center position of the circle corresponding to each key point subset and the position information of each key point in each key point subset, obtaining a relative position result of each key point subset and the circle center position of the corresponding circle; if the circle center position of the circle corresponding to one key point subset in the relative position result is at the left side of the key point subset, determining the radius of the circle corresponding to the one key point subset as a target radius; and if the circle center position of the circle corresponding to the other key point subset in the relative position result is at the right side of the key point subset, setting the radius of the circle corresponding to the other key point subset as a negative radius, and determining the negative radius as a target radius.

In another embodiment, another image detection apparatus is provided, and based on the foregoing embodiment, the determining module 12 is further configured to calculate, based on position information of each of the key points in the key point set corresponding to the region of interest, a distance between the points in the key point set, and obtain a relevant structural parameter of the region of interest; and/or calculating the distance between lines formed by the key points in the key point set based on the position information of each key point in the key point set corresponding to the region of interest to obtain the related structural parameters of the region of interest; and/or calculating the distance between the line formed by each key point in the key point set and the point based on the position information of each key point in the key point set corresponding to the region of interest, so as to obtain the related structural parameters of the region of interest; and/or calculating the area of the surface formed by each key point in the key point set based on the position information of each key point in the key point set corresponding to the region of interest, and obtaining the related structural parameters of the region of interest.

In another embodiment, another image detecting apparatus is provided, and the detecting module 11 may include: the device comprises a detection unit, a candidate point determination unit, a screening unit and a target point determination unit, wherein:

The detection unit is used for inputting the medical image to be detected into the key point detection model to obtain a probability map of at least one key point corresponding to the region of interest in the medical image to be detected;

a candidate point determining unit, configured to determine at least one candidate point corresponding to each key point of the region of interest based on the probability map of the at least one key point;

the screening unit is used for processing at least one candidate point corresponding to each key point of the region of interest by adopting a dynamic programming algorithm to obtain a target point corresponding to each key point of the region of interest;

the target point determining unit is used for obtaining a key point set corresponding to the region of interest based on one target point corresponding to each key point of the region of interest.

Optionally, the candidate point determining unit is further configured to perform binarization processing on the probability map of each key point according to a preset probability threshold, so as to obtain a binarized mask image corresponding to the probability map of each key point; marking the connected domains in each binarized mask image, and determining candidate connected domains with areas larger than a preset area threshold according to the marked connected domains; and calculating a weighted center and an average probability value of probability values in a probability map of the key points corresponding to each candidate connected domain, and obtaining at least one candidate point corresponding to each key point of the region of interest.

The respective modules in the above-described image detection apparatus may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.

In one embodiment, a computer device is provided comprising a memory and a processor, the memory having stored therein a computer program, the processor when executing the computer program performing the steps of:

In one embodiment, the processor when executing the computer program further performs the steps of:

dividing each key point in the key point set on the region of interest based on the position information of each key point in the key point set to obtain at least one key point subset on the region of interest; determining a feature value corresponding to each key point subset based on at least one key point subset; and determining the category of the curvature of the region of interest based on the feature value corresponding to each key point subset.

fitting each key point in each key point subset to determine a circle corresponding to each key point subset; determining a target radius of a circle corresponding to each key point subset based on the circle corresponding to each key point subset and the position information of each key point in each key point subset; the target radius is a radius with positive and negative signs; and obtaining curvature values corresponding to the key point subsets according to the target radius of the circle corresponding to the key point subsets, and determining the curvature values corresponding to the key point subsets as the characteristic values corresponding to the key point subsets.

obtaining the circle center position of the circle corresponding to each key point subset according to the circle corresponding to each key point subset; based on the circle center position of the circle corresponding to each key point subset and the position information of each key point in each key point subset, obtaining a relative position result of each key point subset and the circle center position of the corresponding circle; if the circle center position of the circle corresponding to one key point subset in the relative position result is at the left side of the key point subset, determining the radius of the circle corresponding to the one key point subset as a target radius; and if the circle center position of the circle corresponding to the other key point subset in the relative position result is at the right side of the key point subset, setting the radius of the circle corresponding to the other key point subset as a negative radius, and determining the negative radius as a target radius.

calculating the distance between the points in the key point set based on the position information of each key point in the key point set corresponding to the region of interest, and obtaining the related structural parameters of the region of interest; and/or calculating the distance between lines formed by the key points in the key point set based on the position information of each key point in the key point set corresponding to the region of interest to obtain the related structural parameters of the region of interest; and/or calculating the distance between the line formed by each key point in the key point set and the point based on the position information of each key point in the key point set corresponding to the region of interest, so as to obtain the related structural parameters of the region of interest; and/or calculating the area of the surface formed by each key point in the key point set based on the position information of each key point in the key point set corresponding to the region of interest, and obtaining the related structural parameters of the region of interest.

inputting the medical image to be detected into a key point detection model to obtain a probability map of at least one key point corresponding to the region of interest in the medical image to be detected; determining at least one candidate point corresponding to each key point of the region of interest based on the probability map of the at least one key point; processing at least one candidate point corresponding to each key point of the region of interest by adopting a dynamic programming algorithm to obtain a target point corresponding to each key point of the region of interest; and obtaining a key point set corresponding to the region of interest based on one target point corresponding to each key point of the region of interest.

performing binarization processing on the probability map of each key point according to a preset probability threshold value to obtain a binarization mask image corresponding to the probability map of each key point; marking the connected domains in each binarized mask image, and determining candidate connected domains with areas larger than a preset area threshold according to the marked connected domains; and calculating a weighted center and an average probability value of probability values in a probability map of the key points corresponding to each candidate connected domain, and obtaining at least one candidate point corresponding to each key point of the region of interest.

In one embodiment, a readable storage medium is provided having a computer program stored thereon, which when executed by a processor, performs the steps of:

In one embodiment, the computer program when executed by the processor further performs the steps of:

Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the various embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), memory bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.

The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.

The above examples merely represent a few embodiments of the present application, which are described in more detail and are not to be construed as limiting the scope of the invention. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application is to be determined by the claims appended hereto.

Claims

1. An image detection method, the method comprising:

Determining related structural parameters of the region of interest based on the key point set corresponding to the region of interest; the relevant structural parameters are used for representing the morphology of the region of interest;

the relevant structural parameters of the region of interest comprise the curvature of the region of interest, and the key point set comprises the position information of each key point; the method further comprises the steps of:

determining a feature value corresponding to each key point subset based on the at least one key point subset;

2. The method of claim 1, wherein determining the class of curvature of the region of interest based on the feature values corresponding to each of the keypoint subsets comprises:

inputting the characteristic value corresponding to each key point subset into a classification model, and determining the class of the curvature of the region of interest; the classification model is obtained by training based on a sample medical image set, wherein the sample medical image set comprises feature values corresponding to each key point subset of the region of interest in a training medical image and annotation categories of the curvature of the region of interest in the training medical image.

3. The method according to claim 1 or 2, wherein said determining, based on said at least one keypoint subset, a corresponding feature value for each said keypoint subset comprises:

and obtaining a curvature value corresponding to each key point subset according to the target radius of the circle corresponding to each key point subset, and determining the curvature value corresponding to each key point subset as a characteristic value corresponding to each key point subset.

4. The method of claim 3, wherein determining the target radius of the circle corresponding to each of the keypoint subsets based on the circle corresponding to each of the keypoint subsets and the location information of the respective keypoints in each of the keypoint subsets comprises:

Based on the circle center position of the circle corresponding to each key point subset and the position information of each key point in each key point subset, obtaining a relative position result of the circle center position of each key point subset and the corresponding circle;

5. The method according to claim 1 or 2, wherein the determining the relevant structural parameters of the region of interest based on the set of key points corresponding to the region of interest comprises:

calculating the distance between points in the key point set based on the position information of each key point in the key point set corresponding to the region of interest, and obtaining the related structural parameters of the region of interest;

And/or calculating the distance between lines formed by the key points in the key point set based on the position information of each key point in the key point set corresponding to the region of interest, so as to obtain the related structural parameters of the region of interest;

and/or calculating the area of the surface formed by each key point in the key point set based on the position information of each key point in the key point set corresponding to the region of interest, so as to obtain the related structural parameters of the region of interest.

6. The method according to claim 1 or 2, wherein the inputting the medical image to be detected into a keypoint detection model to obtain the set of keypoints corresponding to the region of interest comprises:

inputting the medical image to be detected into a key point detection model to obtain a probability map of at least one key point corresponding to a region of interest in the medical image to be detected;

7. The method of claim 6, wherein determining at least one candidate point for each keypoint of the region of interest based on the probability map of the at least one keypoint comprises:

8. The method according to claim 1 or 2, wherein each of the keypoints subsets comprises at least three of the keypoints, and wherein the position information of the keypoints in each of the keypoints subsets has continuity.

9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any one of claims 1 to 8 when the computer program is executed.

10. A readable storage medium having stored thereon a computer program, which when executed by a processor realizes the steps of the method according to any of claims 1 to 8.