CN113706600A

CN113706600A - Method, apparatus and medium for measuring critical dimensions of a body part

Info

Publication number: CN113706600A
Application number: CN202110956493.6A
Authority: CN
Inventors: 喻剑舟; 汤兆洋; 刘健; 王栋梁
Original assignee: Beijing Ouying Information Technology Co Ltd
Current assignee: Beijing Ouying Information Technology Co Ltd
Priority date: 2021-08-19
Filing date: 2021-08-19
Publication date: 2021-11-26

Abstract

Embodiments of the present disclosure relate to methods, devices, and media for measuring a critical dimension of a body part. According to the method, a medical image picture of a body part is acquired; determining a plurality of key points associated with a key size to be measured of the body part on the medical image picture based on a trained key point detection model associated with the body part, wherein the key size to be measured at least comprises at least one of a key angle to be measured and a key line length to be measured; for each key size to be measured, determining a side line associated with the key size to be measured on the medical image picture based on the key point associated with the key size to be measured in the plurality of key points; and calculating the value of the critical dimension to be measured based on the edge line. Therefore, the key size of the body part can be accurately and quickly measured without the need of the user to have related professional knowledge.

Description

Method, apparatus and medium for measuring critical dimensions of a body part

Technical Field

Embodiments of the present disclosure relate generally to the field of smart detection and, more particularly, to a method, apparatus, and medium for measuring a critical dimension of a body part.

Background

With the popularization of mobile medical treatment and cloud imaging, it is often necessary to determine the growth of a body part by means of medical image pictures (e.g., X-ray images or magnetic resonance images, etc.). In particular, it is desirable to measure critical dimensions of a body part based on medical image pictures, such as, for example only, for the hip, it is often desirable to measure critical dimensions such as the CE angle (edge center angle) of the hip joint; for the cervical vertebrae, it is usually necessary to measure the critical dimensions of the cervical intervertebral disc, such as the anterior height (anterior height), the middle height (middle height), the posterior height (spatial height), the intervertebral angle (intervertebral angle), the sagittal diameter (sagittal diameter), and the transverse diameter (transverse diameter). However, the measurement based on the medical image picture needs to quantitatively analyze the relationship between the anatomical key points in the image picture by combining the image features and the subject features, so a measurer has to have a very high level of expertise to accurately locate the anatomical key points and accurately measure the required size based on the key points.

Therefore, there is a need to provide a technique for automatically measuring medical image pictures of body parts, so as to accurately and rapidly measure the critical dimensions of the body parts without requiring the user to have professional knowledge.

Disclosure of Invention

In view of the above problems, the present disclosure provides a method and apparatus for measuring key dimensions of a body part, which can accurately and quickly measure key dimensions of a body part by any user without special learning of related professional knowledge by automatically determining key points associated with key dimensions to be measured using a trained key point detection model and automatically measuring the key dimensions based on the key points.

According to a first aspect of the present disclosure, there is provided a method for measuring a critical dimension of a body part, the method comprising: acquiring a medical image picture of the body part; determining a plurality of key points associated with a key size to be measured of the body part on the medical image picture based on a trained key point detection model associated with the body part, wherein the key size to be measured at least comprises at least one of a key angle to be measured and a key line length to be measured; for each key size to be measured, determining an edge line associated with the key size to be measured on the medical image picture based on a key point associated with the key size to be measured in the plurality of key points; and calculating the value of the critical dimension to be measured based on the edge line.

According to a second aspect of the present disclosure, there is provided a computing device comprising: at least one processor; and a memory communicatively coupled to the at least one processor; the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of the first aspect of the disclosure.

In a third aspect of the present disclosure, there is provided a non-transitory computer readable storage medium having stored thereon computer instructions, characterized in that the computer instructions are for causing the computer to perform the method of the first aspect of the present disclosure.

In some embodiments, determining, on the medical image picture, a plurality of keypoints associated with a measured critical dimension of the body part based on a trained keypoint detection model associated with the body part comprises: inputting the medical image pictures into the trained keypoint detection model to obtain a plurality of keypoint heat maps; extracting the position with the highest heat value in each key point heat map as a hot point; and determining the coordinates of the corresponding key points respectively based on the coordinates of each hot spot.

In some embodiments, for the key angle to be measured, determining, on the medical image, an edge associated with the key size to be measured based on a key point associated with the key size to be measured in the plurality of key points includes: connecting four key points in the plurality of key points, which are associated with the key angle to be detected, into a first line segment and a second line segment; determining whether the first line segment and the second line segment intersect; in response to determining that the first line segment and the second line segment intersect, determining that the first line segment and the second line segment are side lines associated with the key angle to be measured, wherein the key angle to be measured is an acute angle or a right angle included angle between the first line segment and the second line segment; and in response to determining that the first line segment and the second line segment do not intersect, translating the second line segment such that feature points of the second line segment coincide with feature points of the first line segment, and determining that the first line segment and the translated second line segment are edges associated with the key angle to be measured, wherein the key angle to be measured is an acute or right angle between the first line segment and the translated second line segment.

In some embodiments, the feature point of the second line segment is a midpoint of the second line segment, and the feature point of the first line segment is a midpoint of the first line segment.

In some embodiments, for the key angle to be measured, determining, on the medical image, an edge associated with the key size to be measured based on a key point associated with the key size to be measured in the plurality of key points includes: determining a first reference line associated with the key angle to be measured based on a first key point and a second key point associated with the key angle to be measured in the plurality of key points; determining a second reference line associated with the key angle to be measured based on a third key point and a fourth key point associated with the key angle to be measured in the plurality of key points, wherein the key angle to be measured is an acute angle or a right-angle included angle between the first reference line and the second reference line.

In some embodiments, determining the first reference line associated with the key angle to be measured based on the first and second key points of the plurality of key points associated with the key angle to be measured comprises: connecting the first key point and the second key point into a third reference line; and drawing a line perpendicular to the third reference line on the medical image picture as the first reference line.

In some embodiments, determining the second reference line associated with the key angle to be measured based on the third and fourth key points of the plurality of key points associated with the key angle to be measured comprises: drawing a straight line passing through the third key point and the fourth key point as the second reference line.

In some embodiments, the determining, on the medical image, an edge for associating with the critical dimension to be measured based on a keypoint of the plurality of keypoints associated with the critical dimension to be measured includes: connecting two key points associated with each key line segment to be tested in the plurality of key points so as to obtain the key line segment to be tested; and wherein calculating the value of the critical dimension to be measured based on the edge comprises: and calculating the length of the key line segment to be measured by using a scale, wherein the length of the scale is predetermined.

In some embodiments, calculating the length of the key line segment to be measured using a scale comprises: and comparing the pixels occupied by the key line segment to be measured with the pixels occupied by the scale so as to convert the length of the key line segment to be measured based on the length of the scale and the comparison result.

In some embodiments, wherein the keypoint detection model is trained via: obtaining sample medical image pictures of the body part, wherein the sample medical image pictures all comprise a plurality of labeled key points associated with the key dimension to be measured; inputting the sample medical image picture into the key point detection model to obtain a plurality of key point heat maps; extracting the position with the highest heat value in each key point heat map as a hot point; and adjusting the neural network parameters of the key point detection model based on the coordinates of each hot point and the corresponding marked key point.

It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.

Drawings

The above and other features, advantages and aspects of various embodiments of the present disclosure will become more apparent by referring to the following detailed description when taken in conjunction with the accompanying drawings. In the drawings, like or similar reference characters designate like or similar elements.

Fig. 1 shows a schematic view of a system 1 for implementing a method for measuring a critical dimension of a body part according to an embodiment of the invention.

Fig. 2 shows a flow diagram of a method 200 for measuring a critical dimension of a body part according to an embodiment of the present disclosure.

FIG. 3 shows a schematic diagram of an example keypoint detection model 300, according to an embodiment of the present disclosure.

FIG. 4 shows a flow diagram of a method 400 for training a keypoint detection model, according to an embodiment of the disclosure.

Fig. 5 shows a flow diagram of a method 500 for determining a plurality of keypoints associated with a measured critical dimension of a body part, in accordance with an embodiment of the present disclosure.

FIG. 6 shows a schematic diagram of a method of determining keypoints based on a keypoint detection model, according to an embodiment of the disclosure.

Fig. 7 shows a flowchart of a method 700 for determining an edge associated with a key angle to be measured, in accordance with an embodiment of the present disclosure.

Fig. 8 shows a flow diagram of a method 800 for determining an edge associated with a key angle to be measured, in accordance with an embodiment of the present disclosure.

Fig. 9 shows a block diagram of an electronic device according to an embodiment of the disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

The term "include" and variations thereof as used herein is meant to be inclusive in an open-ended manner, i.e., "including but not limited to". Unless specifically stated otherwise, the term "or" means "and/or". The term "based on" means "based at least in part on". The terms "one example embodiment" and "one embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first," "second," and the like may refer to different or the same object. Other explicit and implicit definitions are also possible below.

As described above, the measurement based on the medical image picture requires quantitative analysis of the relationship between the anatomical key points in the image picture in combination with the image features and the subject features, so the measurer must have a very high level of expertise to accurately locate the anatomical key points and accurately measure the required dimensions based on the key points. However, the conventional method is generally implemented manually by medical professionals, which is inefficient on the one hand, and only professionals with relevant professional knowledge may accurately measure the relevant dimensions on the other hand, thereby limiting the use of the general public.

To address at least in part one or more of the above issues and other potential issues, an example embodiment of the present disclosure presents a method for measuring a critical dimension of a body part, comprising: acquiring a medical image picture of a body part; determining, on the medical image, a plurality of keypoints associated with a critical dimension of the body part to be measured based on a trained keypoint detection model associated with the body part; for each key size to be measured, determining a side line associated with the key size to be measured on the medical image picture based on the key point associated with the key size to be measured in the plurality of key points; and calculating the value of the critical dimension to be measured based on the edge line. In this way, the key size of the body part can be accurately and quickly measured by the general public without the requirement that the user has related professional knowledge.

Fig. 1 shows a schematic view of a system 1 for implementing a method for measuring a critical dimension of a body part according to an embodiment of the invention. As shown in fig. 1, system 1 includes a computing device 10, a server 20, and a network 30. Computing device 10 and server 20 may interact with data via network 30. Here, the server 20 may be, for example, a server of a service provider dedicated to providing medical image measurement services, and the computing device 10 may communicate with the server 20 via the network 30 with the server 20 to implement measurement of a critical dimension in a medical image picture. The computing device 10 may include at least one processor 110 and at least one memory 120 coupled to the at least one processor 110, the memory 120 having stored therein instructions 130 executable by the at least one processor 110, the instructions 130 when executed by the at least one processor 110 performing the method 200 as described below. Note that in this context, computing device 10 may be part of server 20 or may be separate from server 20. The specific structure of computing device 10 or server 20 may be described, for example, in connection with FIG. 9, below. Here, the computing device 10 may be a mobile electronic device or a stationary electronic device, such as a cell phone, tablet, desktop, or the like.

Fig. 2 shows a flow diagram of a method 200 for measuring a critical dimension of a body part according to an embodiment of the present disclosure. The method 200 may be performed by the computing device 10 as shown in FIG. 1, or may be performed at the electronic device 900 shown in FIG. 9. For example, portions related to the keypoint detection model may be executed by server 20 while other portions are executed by computing device 10. It should be understood that method 200 may also include additional blocks not shown and/or may omit blocks shown, as the scope of the present disclosure is not limited in this respect.

At step 202, computing device 10 obtains a medical image picture of the body part. In the present disclosure, the medical image picture may be a picture in various formats, such as a picture in JPG format or a picture in Dicom (Digital Imaging and Communications in Medicine) format.

In the present disclosure, a medical image refers to an image of internal tissues obtained in a non-invasive manner with respect to a human body or a certain body part of the human body for medical treatment or medical research, for example, an image of a body part obtained by X-ray imaging (X-CT), magnetic resonance imaging, nuclear medicine imaging, ultrasonic imaging, or the like. In addition, in the present disclosure, the term "medical image picture related to a body part" is not limited to indicate a specific body part of a human body (for example, hip, knee, cervical vertebra, etc.), but may also indicate the whole body of the human body or a combination of a plurality of body parts.

At step 204, computing device 10 determines, on the medical image picture, a plurality of keypoints associated with a measured critical dimension of the body part based on a trained keypoint detection model associated with the body part. In the present disclosure, the critical dimension to be measured at least includes at least one of a critical angle to be measured and a critical line length to be measured.

For example, in the case where the body part to be examined is a hip, the measurement parameter that is usually required is the hip CE angle, and therefore, key points such as a hip outer edge point, a femoral head center point, a tear drop point, etc. need to be marked in a medical image of the hip. In the present disclosure, the keypoints are located or determined on the medical image picture by invoking a trained keypoint detection model associated with the hip. The keypoint detection model may be located on server 20, and computing device 10 enables the localization of keypoints by remotely invoking the model. The keypoint detection model may also be downloaded to or stored locally on computing device 10 for direct local invocation to locate keypoints.

In some embodiments, the keypoint detection model in the present disclosure may adopt an architecture of a High-resolution network (HRNet) as shown in fig. 3, where the network takes a High-resolution subnet as a first stage, then gradually increases the High-resolution to a low-resolution subnet, forms more stages, and connects the multiple-resolution subnets in parallel. The multi-scale iterative fusion is also performed by repeatedly exchanging information over parallel multi-resolution subnetworks throughout the process, thereby enabling the maintenance of a high resolution representation throughout the process. In the present application, the corresponding keypoints may be estimated based on the high resolution points (i.e., the highest heating value points) of the network output.

In some embodiments, the keypoint detection model of the present disclosure may also employ other neural Network architectures, such as Deep residual Network (ResNet), Houglass Network, Generative Adaptive Network (GAN), and the like. In some embodiments, the key point detection models may also be trained based on a plurality of different neural network architectures, and one of the key point detection models with the best learning effect is selected as the key point detection model that is finally used for measuring the key size of the body part.

In some embodiments, the keypoint detection model may be trained using a plurality of sample medical image pictures. For example, a sample medical image picture comprising a plurality of labeled keypoints associated with a critical dimension of a body part to be measured may be input into the keypoint detection model, which, upon receiving the sample medical image picture, outputs a plurality of keypoint heatmaps; the position with the highest heat value (i.e., the highest resolution) can be extracted from each key point heat map as a hot spot, and then the neural network parameters of the key point detection model are adjusted based on the coordinates of each hot spot and the corresponding coordinates of the labeled key points, respectively. The method 400 for training the keypoint detection model is described in further detail below in conjunction with FIG. 4.

At step 206, for each critical dimension to be measured, computing device 10 determines an edge line associated with the critical dimension to be measured on the medical image based on a keypoint of the plurality of keypoints associated with the critical dimension to be measured.

In the present disclosure, the medically-defined rules associated with each critical dimension to be measured may be pre-established in computing device 10 and stored in a memory device for recall in connection with the measurement of the relevant dimension. The medical definition rules can be formulated according to the actual medical definition of each critical dimension to be measured under the guidance of professionals, so that the determination mode of each critical dimension to be measured can be correctly reflected. In some embodiments, for each critical dimension to be measured, an edge line associated with the critical dimension to be measured may be determined on the medical image based on a critical point associated with the critical dimension to be measured among the plurality of critical points by invoking a medical definition rule associated with the critical dimension to be measured. Specifically, when measuring a certain critical dimension to be measured, computing device 10 may determine which edges need to be determined by invoking the associated medical definition rules, and thus which keypoints are associated with those edges. Thus, the computer can first find the associated keypoints on the medical image picture in an inverse manner to the above, and determine the associated edge accordingly. In the present disclosure, various different methods may be used to determine the edge associated with the critical dimension to be measured, depending on the medically-defined rules associated with each critical dimension to be measured. A method for determining an edge associated with a critical dimension to be measured according to embodiments of the present disclosure is described in further detail below in conjunction with fig. 7-8. It is to be understood, however, that the present disclosure is not so limited and any other method as would occur to one skilled in the art based on the present disclosure may be used to determine the edge of the critical dimension to be measured.

At step 208, computing device 10 calculates a value for the critical dimension to be measured based on the edge.

In the disclosure, for the length of the key line segment to be measured, the length of the key line segment to be measured can be calculated by using a scale, wherein the length of the scale is predetermined. In some embodiments, in the case of the medical image picture in Dicom format, the scale is a scale built into the system. In another implementation, in the case where the system has no built-in scale, the scale may be defined in advance by drawing a line segment of a predetermined length on the medical image picture before calculating the length of the key line segment to be measured. The scale may be represented in the form of a predetermined User Interface (UI). After the scale is determined, the length of the key line segment to be measured can be converted based on the length of the scale and the comparison result by comparing the pixels occupied by the key line segment to be measured with the pixels occupied by the scale. For example, if it is determined that the ratio of the pixels occupied by the key line segment to be measured to the pixels occupied by the scale is 2:1 by comparing the pixels occupied by the key line segment to be measured with the pixels occupied by the scale, the length of the key line segment to be measured can be converted to 20 pixels in the case where the length of the scale is 10 pixels.

In the disclosure, for the key angle to be measured, a right triangle may be drawn by using the determined angles of the edges as vertices, and then a value of the key angle to be measured may be determined by using a trigonometric function.

In the scheme, the key points related to the key sizes to be measured are automatically determined based on the trained key point detection model, and the key sizes are automatically measured based on the key points, so that any user can accurately and quickly measure the key sizes of the body parts without specially learning related professional knowledge.

FIG. 4 shows a flow diagram of a method 400 for training a keypoint detection model, according to an embodiment of the disclosure. The method 400 may be performed by the computing device 10 as shown in FIG. 1, or may be performed at the electronic device 900 shown in FIG. 9. It should be understood that method 400 may also include additional blocks not shown and/or may omit blocks shown, as the scope of the disclosure is not limited in this respect.

At step 402, computing device 10 obtains a sample medical image picture for a body part, the sample medical image picture including a plurality of labeled keypoints associated with a critical dimension of the body part under test. In the present disclosure, the labeled keypoints can be labeled on the sample medical image by a professional healthcare worker (e.g., the professional physician) according to the critical dimensions to be measured.

At step 404, computing device 10 inputs the acquired sample medical image pictures into a keypoint detection model to obtain a plurality of keypoint heat maps.

In the present disclosure, a keypoint heat map is obtained for each of the labeled keypoints on the sample medical image. In some embodiments, to increase the speed of training, the size of the keypoint heat map may be smaller than the size of the sample medical image picture. For example, the size of the acquired sample medical image picture may be 512x512 pixels, while the size of each keypoint heat map may be 128x128 pixels.

At step 406, computing device 10 extracts the location in each keypoint heatmap where the heat value is highest as the hotspot. As previously mentioned, the location of the highest calorific value here may refer to the coordinates of the point with the highest resolution in the keypoint heat map.

At step 408, computing device 10 adjusts neural network parameters of the keypoint detection model based on the coordinates of each hotspot and the coordinates of the corresponding labeled keypoint, respectively. In some embodiments, by having the neural network parameters of the keypoint detection model adjusted, the error between the coordinates of each hotspot and the coordinates of the corresponding labeled keypoint can be made small.

If the size of the key point heat map obtained in step 404 is different from the size of the sample medical image picture, before step 408, the coordinates of the hot spot are transformed and mapped to obtain the coordinates of the hot spot in the coordinate system of the medical image picture, and then the transformed coordinates of the hot spot are compared with the corresponding coordinates of the key point to determine the error between the two, and then the key point detection model is adjusted based on the error.

In some embodiments, the computing device 10 obtains a plurality of sample medical image pictures for the body part and performs

step

404 and 408 on each sample medical image picture in sequence until the error between the coordinates of each hotspot and the coordinates of the corresponding labeled keypoint falls within a predetermined threshold. When the error between the coordinates of each hotspot and the coordinates of the corresponding labeled keypoint falls within a predetermined threshold, it is an indication that the training resulted in a keypoint detection model that met expectations (i.e., met the actual usage requirements).

In some embodiments, the computing device 10 obtains a predetermined number of sample medical image pictures about the body part and performs the

method

404 and 408 on each sample medical image picture in order to obtain a keypoint detection model that is consistent with expectations. In some embodiments, to determine whether the trained keypoint detection model is the expected keypoint detection model, a predetermined number of detected medical image pictures are used to detect the trained keypoint detection model to determine whether the trained keypoint detection model meets the expected requirements. Similar to the sample medical image, the inspection medical image also includes a plurality of labeled key points associated with the critical dimension of the body part to be inspected. In the present disclosure, if the error between the coordinates of the hot spot in each keypoint heat map obtained based on detecting the medical image picture and the coordinates of the corresponding labeled keypoint is less than a predetermined threshold, it indicates that the trained keypoint detection model meets the expected requirements, and belongs to the expected trained keypoint detection model. If the trained keypoint detection model is detected to be not in accordance with the expected requirements, the sample medical image picture continues to be trained by using the additional sample medical image picture until the expected trained keypoint detection model is obtained. In the present disclosure, the number of detection medical image pictures used for detecting the passed keypoint detection model may be smaller than the number of sample medical image pictures used for training the sample medical image pictures. For example only, the number of sample medical image pictures used for training the sample medical image pictures may be 1000, while the number of detection medical image pictures used for detecting the passed keypoint detection model may be 300.

As mentioned above, the keypoint detection model is a neural network model formed by connecting a plurality of multi-resolution subnetworks in parallel, and the required model can be learned quickly by adjusting parameters of the neural network. After the keypoint detection model is trained, the keypoint detection model can be used to determine the position of the keypoint.

By adopting the above means, the key point detection model meeting the actual use requirements can be trained quickly by the method, and then a plurality of key points associated with the body part can be determined accurately and quickly based on the key point detection model.

A method 500 for determining a plurality of keypoints associated with a measured critical dimension of a body part is described below in conjunction with fig. 5 and 6. Fig. 5 shows a flow diagram of a method 500 for determining a plurality of keypoints associated with a measured critical dimension of a body part, in accordance with an embodiment of the present disclosure. FIG. 6 shows a schematic diagram of a method of determining keypoints based on a keypoint detection model, according to an embodiment of the disclosure. The method 500 may be performed by the computing device 10 as shown in FIG. 1, or may be performed at the electronic device 900 shown in FIG. 9. It should be understood that method 500 may also include additional blocks not shown and/or may omit blocks shown, as the scope of the disclosure is not limited in this respect.

At step 502, computing device 10 inputs medical image pictures into the trained keypoint detection model to obtain a plurality of keypoint heat maps.

In some embodiments, the size of the keypoint heat map is smaller than the size of the input medical image picture. For example, the size of the input medical image picture may be 512x512 pixels, and the size of each keypoint heat map may be 128x128 pixels.

As shown in fig. 6, after the medical image of the hip is inputted into the keypoint detection model associated therewith, the keypoint detection model outputs a plurality of keypoint heat maps associated with the hip.

At step 504, computing device 10 extracts the location in each keypoint heatmap where the heat value is highest as the hotspot. For example, in FIG. 6, the hotspot in each keypoint heat map is the point in the keypoint heat map with the highest resolution.

At step 506, computing device 10 determines coordinates of the corresponding keypoints based on the coordinates of each hotspot, respectively.

If the size of the key point heat map is different from the size of the input medical image picture, before performing step 506, the computing device 10 further needs to perform transformation mapping on the coordinates of the hot spot to obtain the coordinates of the hot spot in the coordinate system of the medical image picture, and then determine the transformed coordinates of the hot spot as the coordinates of the corresponding key point.

By employing the above means, the present disclosure is able to automatically and accurately determine a plurality of key points associated with a body part without requiring the user to possess any expertise.

Fig. 7 shows a flowchart of a method 700 for determining an edge associated with a key angle to be measured, in accordance with an embodiment of the present disclosure. Method 700 may be performed by computing device 10 as shown in FIG. 1, or may be performed at electronic device 900 as shown in FIG. 9. It should be understood that method 700 may also include additional blocks not shown and/or may omit blocks shown, as the scope of the present disclosure is not limited in this respect.

At step 702, computing device 10 connects four keypoints of the plurality of keypoints associated with the keypoint angle to be measured into a first line segment and a second line segment. As mentioned above, the four keypoints associated with the key angles to be measured can be determined by invoking the medically defined rules associated with the key angles to be measured.

Although the first line segment and the second line segment are obtained based on the keypoints determined by the keypoint detection model in step 702, such line segments may also be obtained by a user through a touch operation in actual use. For example, a first touch point of the user during the touch operation may be used as a starting point of the first line segment, and a touch point when the user's finger leaves the screen may be used as an ending point of the first line segment. The second line segment may be determined in the same manner.

At step 704, computing device 10 determines whether the first line segment and the second line segment intersect. In some embodiments, whether the first line segment and the second line segment intersect may be determined according to whether the first line segment and the second line segment have the same coordinate point.

In step 706, in response to the computing device 10 determining that the first line segment and the second line segment intersect, the computing device 10 determines that the first line segment and the second line segment are side lines associated with a key angle to be measured, where the key angle to be measured is an acute angle or a right angle included angle between the first line segment and the second line segment, that is, an included angle in a range of 0 ° to 90 °.

At step 708, in response to computing device 10 determining that the first line segment and the second line segment do not intersect, computing device 10 translates the second line segment such that the feature point of the second line segment coincides with the feature point of the first line segment, and determines that the first line segment and the translated second line segment are edges associated with a key angle to be measured, when the key angle to be measured is an acute or right angle between the first line segment and the translated second line segment.

In some embodiments, the feature point of the second line segment may be a midpoint of the second line segment, and the feature point of the first line segment may be a midpoint of the first line segment. Of course, the above operations may be implemented by taking other feature points of the first line segment and the second line segment according to actual situations, for example, one end point of the second line segment may be taken as its feature point, and one end point of the first line segment may be taken as its feature point, or a midpoint of the second line segment may be taken as its feature point, and one end point of the first line segment may be taken as its feature point.

In this case, the first line segment and the second line segment may be drawn on the medical image picture in a solid line manner, and the translated second line segment may be drawn on the medical image picture in a dashed line manner.

In addition, in the present disclosure, the endpoints of the first line segment and the second line segment may be moved through a touch operation or the first line segment and the second line segment may be translated integrally according to the actual use requirement.

By adopting the means, the method and the device can automatically identify four key points related to the key angle to be detected, automatically determine two line segments related to the key angle to be detected by utilizing the four key points, and further accurately determine the side line of the key angle to be detected based on the two line segments, thereby being beneficial to quickly and accurately determining the value of the key angle to be detected.

Fig. 8 shows a flow diagram of a method 800 for determining an edge associated with a key angle to be measured, in accordance with an embodiment of the present disclosure. Method 800 may be performed by computing device 10 as shown in FIG. 1, or may be performed at electronic device 900 as shown in FIG. 9. It should be understood that method 800 may also include additional blocks not shown and/or may omit blocks shown, as the scope of the present disclosure is not limited in this respect.

At step 802, computing device 10 determines a first reference line associated with a key angle to be measured based on a first and second keypoints of the plurality of keypoints associated with the key angle to be measured. In some embodiments, the first keypoint and the second keypoint may be connected into a third reference line, and a line perpendicular to the third reference line may be drawn on the medical image picture as the first reference line. Of course, the first reference line may also be another line (for example, a connection line between the first key point and the second key point, etc.) according to the key angle to be measured, which needs to be measured actually, and may be specifically determined by the computing device 10 based on the medical definition rule associated with the key angle to be measured.

In step 804, computing device 10 determines a second reference line associated with the key angle to be measured based on a third key point and a fourth key point associated with the key angle to be measured in the plurality of key points, and the key angle to be measured is an acute angle or a right angle included angle between the first reference line and the second reference line. For example, in some embodiments, a straight line passing through the third keypoint and the fourth keypoint is drawn as the second reference line. Of course, the second reference line may also be another line according to the critical angle to be measured that needs to be measured actually, and may be specifically determined by the computing device 10 based on the medical definition rule associated with the critical angle to be measured.

For example, for a hip, the CE angle typically needs to be measured, by way of example only. When measuring the CE angle, the CE angle is known to be the angle between the perpendicular line passing through the center point of the femoral head and the outer edge of the acetabulum by calling the medically defined rule associated with the CE angle. From this, it can be determined that a reference line for the line connecting the hip outer rim point, the femoral head center point and the tear drop point and perpendicular to the tear drop points on both sides, and the line connecting the acetabular outer rim point and the femoral head center point are associated with this CE angle. Thus, by using the method shown in fig. 8, the CE angle can be measured by selecting the acetabular outer edge point v1, the femoral head center v4 and the bilateral tear drop point v6, and drawing the connecting line of v1 and v4 and the vertical reference line passing through v4 in the related medical image picture.

Of course, in order to facilitate users with certain professional knowledge, the user can determine the key angle to be measured in a touch manner in the disclosure. In some embodiments, a broken line can be continuously drawn through touch operation, and the broken point is an angle vertex, so as to generate an angle value ranging from 0 ° to 180 °. Specifically, in the touch process, a first touch point is taken as a starting point (point a) of an angle, a point c is generated at the position where a finger is located in the moving process as a point b, and a predetermined number of pixels (for example, 3 pixels) are moved on a screen, and at the same time, a judgment is performed once, and if an angle formed by lines ab and bc reaches a predetermined threshold range (for example, a threshold range of 15 ° to 170 °), the point b is selected as a vertex of an angle to be measured, and the lines ab and bc are two side lines of the angle to be measured. If the angle reaches the range, c is selected as the vertex of the angle to be measured, and the lines ac and cd are two side lines of the angle to be measured. After the angle vertex is generated, the point when the finger finally leaves the screen is the end point of the angle. After the delineation is completed, three points of the angle support the movement operation.

By adopting the means, the method and the device can automatically identify four key points related to the key angle to be detected, automatically determine two reference lines related to the key angle to be detected by utilizing the four key points, and further accurately determine the side line of the key angle to be detected by utilizing the two reference lines, so that the value of the key angle to be detected can be rapidly and accurately determined.

In some embodiments, for the length of the key line segment to be measured, the associated key line segment to be measured can be obtained by directly connecting two key points associated with each key line segment to be measured in the plurality of key points. As mentioned above, the two keypoints associated with the key-line segment under test are determined by invoking a medical definition rule associated with the key-line segment length under test.

In the present disclosure, the end points of the edges may be adjusted automatically by the system or manually by the user, and the positions of the edges may also be adjusted in translation automatically by the system or manually by the user.

By adopting the means, the method and the device can automatically identify two key points related to the length of the key line segment to be detected, and automatically identify the key line segment to be detected by utilizing the two key points, so that the method and the device are favorable for quickly and accurately determining the length value of the key line segment to be detected.

Fig. 9 illustrates a block diagram of an electronic device 900 suitable for implementing embodiments of the present invention. Electronic device 900 in the present disclosure may be, for example, computing device 10 or server 20 as described above.

As shown in fig. 9, electronic device 900 may include one or more Central Processing Units (CPUs) 910 (only one shown schematically) that may perform various suitable actions and processes in accordance with computer program instructions stored in a Read Only Memory (ROM)920 or loaded from a storage unit 980 into a Random Access Memory (RAM) 930. In the RAM 930, various programs and data required for the operation of the electronic device 900 may also be stored. The CPU 910, ROM 920, and RAM 930 are connected to each other via a bus 940. An input/output (I/O) interface 950 is also connected to bus 940.

Various components in electronic device 900 are connected to I/O interface 950, including: an input unit 960 such as a keyboard, a mouse, etc.; an output unit 970 such as various types of displays, speakers, and the like; a storage unit 980 such as a magnetic disk, optical disk, or the like; and a communication unit 990 such as a network card, a modem, a wireless communication transceiver, or the like. The communication unit 990 allows the electronic device 900 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.

The

methods

200, 400, 500, 700, 800 described above may be performed, for example, by the CPU 910 of the electronic device 900 (e.g., computing device 10 or server 20). For example, in some embodiments, the

methods

200, 400, 500, 700, 800 may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 980. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 900 via the ROM 920 and/or the communication unit 990. When loaded into RAM 930 and executed by CPU 910, may perform one or more of the operations of

methods

200, 400, 500, 700, 800 described above. Further, the communication unit 990 may support wired or wireless communication functions.

Those skilled in the art will appreciate that the electronic device 900 shown in fig. 9 is merely illustrative. In some embodiments, computing device 10 or server 20 may contain more or fewer components than electronic device 900.

Methods

200, 400, 500, 700, 800 for measuring a critical dimension of a body part and electronic device 900 that may be used as computing device 10 or server 20 in accordance with the present invention are described above with reference to the figures. However, it will be appreciated by those skilled in the art that the performance of the steps of the

methods

200, 400, 500, 700, 800 is not limited to the order shown in the figures and described above, but may be performed in any other reasonable order. Further, the electronic device 900 does not necessarily include all of the components shown in fig. 9, it may include only some of the components necessary to perform the functions described in the present invention, and the manner of connecting the components is not limited to the form shown in the drawings.

The present invention may be methods, apparatus, systems and/or computer program products. The computer program product may include a computer-readable storage medium having computer-readable program instructions embodied therein for carrying out aspects of the present invention.

In one or more exemplary designs, the functions described herein may be implemented in hardware, software, firmware, or any combination thereof. For example, if implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium.

The units of the apparatus disclosed herein may be implemented using discrete hardware components, or may be integrally implemented on a single hardware component, such as a processor. For example, the various illustrative logical blocks, modules, and circuits described in connection with the invention may be implemented or performed with a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein.

Those of skill would further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both.

The previous description of the invention is provided to enable any person skilled in the art to make or use the invention. Various modifications to the disclosure will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other variations without departing from the spirit or scope of the disclosure. Thus, the present invention is not intended to be limited to the examples and designs described herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for measuring a critical dimension of a body part, the method comprising:

acquiring a medical image picture of the body part;

determining a plurality of key points associated with a key size to be measured of the body part on the medical image picture based on a trained key point detection model associated with the body part, wherein the key size to be measured at least comprises at least one of a key angle to be measured and a key line length to be measured;

for each key size to be measured, determining an edge line associated with the key size to be measured on the medical image picture based on a key point associated with the key size to be measured in the plurality of key points; and

and calculating the value of the critical dimension to be measured based on the sideline.

2. The method of claim 1, wherein determining a plurality of keypoints associated with a measured critical dimension of the body part on the medical image picture based on a trained keypoint detection model associated with the body part comprises:

inputting the medical image pictures into the trained keypoint detection model to obtain a plurality of keypoint heat maps;

extracting the position with the highest heat value in each key point heat map as a hot point; and

and determining the coordinates of the corresponding key points respectively based on the coordinates of each hot point.

3. The method of claim 1, wherein for the key angle to be measured, determining an edge associated with the key size to be measured on the medical image picture based on a key point associated with the key size to be measured from the plurality of key points comprises:

connecting four key points in the plurality of key points, which are associated with the key angle to be detected, into a first line segment and a second line segment;

determining whether the first line segment and the second line segment intersect;

in response to determining that the first line segment and the second line segment intersect, determining that the first line segment and the second line segment are side lines associated with the key angle to be measured, wherein the key angle to be measured is an acute angle or a right angle included angle between the first line segment and the second line segment; and

in response to determining that the first line segment and the second line segment do not intersect, translating the second line segment such that feature points of the second line segment coincide with feature points of the first line segment, and determining that the first line segment and the translated second line segment are edges associated with the key angle to be measured, wherein the key angle to be measured is an acute or right angle between the first line segment and the translated second line segment.

4. The method of claim 3, wherein the feature point of the second line segment is a midpoint of the second line segment and the feature point of the first line segment is a midpoint of the first line segment.

5. The method of claim 1, wherein for the key angle to be measured, determining an edge associated with the key size to be measured on the medical image picture based on a key point associated with the key size to be measured from the plurality of key points comprises:

determining a first reference line associated with the key angle to be measured based on a first key point and a second key point associated with the key angle to be measured in the plurality of key points;

determining a second reference line associated with the key angle to be measured based on a third key point and a fourth key point associated with the key angle to be measured in the plurality of key points,

and the key angle to be measured is an acute angle or a right-angle included angle between the first reference line and the second reference line.

6. The method of claim 5, wherein determining a first reference line associated with the key angle under test based on a first and a second of the plurality of key points associated with the key angle under test comprises:

connecting the first key point and the second key point into a third reference line; and

drawing a line perpendicular to the third reference line on the medical image picture as the first reference line.

7. The method of claim 5, wherein determining a second reference line associated with the key angle under test based on a third and a fourth of the plurality of key points associated with the key angle under test comprises:

drawing a straight line passing through the third key point and the fourth key point as the second reference line.

8. The method of claim 1, wherein the critical segment length to be measured, determining an edge on the medical image picture for association with the critical dimension to be measured based on a critical point of the plurality of critical points associated with the critical dimension to be measured comprises:

connecting two key points associated with each key line segment to be tested in the plurality of key points so as to obtain the key line segment to be tested;

and wherein calculating the value of the critical dimension to be measured based on the edge comprises: and calculating the length of the key line segment to be measured by using a scale, wherein the length of the scale is predetermined.

9. The method of claim 8, wherein calculating the length of the key line segment to be measured using a scale comprises:

and comparing the pixels occupied by the key line segment to be measured with the pixels occupied by the scale so as to convert the length of the key line segment to be measured based on the length of the scale and the comparison result.

10. The method of claim 1, wherein the keypoint detection model is trained via:

obtaining sample medical image pictures of the body part, wherein the sample medical image pictures all comprise a plurality of labeled key points associated with the key dimension to be measured;

inputting the sample medical image picture into the key point detection model to obtain a plurality of key point heat maps;

extracting the position with the highest heat value in each key point heat map as a hot point;

and adjusting the neural network parameters of the key point detection model based on the coordinates of each hot point and the corresponding marked key point.

11. A computing device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor;

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-10.

12. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-10.