TWM616670U - A system of assisted interpretation of bone medical images with artificial intelligence - Google Patents

Abstract

The invention discloses a system of assisted interpretation of bone medical images with artificial intelligence that includes an input unit, a processing unit and an output unit. The input unit receives a medical image, where the medical image is related to a bone. The processing unit is connected to the input unit. The processing unit executes a deep learning algorithm to calculate the medical image, and extracts a feature point from the medical image to establish a distribution state, and the deep learning algorithm selects one or more types from the plural types according to the distribution state, and another processing unit performs a hotspot image algorithm to calculate the medical image to mark a designated part of the bone in the medical image. The output unit is connected to the processing unit. The output unit outputs the medical image of the designated part of the marked bone and the selected type or the like.

Description

Artificial intelligence assisted bone medical image interpretation system

This creation is a technical field of intelligent medical treatment, especially an artificial intelligence-assisted bone medical image interpretation system for bone-assisted image recognition.

Traditionally, doctors need to check the condition of bones by imaging such as X-ray or MRI. However, it is difficult to distinguish minor damage on bones from X-ray or MRI images. This is common to orthopedics and hand surgeons. Pain points.

For example, the triangular fibrocartilage complex (TFCC) of the wrist is a structure between the distal ulna, the distal radius and the ulnar metacarpal bone. An important role of the distal radioulnar joint. Triangular fibrocartilage complex injuries of the wrist are more likely to occur in pronation and hyperextension of the wrist posture, and may also be combined with distal radius fractures. The more common ball sports injuries include tennis and badminton.

At present, the diagnosis of triangular fibrocartilage complex damage is mainly based on medical imaging. For example, X-ray can see the ulnar variance (ulnar variance, "positive value" ulnar variance, which may represent the compression of the space where TFCC is located, which can be used as a diagnosis of triangular fibrocartilage complex). Reference for body lesions, in addition to observing whether there is cartilage softening or arthritis in the surrounding metacarpal bones); and, Magnetic Resonance Imaging (Magnetic Resonance Imaging) mainly observes whether the triangular fibrocartilage complex has rupture at the ulnar or radial attachment point .

Due to the complex structure of the triangular fibrocartilage complex and the many potential diseases, sometimes the MRI cannot determine whether it is damaged. Some existing research papers have been published on how to use images to improve the clinical interpretation of MRI damage, which is enough to prove that the difficulty in interpreting TFCC damage is a common problem for orthopedic physicians.

In view of this, this creation proposes an artificial intelligence-assisted bone medical image interpretation method and its system to solve the lack of conventional technology.

The first purpose of this creation is to provide an artificial intelligence-assisted bone medical image interpretation method, using artificial intelligence image recognition technology to assist doctors in marking the location and severity of subtle bone injuries.

The second purpose of this creation is to predict and classify the triangular fibrocartilage complex damage of the wrist based on the above-mentioned artificial intelligence-assisted bone medical image interpretation method.

The third purpose of this creation is to predict the age of bones based on the aforementioned artificial intelligence-assisted bone medical image interpretation method.

The fourth purpose of this creation is based on the aforementioned artificial intelligence-assisted bone medical image interpretation method, which can use existing medical images and the determined results as medical sample images to train deep learning algorithms.

The fifth purpose of this creation is based on the above-mentioned artificial intelligence-assisted bone medical image interpretation method, and the deep learning algorithm is used to create types to distinguish the types of bones into normal and abnormal types, and the abnormal parts can also be imported For example, Palmer classification.

The sixth purpose of this creation is to provide a review confirmation form containing the image number and the authenticity/false confirmation column based on the above artificial intelligence-assisted bone medical image interpretation method, so that the doctor can record the corresponding image number in the review confirmation form according to the image number. The results of the review of the sample images to achieve the purpose of optimizing the deep learning algorithm.

The seventh purpose of this creation is to provide an artificial intelligence-assisted bone medical image interpretation system, which is used to realize artificial intelligence-assisted bone medical image interpretation methods.

In order to achieve the above and other purposes, this creation provides an artificial intelligence-assisted bone medical image interpretation method. The artificial intelligence-assisted bone medical image interpretation method includes step S1, receiving a medical image, wherein the medical image is related to a bone; step S2, executing a deep learning algorithm (deep learn algorithm) to calculate the medical image to extract the medical image Take a feature point to establish a distribution state; Step S3 is to execute the deep learning algorithm to select one or more types from the complex number type according to the distribution state; Step S4 is to use a hotspot image algorithm to calculate The medical image is used to mark a designated part of the bone in the medical image; and step S5 is to display the marked medical image of the designated part and its corresponding type to provide auxiliary diagnostic information.

In order to achieve the above and other objectives, this creation provides an artificial intelligence-assisted bone medical image interpretation system, which includes an input unit, a processing unit and an output unit. Input unit connection Receive a medical image, where the medical image is related to a bone. The processing unit is connected to the input unit. The processing unit executes a deep learning algorithm to calculate the medical image, and extracts a feature point from the medical image to establish a distribution state, and the deep learning algorithm selects one or more types from the plural types according to the distribution state, and another processing unit Perform a hotspot image algorithm to calculate the medical image to mark a designated part of the bone in the medical image. The output unit is connected to the processing unit. The output unit outputs the medical image of the designated part of the marked bone and the selected type or the like.

Compared with the conventional technology, the artificial intelligence-assisted bone medical image interpretation method and system provided by this creation can perform image judgment, inspection classification, and injury lesion labeling functions for minor injuries on the bones to assist doctors in medical diagnosis. ; In addition, doctors can also give feedback on the content of this creation to enhance the ability of artificial intelligence to judge.

S11-S15: Method steps

S31-S37: Method steps

2: bones

22: Designated part

10, 10’: Artificial intelligence assisted bone medical image interpretation system

12: Input unit

14: Processing unit

16: output unit

18: Feedback module

MIMG, MIMG': medical imaging

DLA: Deep Learning Algorithm

CLS: Type

HPA: Hotspot image algorithm

FIG. 1 is a schematic flowchart of the artificial intelligence-assisted bone medical image interpretation method of the first embodiment of the present creation.

Figure 2(a) is an X-ray schematic diagram illustrating the bones of Figure 1 of this creation.

Figure 2(b) illustrates the marked hand bone X-ray medical image of Figure 3 of this creation.

Figure 3 is a schematic diagram illustrating the process of training the deep learning algorithm of this creation.

Fig. 4 is a schematic diagram illustrating the structure of training the deep learning algorithm of Fig. 3 of the present creation.

Fig. 5 is a schematic diagram illustrating the distribution of the probability distribution in each type of the creation of Fig. 1.

Fig. 6 is a block diagram of the artificial intelligence-assisted bone medical image interpretation system of the second embodiment of the present creation.

FIG. 7 is a block diagram of the artificial intelligence assisted bone medical image interpretation system of the third embodiment of the present creation.

In order to fully understand the purpose, features, and effects of this creation, the following specific embodiments are used in conjunction with the accompanying drawings to give a detailed description of this creation, which will be described later.

In this creation, "one" or "one" is used to describe the units, components, and components described herein. This is just for the convenience of explanation and to provide a general meaning to the scope of this creation. Therefore, unless it is clearly stated otherwise, this description should be understood to include one or at least one, and the singular number also includes the plural number.

In this creation, the terms "include", "include", "have", "include" or any other similar terms are intended to cover non-exclusive inclusions. For example, an element, structure, product, or device that contains a plurality of elements is not limited to the elements listed herein, but may include those that are not explicitly listed but are generally inherent to the element, structure, product, or device. Other requirements. In addition, unless there is a clear statement to the contrary, the term "or" refers to the inclusive "or" rather than the exclusive "or".

In this creation, the description of the steps is not limited by the order of the preceding and following descriptions. The steps disclosed according to this creation are mainly to facilitate the description of the embodiments. In other embodiments, the smooth execution of the steps can be adjusted or changed.

Please refer to FIG. 1, which is a schematic flowchart of the artificial intelligence-assisted bone medical image interpretation method according to the first embodiment of the present creation. In FIG. 1, the artificial intelligence-assisted bone medical image interpretation method starts at step S11, which is to receive a medical image, for example, the medical image is related to a bone, such as hand bones, skulls, foot bones, hip bones, etc. In addition, medical images can be generated from X-ray machines, X-ray tomography (Computed Tomography), Magnetic Resonance Imaging (Magnetic Resonance Imaging) or from a medical image database. For example, in this embodiment, the bone is taken as an example of the hand bone, and the X-ray medical image of the hand bone in FIG. 2(a) can be referred to. In Figure 2(a), an X-ray schematic diagram of the bone in Figure 1 of this creation is illustrated.

Before performing step S11, the step of training the deep learning algorithm can be performed first, and you can also refer to FIG. 3, which is a schematic diagram illustrating the process of training the deep learning algorithm of this creation. In Fig. 3, the step of training the deep learning algorithm starts at S31, which is to provide plural medical sample images. Among them, the medical sample images are related to bones, which correspond to the image types corresponding to step S11.

Step S32 is to classify the medical sample images to classify the medical sample images into plural types. Refer also to FIG. 4, which is a schematic diagram illustrating the structure of training the deep learning algorithm in FIG. 3 of the present creation. In Figure 4, these types are defined as abnormal types 1A, 1B, 1C and 1D and normal types.

In step S33, a deep learning algorithm is used to calculate the medical sample images corresponding to each of these types, and feature points are extracted from each of the medical sample images of these types.

Step S34 is to record the corresponding distribution states of the feature points, so that the abnormal types Type 1A, Type 1B, Type 1C, Type 1D and the normal types have their corresponding distribution states. Among them, in this embodiment, the number of abnormal types is illustrated by taking 4 as an example. In other embodiments, the number of abnormal types may be less than 4 or more than 4, and is not limited to 4. kind. In yet another embodiment, palmer classification can be used for abnormal types. Palmer classification can be further divided into multiple types such as traumatic injury and degenerative injury.

Step S35 is to mark the designated parts of the bones on the medical sample images. For example, the designated part may be the triangular fibrocartilage complex of the wrist. Also refer to the marked hand bone X-ray medical image in Figure 2(b). Figure 2(b) is an X-ray diagram illustrating the marked bone in Figure 2(a) of this creation. In Figure 2(b), the triangular fibrocartilage of the wrist is calculated by, for example, a hotspot image algorithm, and the triangular fibrocartilage can be marked in a frame on the X-ray medical image of the hand bone. Among them, the hot image The algorithm can be performed by using gradient-based sensitivity analysis and other interpretable AI algorithms.

It is worth noting that, in another embodiment, the deep learning algorithm can further import impact factors for calculation. For example, the impact factors can be age, gender, BMI, disease history, and surgical history. The image factor is associated with the medical image. Irradiated.

Returning to FIG. 3, step S36 is to randomly select a predetermined number of the medical sample images for review by the doctor to generate a review result of the medical sample images. Among them, the medical sample images are marked with designated parts. Referring to Figure 4, the image numbers of the medical sample images and their corresponding true (T)/false (F) confirmation columns are displayed for the review results, so that the doctor can confirm that the medical sample images are calculated by the deep learning algorithm. Is it trustworthy? For example, in this embodiment, if the medical sample image is calculated by the deep learning algorithm to show the correct type, the doctor selects the medical sample image to be classified as true (T) in the true (T)/false (F) confirmation column ; On the contrary, the doctor selects the medical sample image to be classified as false (F) in the true (T)/false (F) confirmation column. Also in this embodiment, the predetermined number of the medical sample images can be set to at least 100 and there are at least 50 normal types of the medical sample images in the predetermined number.

In another embodiment, training the deep learning algorithm may further include step S37, which is to optimize the deep learning algorithm using the review result.

Returning to Fig. 1, following step S12, a deep learning algorithm is executed to calculate the medical image to extract feature points from the medical image to establish a distribution state. The feature points are calculated by the deep learning algorithm to display the probability distribution that matches the type in the distribution state. The deep learning algorithm is a convolutional neural network. After the deep learning algorithm calculates the medical image, it is predicted that the medical image belongs to one or more of these types.

In step S13, a deep learning algorithm is executed to select one or more types from the plural types according to the distribution state, that is, to classify the medical image into one or more types. Also refer to Figure 5, which shows the probability distribution of the feature points of a certain medical image after the deep learning algorithm. Taking Figure 5 as an example, the probability distribution values are distributed in various types of states, but the values in Type 1C and Type 1D are higher, so the deep learning algorithm predicts that the medical image is of Type 1C. Or it may be Type 1C and Type 1D. In other words, in one embodiment, it can be set that when the value of the probability distribution exceeds a predetermined threshold, the medical image corresponding to the feature point is classified into one or more of these types.

Returning to Fig. 1, step S14 is to use the hotspot image algorithm to calculate the medical image to mark a designated part of the bone in the medical image, as shown in Fig. 2(b).

Step S15 is to display the marked medical image of the designated part and its corresponding type, so as to provide auxiliary diagnosis information.

Please refer to FIG. 6, which is a block diagram of the artificial intelligence assisted bone medical image interpretation system according to the second embodiment of the present creation. In FIG. 6, the artificial intelligence-assisted bone medical image interpretation system 10 includes an input unit 12, a processing unit 14 and an output unit 16.

The input unit 12 receives a medical image MIMG. Among them, the medical image MIMG is related to a bone 2.

The processing unit 14 is connected to the input unit 12. The processing unit 14 executes a deep learning algorithm DLA to calculate the medical image MIMG, and extracts a feature point from the medical image MIMG to establish a distribution state DS, and the deep learning algorithm DLA selects one from the complex type CLS according to the distribution state DS For multiple types of CLS, the processing unit 14 executes a hotspot image algorithm HPA to calculate a medical image MIMG to mark a designated part 22 of the bone 2 in the medical image MIMG.

The output unit 16 is connected to the processing unit 14. The output unit 16 outputs the marked medical image MIMG' of the designated part 22 of the bone 2 and the selected type CLS or equivalent type CLS.

Please refer to FIG. 7, which is a block diagram of the artificial intelligence-assisted bone medical image interpretation system according to the third embodiment of the present creation. In FIG. 7, the artificial intelligence-assisted bone medical image interpretation system 10' includes the input unit 12, the processing unit 14 and the output unit 16 of the second embodiment, and further includes a feedback module 18.

The input unit 12, the processing unit 14 and the output unit 16 are as described above and will not be repeated here.

The feedback module 18 is connected to the processing unit 14. The feedback module 18 transmits the medical image of the designated part of the marked bone and/or the selected type or feedback information of these types to optimize the deep learning algorithm.

It is worth noting that in the foregoing embodiment, in addition to the implementation of doctors and medical images in the same medical field, for example, they can also be implemented across medical fields through the cloud. For example, doctors from all over the world can use the results of their feedback. To strengthen and optimize deep learning algorithms.

This creation has been disclosed in the preferred embodiments above, but those familiar with this technology should understand that the embodiments are only used to describe the creation, and should not be interpreted as limiting the scope of the creation. It should be noted that all changes and replacements equivalent to the embodiments should be included in the scope of this creation. Therefore, the scope of protection of this creation shall be subject to the scope of the patent application.

2: bones

22: Designated part

10: Artificial intelligence assisted bone medical imaging interpretation system

12: Input unit

14: Processing unit

16: output unit

MIMG, MIMG': medical imaging

DLA: Deep Learning Algorithm

CLS: Type

HPA: Hotspot image algorithm

Claims (16)

An artificial intelligence-assisted bone medical image interpretation system includes: an input unit for receiving medical images, wherein the medical image is related to bone; a processing unit connected to the input unit, the processing unit executes a deep learning algorithm to calculate the medical Image, and feature points are extracted from the medical image to establish a distribution state, and the deep learning algorithm selects one or more types from a plurality of types according to the distribution state, and the processing unit executes a hotspot image algorithm to calculate the medical image , To mark the designated part of the bone in the medical image; and an output unit connected to the processing unit, the output unit outputting the marked medical image of the designated part of the bone and the selected type or types . The artificial intelligence-assisted bone medical image interpretation system according to claim 1, further comprising a feedback module connected to the processing unit, and the feedback module transmits the marked medical image of the designated part of the bone and the selected The feedback information of this type or at least one of these types is used to optimize the deep learning algorithm. The artificial intelligence-assisted bone medical image interpretation system according to claim 1, wherein the medical image is generated from at least one of an X-ray machine, an X-ray tomography (Computed Tomography), and a Magnetic Resonance Imaging (Magnetic Resonance Imaging). The artificial intelligence assisted bone medical image interpretation system according to claim 3, wherein the designated part is the triangular fibrocartilage complex of the wrist. The artificial intelligence-assisted bone medical image interpretation system according to claim 1, wherein the distribution state is formed by these types, and the deep learning algorithm calculates the feature points to be The distribution state shows that the probability distribution conforms to these types, and the medical image corresponding to the feature point is determined to be classified into one or more types according to the value of the probability distribution. For example, in the artificial intelligence-assisted bone medical image interpretation system of claim 5, wherein the value of the probability distribution exceeds a predetermined threshold, the medical image corresponding to the feature point is classified into one or more of these types. The artificial intelligence-assisted bone medical image interpretation system according to claim 1, wherein the deep learning algorithm is a convolutional neural network (Convolutional neural network), and after the deep learning algorithm calculates the medical image, predicts the medical image Images belong to one or more of these types. The artificial intelligence-assisted bone medical image interpretation system according to claim 1, wherein the hotspot image algorithm is gradient-based sensitivity analysis. The artificial intelligence-assisted bone medical image interpretation system according to claim 1, wherein the deep learning algorithm further introduces an impact factor for calculation, wherein the impact factor is at least one of age, gender, BMI, disease history, and surgical history. The artificial intelligence-assisted bone medical image interpretation system according to claim 1, wherein these types adopt Palmer classification, wherein the Palmer classification is divided into traumatic injury and degenerative injury ). The artificial intelligence-assisted bone medical image interpretation system according to claim 1, wherein the processing unit executes the steps of training the deep learning algorithm as follows: S1: provide plural medical sample images, wherein the medical sample images are related to the bone; S2: Classify the medical sample images to classify the medical sample images into these types; S3: Use the deep learning algorithm to calculate the medical sample images corresponding to each of these types, and extract feature points from the medical sample images of each of these types; S4: record the corresponding ones generated by the feature points Distribution status; S5: mark the designated part of the bone in the medical sample images; and S6: randomly select a predetermined number of the medical sample images for review by the doctor to generate a review result of the medical sample images, where The medical sample images are marked with the designated part. The artificial intelligence-assisted bone medical image interpretation system described in claim 11 further includes step S12, using the review result to optimize the deep learning algorithm. The artificial intelligence assisted bone medical image interpretation system according to claim 11, wherein in step S6, the types are classified into normal types and abnormal types, and the number of abnormal types is one type or plural types. The artificial intelligence-assisted bone medical image interpretation system according to claim 11, wherein in step S11, the review result provides the image numbers of the medical sample images and their corresponding authenticity/false confirmation columns. The artificial intelligence-assisted bone medical image interpretation system according to claim 14, wherein in step S11, a review confirmation form including the image number and the authenticity/false confirmation column is provided for the doctor to review according to the image number The confirmation form records the review results of the medical sample images corresponding to the image number. The artificial intelligence-assisted bone medical image interpretation system according to claim 11, wherein the predetermined number of the medical sample images is at least 100 and there are at least 50 of the normal type of the medical sample images in the predetermined number.

Images