KR102363721B1 - Apparatus and method for measuring body size through estimation of body part for incomplete medical image - Google Patents

Abstract

The present invention relates to an apparatus and method for measuring body size by estimating body parts for incomplete medical images. The provided body by estimating one or more medical images, restoring the incomplete medical image based on the estimated medical image, and measuring and providing the size of a body part for the incomplete medical image based on the restored medical image The present invention relates to an apparatus and method for measuring body size by estimating a body part on an incomplete medical image that allows the size to be used for various purposes.

Description

Apparatus and method for measuring body size through estimation of body parts for incomplete medical images

The present invention relates to an apparatus and method for measuring body size by estimating a body part for an incomplete medical image, and more particularly, when an incomplete medical image in which a part of a body part is damaged or covered is input by a user, the incomplete medical image estimating a medical image for an abnormal part from at least one or more medical images matching the normal part, and recovering the incomplete medical image based on the estimated medical image, It relates to an apparatus and method for measuring and providing the size of the body part based on the present invention.

With the recent rapid development of medical technology and medical equipment, it has become possible to acquire high-precision medical images of the patient, making it possible to check the lesion of the patient through reading the medical image without opening the patient's body. became

The medical image includes a computer tomography (CT) image, a magnetic resonance imaging (MRI) image, an X-ray image, an ultrasound image, and the like.

These medical images, after being used for the purpose of reading, lose their useful value, and are stored in a storage or database owned by an individual and left unattended.

Therefore, if the medical image is not only used for the purpose of reading, but a body size of a body part is measured based on the medical image, and the measured body size can be provided to a user of the corresponding medical image, medical treatment (eg: It can be used for a variety of purposes, such as making products tailored to the user's body, such as implants), clothes, chairs, and bicycles.

On the other hand, in the medical image, there may be cases in which a part of the body is hidden by a specific other part, and the case in which the part of the body is damaged or damaged in the medical image stored in the storage or database may occur.

Therefore, in the present invention, when an incomplete medical image including an abnormal part in which a part of a body part is damaged or covered is input, a medical image matching the incomplete medical image is estimated, and the estimated medical image corresponding to the abnormal part is obtained. It is intended to propose a method of recovering the incomplete medical image based on the body part and measuring the body size of the body part of the incomplete medical image, so that the measured body size can be used for various purposes.

Next, the prior art existing in the technical field of the present invention will be briefly described, and then the technical matters that the present invention intends to achieve differently from the prior art will be described.

First, Korean Patent Application Laid-Open No. 2020-0001736 (2020.01.07.) relates to an ultrasound image display apparatus and system and a method for detecting the size of a living tissue using the same, and displays an ultrasound image of the living tissue on a touch screen, The touch screen recognizes a contact point touched by the user, detects one or more edge portions adjacent to the recognized contact point, and the living body according to whether an edge curve for the detected edge portion is an open curve or a closed curve The present invention relates to an ultrasound image display apparatus and system for detecting tissue size information, and a method for detecting the size of a living tissue using the same.

That is, the prior art describes a method of detecting the size of a living tissue that, when a user touches a specific point in an ultrasound image, measures the size of the biological tissue according to a type of an edge curve adjacent to the specific point.

On the other hand, the present invention restores an incomplete medical image in which a part of a body part is damaged or covered, and measures the body part from the restored medical image. It is clear that there is a significant difference between the prior art and the present invention.

In addition, Korea Patent No. 0954989 (2010.04.20.) relates to an ultrasound diagnosis apparatus and method for measuring the size of an object, in which the contour of the object is extracted from the ultrasound image, and the curvature is rapidly changed on the extracted contour. A block of a predetermined size is set by detecting a plurality of corner points, a pointer moving on a display outputting an ultrasound image recognizes and stores coordinates for a plurality of measurement points that intersect the block or the outline, and stores at least one of the measurement points The present invention relates to an ultrasound diagnosis apparatus and method for measuring a size of an object that measures the size of the object based on the coordinates of the selected measurement point when two or more are selected by a user.

That is, in the prior art, when a user selects a contour line or a corner point in an ultrasound image through a pointer on a display, the size of the object is measured and provided by using the coordinates of the selected contour line or corner point. However, in the present invention, when an incomplete medical image including an abnormal portion is input, a medical image matching the incomplete medical image is estimated and the abnormal portion of the incomplete medical image is restored based on the estimated medical image, It is clear that the prior art does not describe, suggest or imply the technical characteristics of the present invention as measuring the size of a body part from a medical image in which an abnormal part is restored.

The present invention was created to solve the above problems, and when an incomplete medical image including a damaged or covered abnormal part of a body part is input from a user, a medical image matching the incomplete medical image is estimated and , by restoring the abnormal part based on the estimated medical image, measuring the body size of the body part from the restored medical image of the abnormal part, and estimating the body part for the incomplete medical image provided to the user An object of the present invention is to provide an apparatus and method for measuring body size.

In addition, the present invention extracts the user's medical image for a body part symmetrical with the body part of the incomplete medical image, extracts an anonymized individual medical image that most closely matches the incomplete medical image, or extracts the incomplete medical image By extracting anonymized individual medical images matching the body parts and symmetrical body parts of the medical images, the medical images matching the incomplete medical images are estimated, and the abnormal parts of the incomplete medical images are extracted based on the estimated medical images. Another object of the present invention is to provide an apparatus and method for measuring body size by estimating body parts for incomplete medical images that can be easily restored.

The present invention also provides a simple recovery of the abnormal part by identifying a characteristic point of a body part corresponding to the abnormal part of the incomplete medical image in the estimated medical image, and projecting the coordinates of the identified characteristic point on the abnormal part. A device for measuring body size through estimating body parts for incomplete medical images, which connects feature points in the medical image from which the abnormal part is restored to accurately measure and provide the body size in the incomplete medical image, and It is another object to provide the method.

An apparatus for measuring body size by estimating a body part for an incomplete medical image according to an embodiment of the present invention is an incomplete medical image including an abnormal part in which some of the body parts are damaged or covered and a normal part other than the abnormal part. A medical image estimator for estimating a matching medical image, the incomplete medical image by projecting the coordinates of at least one feature point identified for the body part of the estimated medical image corresponding to the abnormal part of the incomplete medical image onto the abnormal part A medical image restoration unit that integrates and restores at least one feature point for the normal part and the feature point projected on the abnormal part, and the integrated feature point in the medical image from which the abnormal part is restored, to measure the body size It is characterized in that it includes a body size measurement unit.

In addition, the medical image estimating unit is characterized in that it comprises extracting the user's medical image of the body part symmetrical with the body part of the incomplete medical image, and estimating a medical image matching the incomplete medical image.

In addition, the medical image estimating unit comprises extracting a medical image that most closely matches the corresponding part of a plurality of anonymized individual medical images as a normal part of the incomplete medical image to estimate the medical image of the abnormal part. do.

In addition, the medical image estimating unit is characterized in that it comprises extracting a plurality of anonymized individual medical images matching the body parts and symmetrical body parts of the incomplete medical image to estimate the incomplete medical image.

In addition, the body size measuring device further comprises a feature point learning unit that determines at least one feature point from a plurality of anonymized individual medical images, and generates a learning model for learning coordinates and labels for the determined at least one feature point, The medical image estimation unit selects at least one learning model according to the body part of the incomplete medical image, applies the incomplete medical image to the selected at least one or more learning models, and determines at least one feature point for the normal part. A feature point recognition unit for recognizing, a medical image extraction unit for extracting a medical image of the user symmetrical with the body part of the incomplete medical image, a coordinate transformation unit for converting the extracted medical image of the user into coordinates for a symmetrical body part, and the It characterized in that it further comprises a feature point checking unit for confirming at least one feature point for the body part corresponding to the abnormal part in the medical image of the user converted into coordinates for the symmetrical body part.

In addition, the medical image estimator may include a medical image extraction unit for extracting at least one anonymized individual medical image matching the recognized at least one or more feature points, and at least a body part corresponding to the normal part from the extracted individual medical image. A similarity calculator that extracts one or more feature points and calculates a similarity between the recognized feature points for the normal part and the extracted feature points, and as a result of calculating the similarity, selects a medical image of a specific individual with the highest similarity and selects the It characterized in that it further comprises a feature point checking unit for confirming at least one feature point for the body part corresponding to the abnormal part.

In addition, the medical image estimating unit, a coordinate converting unit that converts the incomplete medical image into coordinates for symmetrical body parts, selects at least one learning model according to the symmetrical body parts, and selects coordinates for the symmetrical body parts A feature point recognition unit that recognizes at least one feature point for the normal part by applying the incomplete medical image converted to A medical image extraction unit for extracting, a similarity calculation unit for extracting at least one feature point corresponding to the normal part from the extracted individual medical image, and calculating the similarity between the recognized feature point for the normal part and the extracted feature point; As a result of calculating the degree of similarity, the method further comprises a feature point checking unit that selects a medical image of a specific individual having the highest similarity and checks at least one feature point for the body part corresponding to the abnormal part.

In addition, the body size measurement method according to an embodiment of the present invention estimates a medical image matching an incomplete medical image including an abnormal part in which some of the body parts are damaged or covered and a normal part other than the abnormal part. step, by projecting the coordinates of at least one or more feature points identified for the body part of the estimated medical image corresponding to the abnormal part of the incomplete medical image to the abnormal part, and at least one feature point for the normal part of the incomplete medical image; A medical image restoration step of integrating and restoring the feature points projected on the abnormal portion, and a body size measurement step of measuring the body size by connecting the integrated feature points in the medical image from which the abnormal portion is restored. do it with

In addition, the estimating of the medical image may include estimating a medical image matching the incomplete medical image by extracting the user's medical image for a body part symmetrical to the body part of the incomplete medical image.

In addition, the medical image estimating step includes estimating the abnormal portion of the medical image by extracting a medical image that most closely matches the corresponding portion of a plurality of anonymized individual medical images as a normal portion of the incomplete medical image. do it with

In addition, the estimating of the medical image may include estimating the incomplete medical image by extracting a plurality of anonymized individual medical images matching the body parts and symmetrical body parts of the incomplete medical image.

In addition, the method for measuring body size further comprises a feature point learning step of determining at least one feature point from a plurality of anonymized individual medical images, and generating a learning model for learning coordinates and labels for the determined at least one feature point, and , the medical image estimating step may include selecting at least one or more learning models according to the body part of the incomplete medical image, and applying the incomplete medical image to the selected at least one or more learning models, and applying at least one or more learning models for the normal part. A key point recognition step of recognizing a key point, a medical image extraction step of extracting a medical image of a user symmetric with the body part of the incomplete medical image, a coordinate transformation step of converting the extracted medical image of the user into coordinates for a symmetrical body part and a feature point checking step of confirming at least one feature point for a body part corresponding to the abnormal part in the medical image of the user converted into coordinates for the symmetrical body part.

In addition, in the medical image estimation step, at least one or more learning models according to the body parts of the incomplete medical image are selected, and the incomplete medical image is applied to the selected at least one or more learning models, and at least one or more of the normal parts are selected. A key point recognition step of recognizing a key point, a medical image extraction step of extracting at least one anonymized individual medical image matching the at least one recognized characteristic point, and a body part corresponding to the normal part from the extracted individual individual medical image A similarity calculation step of extracting at least one feature point and calculating the similarity between the recognized feature point for the normal part and the extracted feature point, and as a result of calculating the similarity, a medical image of a specific individual with the highest similarity is selected. The method further comprises a feature point checking step of confirming at least one feature point for the body part corresponding to the abnormal part.

In addition, the medical image estimation step includes a coordinate transformation step of converting the incomplete medical image into coordinates for a symmetrical body part, selecting at least one learning model according to the symmetrical body part, and selecting the symmetrical body part. A feature point recognition step of recognizing at least one feature point for the normal part by applying the incomplete medical image converted into coordinates to the selected learning model, and at least one anonymized individual medical image matching the recognized at least one feature point A medical image extraction step of extracting a similarity calculation step of extracting at least one feature point corresponding to the normal part from the extracted individual medical image, and calculating the similarity between the recognized feature point for the normal part and the extracted feature point and a feature point checking step of selecting at least one feature point for a body part corresponding to the abnormal part by selecting a medical image of a specific individual having the highest similarity as a result of calculating the similarity.

As described above, according to the apparatus and method for measuring body size through estimation of body parts for incomplete medical images of the present invention, it is an incomplete medical image including an abnormal part that is damaged or hidden among body parts of a medical image input from a user. In any case, the medical image of the user that is symmetrical with the body part of the incomplete medical image or the individual medical image that most closely matches the incomplete medical image, or the incomplete medical image is symmetrically converted to the most symmetrically transformed incomplete medical image. There is an effect that the abnormal portion can be efficiently restored by using closely matched individual medical images.

In addition, the present invention measures the body size of a specific body part in the medical image from which the abnormal part is restored and provides it to the user, so that the user can use the body size for various purposes, such as medical care and customized product production. It works.

1 is a conceptual diagram for explaining an apparatus and method for measuring body size by estimating a body part with respect to an incomplete medical image according to an embodiment of the present invention.
2 is a diagram illustrating a process of recovering an incomplete medical image using a user's medical image symmetrical to a body part of the incomplete medical image according to an embodiment of the present invention.
3 is a diagram illustrating a process of recovering an incomplete medical image using an anonymized individual medical image according to an embodiment of the present invention.
4 is a diagram illustrating a process of recovering an incomplete medical image using an anonymized individual medical image according to another embodiment of the present invention.
5 is a diagram illustrating a process of recognizing a feature point for a normal part from an incomplete medical image through a learning model according to an embodiment of the present invention.
6 is a block diagram illustrating a configuration of an apparatus for measuring body size by estimating a body part with respect to an incomplete medical image according to an embodiment of the present invention.
7 is a flowchart illustrating a procedure for measuring a body size of a body part of an incomplete medical image based on a user's medical image symmetrical to the body part of the incomplete medical image according to an embodiment of the present invention.
8 is a flowchart illustrating a procedure for measuring a body size of a body part of an incomplete medical image based on an anonymized individual medical image identical to the body part of the incomplete medical image according to an embodiment of the present invention.
9 is a flowchart illustrating a procedure for measuring a body size based on an anonymized individual medical image symmetrical to a body part of an incomplete medical image according to an embodiment of the present invention.
10 is a diagram illustrating a hardware structure of an apparatus for measuring body size by estimating a body part with respect to an incomplete medical image according to an embodiment of the present invention.

Hereinafter, a preferred embodiment of an apparatus and method for measuring body size by estimating a body part for an incomplete medical image of the present invention will be described in detail with reference to the accompanying drawings. Like reference numerals in each figure indicate like elements. In addition, specific structural or functional descriptions of the embodiments of the present invention are only exemplified for the purpose of describing the embodiments according to the present invention, and unless otherwise defined, all used herein, including technical or scientific terms, are Terms have the same meanings as commonly understood by those of ordinary skill in the art to which the present invention pertains. Terms such as those defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present specification. It is preferable not to

1 is a conceptual diagram for explaining an apparatus and method for measuring body size by estimating a body part with respect to an incomplete medical image according to an embodiment of the present invention.

As shown in FIG. 1 , a body size measuring apparatus 100 (hereinafter, referred to as a body size measuring device) through body part estimation for an incomplete medical image according to an embodiment of the present invention is a user terminal 200 . When an incomplete medical image in which a part of the body part includes a normal part and an abnormal part is inputted, a medical image matching the incomplete medical image is estimated, and the abnormal part is restored based on the estimated medical image The body size of the corresponding body part is measured from the medical image in which the part is restored and provided to the user. The abnormal part refers to a body part in which a part of the body part is damaged or covered, and the normal part refers to a body part other than the abnormal part.

The incomplete medical image includes various medical images such as a CT image, an MRI image, an ultrasound image, and the like, and includes at least one or more arbitrary feature points designated by the user as a specific part, and an abnormal part and a normal part determined by the user.

In addition, the body size measuring apparatus 100 may include a medical image of a user of a body part symmetrical to the body part of the incomplete medical image, an anonymized medical image of an individual that most closely matches the incomplete medical image, or the incomplete medical image. Extracts an anonymized medical image of an individual matching the body part and the symmetrical body part of the medical image, estimates the medical image matching the incomplete medical image, and restores the incomplete medical image based on the estimated medical image .

In this case, the body size measuring apparatus 100 first performs the estimation using the user's medical image symmetrical to the body part and the symmetrical body part of the incomplete medical image, and then performs the symmetrical user's medical treatment If there is no image, an anonymized individual medical image that most closely matches the incomplete medical image (that is, an anonymized individual medical image of the same body part as the body part of the incomplete medical image) is extracted and the Estimation is performed, and then, when there is only an anonymized medical image of an individual that matches a symmetrical body part with the body part of the incomplete medical image, the estimation is performed using the symmetrical anonymized medical image Preferably, the present invention is not limited thereto, and a medical image matching the incomplete medical image may be estimated through any one of the above-listed estimation processes.

Meanwhile, restoring an abnormal portion of the incomplete medical image by estimating a medical image matching the incomplete medical image is a key technical feature of the present invention and will be described in detail with reference to FIGS. 2 to 4 .

In addition, the body size measuring device 100 provides the measured body size to the user terminal 200 to customize the body size to a product that the user needs, such as a bicycle, chair, clothes, bed, accessories, etc. It allows it to be used for various purposes, such as being used for manufacturing a dental implant or used for medical purposes such as implant (tooth or spine) procedures.

In addition, recognizing at least one feature point for the normal part of the incomplete medical image is generated by determining at least one feature point from a plurality of anonymized individual medical images, and learning the coordinates and labels for the determined at least one feature point. It is performed by applying the incomplete medical image to one learning model. Recognizing the feature point through the learning model will be described with reference to FIG. 5 .

The plurality of anonymized individual medical images are collected according to a preset period from at least one medical image provider 300 such as a wired or wireless communication terminal owned by a doctor or individual, a server operated by a medical institution such as a hospital or oriental clinic, or the like. , are collected in real time and stored in the individual medical image database 420 .

2 is a diagram illustrating a process of recovering an incomplete medical image using a user's medical image symmetrical to a body part of the incomplete medical image according to an embodiment of the present invention.

As shown in FIG. 2 , when an incomplete medical image is restored using a user's medical image symmetrical to a body part of the incomplete medical image according to an embodiment of the present invention, the body size measuring apparatus 100 is a user terminal At least one learning model is selected according to the result of checking the body part of the incomplete medical image input from 200, loaded from the learning model database 410, and the incomplete medical image is applied to the selected at least one learning model. At least one feature point for the normal part is recognized by applying (①). At this time, the body size measuring apparatus 100 provides a user interface to select the body part, the abnormal part, and the normal part of the incomplete medical image, thereby determining the body part, the abnormal part, and the normal part of the incomplete medical image. Check it.

Also, the body size measuring apparatus 100 searches the database 400, extracts a medical image of the user symmetric to the incomplete medical image (②), and estimates a medical image matching the incomplete medical image.

For example, when the body part of the incomplete medical image is a medical image obtained by photographing the knee joint of the left leg, the body size measuring apparatus 100 may display the medical image of the user obtained by photographing the knee joint of the user's right leg. extract In this case, in the extracted medical image of the user, at least one feature point is recognized through the learning model in advance. However, when the feature point is not recognized for the extracted user's medical image, the feature point is recognized using the learning model.

In addition, the body size measuring apparatus 100 converts the extracted user's medical image into coordinates for a body part symmetrical to the body part (③). In the above example, since the extracted medical image of the user is a medical image of the right leg, and the incomplete medical image is a medical image of the left leg, the extracted medical image of the user is symmetrically converted into a medical image of the left leg .

Also, converting into coordinates for symmetrical body parts means performing left-right symmetrical transformation or vertical symmetrical transformation according to the symmetrical body parts.

Meanwhile, in the left-right symmetric transformation, the x-coordinate of the medical image of the transformation target is subtracted by 1 from the horizontal size value of the medical image of the transformation target, and the coordinate value of the x-coordinate is added to the result of the subtraction once. Further subtraction is performed by converting the result into a coordinate value of the x-coordinate. In this case, the y-coordinate of the symmetrically transformed medical image is set as the y-coordinate of the medical image to be transformed.

In addition, in the vertical symmetric transformation, the y-coordinate of the medical image of the transformation target is subtracted by 1 from the vertical size value of the medical image of the transformation target, and the coordinate value of the y-coordinate is added to the result of the subtraction once. Further subtraction is performed by converting the result into a coordinate value of the y-coordinate. In this case, the x-coordinate of the symmetrically transformed medical image is set as the x-coordinate of the medical image to be transformed.

In addition, the body size measuring apparatus 100 checks at least one characteristic point of a body part corresponding to an abnormal part of the incomplete medical image from the symmetrically transformed medical image of the user (④).

In addition, the body size measuring apparatus 100 projects the coordinates of the identified at least one or more feature points to the abnormal portion of the incomplete medical image, thereby projecting the identified at least one or more feature points to the abnormal portion of the incomplete medical image, The abnormal part of the incomplete medical image is restored by integrating the feature points projected on the abnormal part and the feature points recognized for the normal part of the incomplete medical image (⑤).

3 is a diagram illustrating a process of recovering an incomplete medical image using an anonymized individual medical image according to an embodiment of the present invention.

As shown in FIG. 3 , the process of recovering an incomplete medical image (eg, a medical image obtained by photographing the brain) using an anonymized individual medical image according to an embodiment of the present invention includes the body part of the incomplete medical image. It is performed when the same anonymized individual medical image is stored.

At this time, the body size measuring apparatus 100 recognizes at least one feature point of the normal part through at least one learning model according to the body part (eg, brain) of the incomplete medical image (①).

In addition, the body size measuring apparatus 100 searches the anonymized individual medical image database 420 and extracts at least one individual medical image matching the recognized feature points (②).

In this case, since the learning model is generated by learning each anonymized individual medical image in which feature points are determined for each body part, the body size measuring apparatus 100 recognizes the normal part of an incomplete medical image through the learning model. It may be known which individual medical image is matched with at least one or more feature points.

In addition, the body size measuring apparatus 100 extracts feature points for a body part corresponding to a normal part except for an abnormal part of the incomplete medical image from the extracted individual medical image, and recognizes the normal part of the incomplete medical image. A degree of similarity between at least one feature point and the feature point extracted from each individual medical image is calculated (③).

Here, the similarity calculation is performed through a similarity calculation method such as Euclidean Distance, Manhattan Distance, or Minkowski Distance.

In addition, as a result of calculating the degree of similarity, the body size measuring apparatus 100 selects and extracts a medical image of a specific anonymized individual having the highest similarity among the extracted anonymized individual medical images, thereby matching the incomplete medical image. A medical image is estimated, and at least one feature point of a body part corresponding to the abnormal part is identified in the estimated anonymized medical image of a specific individual (④).

In addition, the body size measuring apparatus 100 projects the coordinates of the at least one or more identified feature points on the abnormal part of the incomplete medical image, and recognizes the feature point projected on the abnormal part and the normal part of the incomplete medical image. By integrating one feature point with each other, the abnormal part of the incomplete medical image is restored (⑤).

4 is a diagram illustrating a process of recovering an incomplete medical image using an anonymized individual medical image according to another embodiment of the present invention.

As shown in FIG. 4 , the process of recovering an incomplete medical image using an anonymized individual medical image according to another embodiment of the present invention is an anonymized individual medical image of a body part symmetrical to the incomplete medical image. It is executed when it is stored.

First, the body size measuring apparatus 100 first performs a symmetric transformation on the incomplete medical image (eg, a right arm photographing a right elbow joint), and a body part (ie, the left side) of the symmetrically transformed incomplete medical image. arm), selects at least one learning model, and recognizes at least one feature point for the normal part of the symmetrically transformed incomplete medical image (①).

Meanwhile, the body size measuring apparatus 100 may perform symmetrical transformation on the incomplete medical image after recognizing at least one feature point for the normal portion of the incomplete medical image.

In addition, the body size measuring apparatus 100 extracts, from the anonymized individual medical image database 420, at least one or more anonymized individual medical images matching the recognized characteristic points, respectively, according to the result of recognizing the characteristic points. (②). In this case, the extracted individual medical image is not extracted from the same medical image (ie, right arm) as the incomplete medical image, but from the same body part (left arm) as the symmetrically transformed incomplete medical image. do.

In addition, the body size measuring apparatus 100 extracts feature points for a body part corresponding to a normal part except for an abnormal part of the symmetrically transformed incomplete medical image from the extracted individual medical image, and the symmetrically transformed incomplete medical image The degree of similarity between at least one recognized feature point for the normal part of , and the feature point extracted from the individual medical image is calculated (③).

Also, as a result of calculating the degree of similarity, the body size measuring apparatus 100 estimates an individual's medical image matching the symmetrically transformed incomplete medical image by extracting an anonymized medical image of an individual with the highest similarity.

Thereafter, the body size measuring apparatus 100 checks at least one feature point of the body part corresponding to the abnormal part of the symmetrically transformed incomplete medical image in the estimated individual medical image (④), and the By projecting (⑤) the identified feature points onto the abnormal portion of the symmetrically transformed incomplete medical image, the symmetrically transformed abnormal portion of the incomplete medical image is restored. At this time, the symmetrically transformed incomplete medical image obtained by restoring the abnormal part is symmetrically transformed into the original body part.

5 is a diagram illustrating a process of recognizing a feature point for a normal part from an incomplete medical image through a learning model according to an embodiment of the present invention.

5, the learning model according to an embodiment of the present invention is generated by learning a plurality of feature points and labels determined from a plurality of anonymized individual medical images for each body part.

In addition, the body size measuring apparatus 100 applies the incomplete medical image to at least one learning model selected according to a result of checking the body part of the incomplete medical image (or symmetrically transformed medical image), and the incomplete medical image Recognize at least one feature point for the normal part of

In this case, the incomplete medical image includes a normal part and an abnormal part of the incomplete medical image designated by the user, and arbitrary individual feature points (eg, A to N) selected by the user in the normal part of the incomplete medical image. and, as for the feature points for the normal part, by selecting the highest value among the probabilities output from each learning model, at least one feature point for the incomplete medical image is extracted, and coordinates and labels for the extracted feature points By confirming, it is recognized.

6 is a block diagram illustrating a configuration of an apparatus for measuring body size by estimating a body part with respect to an incomplete medical image according to an embodiment of the present invention.

As shown in FIG. 6 , the apparatus 100 for measuring body size for an incomplete medical image according to an embodiment of the present invention includes a medical image collecting unit 110 that collects a plurality of anonymized individual medical images, the collection A feature point learning unit 120 that generates a learning model by learning a plurality of anonymized individual medical images, respectively, a medical image input unit 130 that receives a medical image (eg, an incomplete medical image) from the user terminal 200; A medical image restoration unit 140 for restoring the incomplete medical image, a body size measuring unit 150 for measuring a body size of a corresponding body part in the restored medical image, and the incomplete medical image to restore the incomplete medical image and a medical image estimator 160 for estimating a medical image matching the image.

The medical image collection unit 110 collects anonymized individual medical images for each body part from at least one or more medical image providers 300 and stores them in the anonymized individual medical image database 420 .

The feature point learning unit 120 determines one or more feature points for the collected anonymized individual medical image, and labels the determined feature points by giving a label. In addition, the keypoint learning unit 120 generates a learning model for recognizing the keypoint from the input medical image by learning the labeled anonymized individual medical image.

The characteristic point is determined and labeled by a medical imaging expert, such as a doctor or an image reading expert, and is preferably determined as a part in which an anatomical position and label are determined.

The medical image input unit 130 performs a function of receiving an incomplete medical image including an abnormal portion and a normal portion from the user terminal 200 .

The medical image estimating unit 160 is for estimating a medical image matching the incomplete medical image. It is configured to include a 164 and a coordinate transformation unit 165 .

In addition, the medical image estimator 160, when estimating a medical image matching the incomplete medical image, uses the user's medical image symmetrical to the body part of the incomplete medical image, the same body as the incomplete medical image. In the case of using an anonymized individual medical image for a part, the operation is different depending on the case of using an anonymized individual medical image symmetrical to the body part of the incomplete medical image.

First, in the case of using a medical image of a user symmetric to a body part of an incomplete medical image, the operation of each component of the medical image estimator 160 will be described. At this time, the feature point recognition unit 161 identifies the body part of the incomplete medical image, selects at least one learning model according to the confirmed body part, and recognizes at least one feature point for the normal part of the incomplete medical image. do.

Also, the medical image extraction unit 162 extracts a medical image of a user symmetrical to a body part of the incomplete medical image, and estimates a medical image matching the incomplete medical image.

The coordinate converter 165 converts the extracted medical image of the user into coordinates for symmetrical body parts, and symmetrically transforms the extracted medical image of the user.

In addition, the feature point check unit 164 identifies at least one feature point for a body part corresponding to the abnormal part of the incomplete medical image in the extracted and symmetrically transformed medical image of the user.

Thereafter, the medical image restoration unit 140 projects the coordinates of the at least one or more identified feature points to the abnormal portion of the incomplete medical image, and projects the identified at least one or more feature points to the abnormal portion of the incomplete medical image, At least one feature point recognized for the normal part of the incomplete medical image and the projected at least one feature point are integrated with each other to restore the abnormal part of the incomplete medical image.

Hereinafter, an operation of each component of the medical image estimator 160 will be described when there is an anonymized individual medical image identical to the body part of the incomplete medical image. At this time, the feature point recognition unit 161 recognizes at least one feature point for the normal part of the incomplete medical image through at least one learning model according to the body part of the incomplete medical image.

In addition, the medical image extraction unit 162 extracts, from the anonymized individual medical image database 420, at least one or more anonymized individual medical images matching at least one feature point for the normal part of the recognized incomplete medical image. .

The similarity calculator 163 is configured to calculate the remaining body parts (ie, body parts corresponding to normal parts) other than the body parts corresponding to the abnormal parts of the incomplete medical images in the extracted at least one or more anonymized individual medical images. At least one feature point is extracted. In addition, the similarity calculator 163 calculates the degree of similarity between the extracted at least one feature point and at least one feature point recognized for the normal part of the incomplete medical image for each extracted anonymized medical image.

As a result of calculating the degree of similarity, the feature point check unit 164 selects and extracts an anonymized medical image of a specific individual having the highest similarity. In this case, the medical image estimator 160 estimates the selected and extracted medical image of a specific individual as a medical image that most closely matches the incomplete medical image. Thereafter, the feature point check unit 164 checks at least one feature point of the body part corresponding to the abnormal part of the incomplete medical image in the estimated medical image of the specific individual.

The medical image restoration unit 140 projects the coordinates of the at least one or more identified feature points to the abnormal portion of the incomplete medical image, and at least one or more feature points recognized for the normal portion of the incomplete medical image and the projected at least one feature point. By integrating one or more feature points with each other, an abnormal portion of the incomplete medical image is restored.

Hereinafter, an operation of each component of the medical image estimator 160 will be described in the case where only anonymized individual medical images symmetrical to body parts of the incomplete medical image exist. In this case, the coordinate converter 165 symmetrically transforms the incomplete medical image by converting the incomplete medical image into coordinates for symmetrical body parts.

The feature point recognition unit 161 selects at least one learning model according to the symmetrical body part, and converts it into coordinates for the symmetrical body part. At least one feature point for the normal part is recognized.

In addition, the medical image extraction unit 162 extracts at least one anonymized individual medical image matching at least one or more characteristic points recognized for the normal part of the symmetrically transformed incomplete medical image. In this case, the extracted anonymized individual medical image is identical to the body part of the symmetrically transformed incomplete medical image.

In addition, the similarity calculator 163 extracts at least one feature point for a body part corresponding to a normal part of the symmetrically transformed incomplete medical image from the at least one or more anonymized individual medical images extracted, and the extracted at least one A degree of similarity between one or more feature points and at least one feature point recognized for the normal part of the symmetrically transformed incomplete medical image is calculated for each extracted anonymized medical image.

In addition, as a result of calculating the degree of similarity, the feature point check unit 164 selects and extracts an anonymized medical image of a specific individual having the highest similarity. In this case, the medical image estimator 160 estimates that the extracted medical image of the specific individual is a medical image that most closely matches the incomplete medical image. Thereafter, the feature point check unit 164 checks at least one feature point for a body part corresponding to an abnormal part of the symmetrically transformed incomplete medical image in the estimated medical image of the specific individual, and at least one of the identified By projecting the coordinates of the above feature points onto the abnormal part of the symmetrically transformed incomplete medical image, and integrating at least one feature point recognized for the normal part of the incomplete medical image and the projected at least one feature point with each other, the symmetric transformation Recovers an abnormal part of an incomplete medical image. In this case, the symmetrically transformed incomplete medical image from which the abnormal part is restored is symmetrically transformed to the original body part.

7 is a flowchart illustrating a procedure for measuring a body size of a body part of an incomplete medical image based on a user's medical image symmetrical to the body part of the incomplete medical image according to an embodiment of the present invention.

As shown in FIG. 7 , the procedure for measuring the body size of the body part of the incomplete medical image based on the user's medical image symmetric to the body part of the incomplete medical image according to an embodiment of the present invention is first, The body size measuring apparatus 100 performs a medical image input step of receiving an incomplete medical image including an abnormal part and a normal part from the user (S110).

Next, the body size measuring apparatus 100 performs a feature point recognition step of recognizing a feature point for the normal part of the incomplete medical image (S120).

In the feature point recognition step, at least one learning model is selected according to the body part of the incomplete medical image, and the feature point is recognized by applying the incomplete medical image to the selected learning model. Meanwhile, the body size measuring apparatus 100 determines at least one feature point for each body part in a plurality of anonymized individual medical images, learns coordinates and labels for the determined at least one feature point, and generates the learning model The feature point learning step is performed in advance, and the generated learning model is stored and managed in the learning model database 410 .

Next, the body size measuring apparatus 100 includes a medical image extraction step of extracting a medical image of the user symmetrical to the body part of the incomplete medical image, and converting the extracted medical image of the user into coordinates for the symmetrical body part. A coordinate transformation step of converting and symmetrical transformation is performed (S130), and a characteristic point checking step of confirming characteristic points of a body part corresponding to the normal part of the incomplete medical image in the symmetrically transformed medical image of the user is performed (S140). ).

Here, the symmetric transformation is a transformation into coordinates for a body part symmetric with a body part of a specific medical image, and since it has been described with reference to FIG. 1 , further detailed description will be omitted.

Next, the body size measuring apparatus 100 projects the coordinates of the identified feature points on the abnormal portion to perform a medical image restoration step of restoring the abnormal portion (S150). The restoration step is performed by projecting the coordinates of the identified feature points on an abnormal portion of the incomplete medical image, and integrating the projected feature points with the feature points recognized for the normal portion of the incomplete medical image.

Next, the body size measuring apparatus 100 performs a body size measurement step of measuring the body size of the body part of the incomplete medical image from which the abnormal part is restored and providing it to the user terminal 200 (S160). As described above, the body size is performed by interconnecting feature points of the reconstructed incomplete medical image.

8 is a flowchart illustrating a procedure for measuring a body size of a body part of an incomplete medical image based on an anonymized individual medical image identical to the body part of the incomplete medical image according to an embodiment of the present invention.

As shown in FIG. 8 , the procedure for measuring body size based on the anonymized individual medical image identical to the body part of the incomplete medical image according to an embodiment of the present invention is, first, the body size measuring apparatus 100, A medical image input step of receiving an incomplete medical image from the user (S210), and a feature point recognition step of recognizing at least one feature point for the normal portion of the received incomplete medical image are performed (S220).

Next, a medical image extraction step of extracting at least one or more anonymized individual medical images each matching at least one feature point for the normal part of the recognized incomplete medical image is performed (S230).

Next, the body size measuring apparatus 100 extracts at least one feature point for a body part corresponding to an abnormal part of the incomplete medical image from the extracted anonymized individual medical image, and extracts the extracted feature point and the incomplete medical image A similarity calculation step of calculating the similarity between the feature points recognized in the image for each extracted individual medical image is performed (S240).

Next, the body size measuring apparatus 100 checks at least one feature point for a body part corresponding to an abnormal part of the incomplete medical image in the medical image of an anonymized individual having the highest similarity as a result of calculating the similarity. A feature point checking step (S250) is performed, and a medical image recovery step of restoring the abnormal portion is performed by projecting the coordinates of the identified feature point onto the abnormal portion of the incomplete medical image (S260).

Thereafter, the body size measuring apparatus 100 performs a body size measurement step of measuring the body size of the body part of the incomplete medical image from which the abnormal part is restored and providing it to the user terminal 200 (S270).

9 is a flowchart illustrating a procedure for measuring a body size based on an anonymized individual medical image symmetrical to a body part of an incomplete medical image according to an embodiment of the present invention.

As shown in FIG. 9, the procedure for measuring body size based on anonymized individual medical images symmetrical to body parts of incomplete medical images according to an embodiment of the present invention is first, the body size measuring apparatus 100 is , when an incomplete medical image is input from the user terminal 200 (S310), a coordinate transformation step of symmetrically transforming the incomplete medical image by converting it into coordinates of a body part symmetric to the body part of the incomplete medical image, and the symmetric A feature point recognition step of recognizing at least one feature point for the normal part of the transformed incomplete medical image is performed (S320). Meanwhile, since the symmetric transformation and recognizing the feature point have been described above, a further detailed description will be omitted.

Next, the body size measuring apparatus 100 performs a medical image extraction step of extracting at least one anonymized individual medical image each matching the recognized feature points (S330). In this case, the extracted anonymized individual medical image is a medical image of a body part symmetrical to the received incomplete medical image, and is a medical image of the same body part as the symmetrically transformed incomplete medical image.

Next, the body size measuring apparatus 100 extracts at least one feature point for a body part corresponding to a normal part of the symmetrically transformed incomplete medical image from the extracted anonymized individual medical image, and extracts the extracted feature point and the A similarity calculation step of calculating the similarity with respect to the feature points recognized in the symmetrically transformed incomplete medical image is performed (S340).

Next, the body size measuring apparatus 100 performs a feature point checking step of confirming feature points of a body part corresponding to an abnormal part of the symmetrically transformed incomplete medical image in an anonymized medical image of an individual with the highest similarity (S350) ), and projecting the coordinates of the identified feature points onto the abnormal portion of the symmetrically transformed incomplete medical image to perform a medical image restoration step of restoring the symmetrically transformed abnormal portion of the incomplete medical image (S360).

Thereafter, the body size measuring apparatus 100 symmetrically converts the symmetrically transformed incomplete medical image from which the abnormal part is restored back to the original body part, and the body size for the body part of the symmetrically transformed incomplete medical image again. Performs a body size measurement step of measuring (S370).

10 is a diagram illustrating a hardware structure of an apparatus for measuring body size by estimating a body part with respect to an incomplete medical image according to an embodiment of the present invention.

As shown in FIG. 10 , the hardware structure of the apparatus 100 for measuring body size according to an embodiment of the present invention includes a central processing unit 1000 , a memory 2000 , a user interface 3000 , and a database interface 4000 . ), a network interface 5000, a web server 6000, and the like.

The user interface 3000 provides input and output interfaces to the user by using a graphical user interface (GUI).

The database interface 4000 provides a storage interface between the database 400 and a hardware structure, and the network interface 5000 provides a network connection between devices owned by a user.

The web server 6000 provides a means for a user to access the body size measuring device 100 via a network. Most users can remotely access the web server and use the body size measuring apparatus 100 .

Each step of the above-described configuration or method may be implemented as computer-readable code on a computer-readable recording medium or transmitted through a transmission medium. The computer-readable recording medium is a data storage device capable of storing data that can be read by a computer system. Examples of computer-readable recording media include, but are not limited to, databases, ROMs, RAMs, CD-ROMs, DVDs, magnetic tapes, floppy disks, and optical data storage devices. The transmission medium may include a carrier wave transmitted through the Internet or various types of communication channels.

In addition, the computer readable recording medium may be distributed through a network coupled computer system so that the computer readable code is stored and executed in a distributed manner.

In addition, at least one or more components applied to the present invention may include or be implemented by a processor such as a central processing unit (CPU) 1000, a microprocessor, etc. that perform each function, and two or more of the components are It may be combined into one single component to perform all operations or functions for two or more combined components. In addition, a part of at least one or more components applied to the present invention may be performed by other components among these components. In addition, communication between the components may be performed through a bus (not shown).

In this way, even when an incomplete medical image is input, the present invention estimates at least one or more medical images matching the incomplete medical image and restores the incomplete medical image based on the estimated medical image, so that the body part of the incomplete medical image is It has the effect of making it possible to accurately measure the body size for

As described above, the present invention has been described with reference to the embodiments shown in the drawings, which are merely exemplary, and those of ordinary skill in the art can make various modifications and equivalent other embodiments therefrom. You will understand that it is possible. Therefore, the technical protection scope of the present invention should be determined by the following claims.

100: Device for measuring body size through estimation of body parts for incomplete medical images 110: Medical image collecting unit
120: feature point learning unit 130: medical image input unit
140: medical image restoration unit 150: body size measurement unit
160: medical image estimator 161: feature point recognition unit
162: medical image extraction unit 163: similarity calculation unit
164: feature point confirmation unit 165: coordinate transformation unit
200: user terminal 300: medical image provider
400: database

Claims (14)

a medical image estimator for estimating a medical image matching an incomplete medical image including an abnormal part that is partially damaged or covered;
The incomplete medical image is obtained by projecting the coordinates of at least one feature point for the body part corresponding to the abnormal part of the estimated medical image onto the abnormal part and integrating it with at least one feature point for the normal part of the incomplete medical image. a medical image restoration unit that restores; and
Including; a body size measuring unit for measuring the body size in the restored medical image;
The medical image estimation unit,
Extracting a user's medical image for a body part symmetrical with the body part of the incomplete medical image, or extracting a medical image that most closely matches the normal part among a plurality of anonymized individual medical images as the normal part of the incomplete medical image extracting or extracting the closest matching medical image among a plurality of anonymized individual medical images for a body part symmetrical with the normal part of the incomplete medical image to estimate a medical image matching the incomplete medical image A device for measuring body size by estimating body parts for incomplete medical images. delete delete The method according to claim 1,
The medical image estimation unit,
It further includes; a coordinate conversion unit for converting the extracted medical image of the user into coordinates for a symmetrical body part, or converting the incomplete medical image into coordinates for a symmetrical body part;
An apparatus for measuring body size by estimating a body part for an incomplete medical image, characterized in that the medical image matching the incomplete medical image is estimated by using the body part converted into the coordinates. The method according to claim 1,
The body size measuring device,
A feature point learning unit that determines at least one feature point from a plurality of anonymized individual medical images, and generates a learning model for learning coordinates and labels for the determined at least one feature point; further comprising,
The medical image estimation unit,
a feature point recognition unit that selects at least one learning model according to a body part of the incomplete medical image, and applies the incomplete medical image to the selected at least one or more learning models to recognize at least one feature point for the normal part;
a medical image extraction unit for extracting a user's medical image for a body part symmetrical to the body part of the incomplete medical image;
a coordinate converter for converting the extracted medical image of the user into coordinates for symmetrical body parts; and
Incomplete medical image, characterized in that it further comprises; A device for measuring body size through estimation of body parts. The method according to claim 1,
The body size measuring device,
A feature point learning unit that determines at least one feature point from a plurality of anonymized individual medical images, and generates a learning model for learning coordinates and labels for the determined at least one feature point; further comprising,
The medical image estimation unit,
a feature point recognition unit that selects at least one learning model according to a body part of the incomplete medical image, and applies the incomplete medical image to the selected at least one or more learning models to recognize at least one feature point for the normal part;
a medical image extraction unit for extracting at least one anonymized individual medical image matching the recognized at least one feature point;
a similarity calculator for extracting at least one feature point for a body part corresponding to the normal part from the extracted individual medical image, and calculating a similarity between the recognized feature point for the normal part and the extracted feature point; and
as a result of calculating the degree of similarity, a feature point checking unit that selects a medical image of a specific individual having the highest degree of similarity and identifies at least one feature point for the body part corresponding to the abnormal part; A device for measuring body size by estimating body parts for medical images. The method according to claim 1,
The body size measuring device,
A feature point learning unit that determines at least one feature point from a plurality of anonymized individual medical images, and generates a learning model for learning coordinates and labels for the determined at least one feature point; further comprising,
The medical image estimation unit,
a coordinate converter converting the incomplete medical image into coordinates for symmetrical body parts;
At least one learning model is selected according to the body part symmetrical with the incomplete medical image, and the incomplete medical image converted into coordinates for the symmetrical body part is applied to the selected learning model, and at least one learning model for the normal part is selected. a feature point recognition unit for recognizing the above feature points;
a medical image extraction unit for extracting at least one anonymized individual medical image matching the recognized at least one feature point;
a similarity calculator for extracting at least one feature point corresponding to the normal part from the extracted individual medical image, and calculating a similarity between the recognized feature point for the normal part and the extracted feature point; and
as a result of calculating the degree of similarity, a feature point checking unit that selects a medical image of a specific individual having the highest degree of similarity and identifies at least one feature point for the body part corresponding to the abnormal part; A device for measuring body size by estimating body parts for medical images. A medical image estimation step of estimating a medical image matching an incomplete medical image including an abnormal part that is partially damaged or covered in an apparatus for measuring body size by estimating a body part for an incomplete medical image;
In the apparatus for measuring the body size, coordinates of at least one feature point for a body part corresponding to the abnormal part of the estimated medical image are projected onto the abnormal part, and at least one feature point for the normal part of the incomplete medical image and a medical image restoration step of integrating and restoring the incomplete medical image; and
In the body size measuring device, a body size measuring step of measuring the body size in the restored medical image; includes,
The medical image estimation step is
Extracting a user's medical image for a body part symmetrical with the body part of the incomplete medical image, or extracting a medical image that most closely matches the normal part among a plurality of anonymized individual medical images as the normal part of the incomplete medical image extracting or extracting the closest matching medical image among a plurality of anonymized individual medical images for a body part symmetrical with the normal part of the incomplete medical image to estimate a medical image matching the incomplete medical image A method of measuring body size by estimating body parts for incomplete medical images. delete delete 9. The method of claim 8,
The medical image estimation step is
The method further includes; a coordinate transformation step of converting the extracted medical image of the user into coordinates for a symmetrical body part, or converting the incomplete medical image into coordinates for a symmetrical body part;
A method of measuring body size by estimating a body part for an incomplete medical image, characterized in that a medical image matching the incomplete medical image is estimated using the body part converted into the coordinates. 9. The method of claim 8,
The method of measuring the body size,
In the body size measuring device, determining at least one feature point from a plurality of anonymized individual medical images, and generating a learning model for learning coordinates and labels for the determined at least one feature point; ,
The medical image estimation step is
a feature point recognition step of selecting at least one learning model according to a body part of the incomplete medical image, and applying the incomplete medical image to the selected at least one learning model to recognize at least one feature point for the normal part;
a medical image extraction step of extracting a user's medical image for a body part symmetrical to the body part of the incomplete medical image;
a coordinate transformation step of converting the extracted user's medical image into coordinates for symmetrical body parts; and
In an incomplete medical image, characterized in that it further comprises; a feature point checking step of confirming at least one feature point for the body part corresponding to the abnormal part in the medical image of the user converted into the coordinates for the symmetrical body part A method of measuring body size by estimating body parts. 9. The method of claim 8,
The method of measuring the body size,
In the body size measuring device, a feature point learning step of determining at least one feature point from a plurality of anonymized individual medical images, and generating a learning model for learning coordinates and labels for the determined at least one feature point further comprises; ,
The medical image estimation step is
a feature point recognition step of selecting at least one learning model according to a body part of the incomplete medical image, and applying the incomplete medical image to the selected at least one learning model to recognize at least one feature point for the normal part;
a medical image extraction step of extracting at least one anonymized individual medical image matching the recognized at least one or more feature points;
a similarity calculation step of extracting at least one feature point for a body part corresponding to the normal part from the extracted individual medical image, and calculating a similarity between the feature point recognized for the normal part and the extracted feature point; and
a feature point checking step of selecting at least one feature point for a body part corresponding to the abnormal part by selecting a medical image of a specific individual having the highest similarity as a result of calculating the similarity; A method of measuring body size through estimation of body parts on medical images. 9. The method of claim 8,
The method of measuring the body size,
In the body size measuring device, a feature point learning step of determining at least one feature point from a plurality of anonymized individual medical images, and generating a learning model for learning coordinates and labels for the determined at least one feature point further comprises; ,
The medical image estimation step is
a coordinate converter converting the incomplete medical image into coordinates for symmetrical body parts;
At least one learning model is selected according to the body part symmetrical with the incomplete medical image, and the incomplete medical image converted into coordinates for the symmetrical body part is applied to the selected learning model, and at least one learning model for the normal part is selected. a feature point recognition step of recognizing the above feature points;
a medical image extraction step of extracting at least one anonymized individual medical image matching the recognized at least one or more feature points;
a similarity calculation step of extracting at least one feature point corresponding to the normal part from the extracted individual medical image, and calculating a similarity between the recognized feature point for the normal part and the extracted feature point; and
a feature point checking step of selecting at least one feature point for a body part corresponding to the abnormal part by selecting a medical image of a specific individual having the highest similarity as a result of calculating the similarity; A method of measuring body size through estimation of body parts on medical images.

Images