KR102258070B1 - Method for evaluating foot type and device evaluating foot type using the same - Google Patents

Abstract

The present invention includes the steps of receiving a medical image of the foot of an individual, using a region prediction model configured to predict a talus region and a first metatarsal region based on the medical image of the foot, Predicting the talus region and the first metatarsal region, determining the talus-first metatarsal angle based on the centerline for each of the predicted talus region and the first metatarsal region, 1 It provides a foot type prediction method and a foot type evaluation device using the same, comprising the step of evaluating the type of the individual's foot based on the angle between the metatarsal bones.

Description

Foot type evaluation method and device for evaluating foot type using the same {METHOD FOR EVALUATING FOOT TYPE AND DEVICE EVALUATING FOOT TYPE USING THE SAME}

The present invention relates to a foot type evaluation method and a device using the same, and more particularly, to a method and device configured to predict the foot type based on a medical image.

The medical imaging apparatus is a device for obtaining an image of an internal structure of a target object. Such a medical imaging apparatus is a non-invasive test apparatus performed without causing pain to the human body, and photographs and processes structural details within the body, internal tissues, and flows of fluids, and displays them to a medical professional. Medical personnel may diagnose a patient's health condition and disease by using a medical image output from a medical imaging device.

Medical imaging devices include magnetic resonance imaging (MRI) devices, computed tomography (CT) devices, X-ray devices, and ultrasound diagnostic devices for providing magnetic resonance images. Etc. Medical images acquired from such a medical imaging apparatus may be used for diagnosis of a disease.

Meanwhile, in a diagnosis based on a medical image, the measurement process may be very important in diagnosing various lesions and further determining a stage of progression of a disease. At this time, for accurate and highly reproducible measurement, in particular, in measurement based on a 2D planar medical image, a reference point such as an anatomical landmark may be important. However, the anatomical landmarks may exist in different positions for each diagnosis entity, and may be determined differently according to external factors such as an individual's posture and a medical professional's skill level. For this reason, measurement based on medical images, in particular, measurement analysis based on 2D planar medical images, may have poor accuracy and reproducibility. In this case, the inaccuracy of the measured value may lower the reliability of the diagnosis of the disease and the evaluation result of the progression of the disease.

In particular, in the case of the flatfoot examination, the position of the anatomical index varies depending on the posture of the foot or leg, and the position thereof at the time of photographing, which leads to inaccurate measurement, and consequently, a diagnosis result of flatfoot with low reliability can be provided. At this time, flat feet among the types of feet are also related to the judgment of military service in the Republic of Korea, so accurate diagnosis of the types of feet can be an issue of great social interest.

As the accuracy of diagnosis is more demanded for the improvement of medical services, there is a constant demand for the development of a new evaluation method that can evaluate diagnosis based on medical images, especially the type of foot such as flat feet with high accuracy. .

The technology that is the background of the invention has been prepared to facilitate understanding of the present invention. It should not be understood as an admission that the matters described in the technology underlying the invention exist as prior art.
Unexamined Patent Publication No. 10-2019-0023003 (2019.03.07.)

On the other hand, the inventors of the present invention focused on the talus-first metatarsal angle formed by the talus and the first metatarsal bone in relation to evaluation of the type of foot such as flat foot and concave foot.

More specifically, the inventors of the present invention note that the angle between the talus-first metatarsal bone measured for the medical image of the individual's foot may be measured differently depending on the anatomical structure of the foot that is different for each individual, and the skill level of a medical practitioner. I did.

At this time, the inventors of the present invention attempted to supplement the limitations and problems of the conventional system for evaluating the type of feet such as flat feet based on medical images by introducing a system based on an artificial intelligence algorithm.

The inventors of the present invention have recognized that a prediction model learned from data of a foot medical image can be used in connection with a diagnosis based on a foot medical image.

In particular, the inventors of the present invention determine the region of the talus and the first metatarsal bone with respect to the foot medical image, and predict a predetermined feature point according to the anatomical structure of the talus and the first metatarsal, thereby reproducibly evaluating and providing the type of foot. It was noted that it could be.

More specifically, the inventors of the present invention constructed a prediction model to predict the regions of the talus and the first metatarsal bone. Accordingly, the talus center line and the first metatarsal center line are extracted based on the feature points determined in advance according to the anatomy of the talus and the foot within each predicted region, and the talus-first metatarsal angle, which is the angle between the center lines, will be calculated. It could be confirmed. That is, the inventors of the present invention were able to recognize that the foot type can be evaluated and provided with high accuracy by adding measurement information using artificial intelligence in addition to the reading of the foot medical image by the medical staff.

As a result, the inventors of the present invention predict a feature point and a centerline for the region of the talus and the first metatarsal bone predicted using a predictive model, and based on this, a new foot medical treatment that can provide an angle between the talus and the first metatarsal bone. We came to develop an image-based foot type evaluation system.

In particular, the inventors of the present invention, by providing a new foot type evaluation system, the anatomical structure of the foot, and the limitations of the conventional medical image-based foot type evaluation system, such as occurrence of a diagnosis error according to the skill level of a medical practitioner. I could expect to get over it.

On the other hand, the inventors of the present invention note that accurate segmentation may be difficult when the resolution of the input image is high in the learning process of region prediction of the predictive model, or when the shape to be segmented and the brightness value of the surrounding region are not clearly classified. I did.

In particular, the inventors of the present invention have noted that when the number of training data is small, the accuracy of classification may be degraded as it can be learned without considering the core area.

Accordingly, the inventors of the present invention set a region of interest (ROI) such as the talus region or the first metatarsal region with respect to the training foot medical image in the prediction process, and cropped to include only the main core regions to predict the model. By learning, it was expected to improve the accuracy of the region division for the region prediction model.

In other words, the inventors of the present invention were able to develop a prediction model capable of learning centered on a core region so that prediction within the correct answer region, in which the actual talus or the first metatarsal bone exists, is activated.

Accordingly, the problem to be solved by the present invention is to predict the talus region and the first metatarsal region for the foot medical image using a region prediction model configured to predict the talus region and the first metatarsal region based on the received foot medical image. , To provide a method for predicting a foot type, configured to determine and provide a talus-first metatarsal angle.

Another problem to be solved by the present invention is to predict a predetermined feature point according to the talus region and the first metatarsal region based on the received foot medical image, and the angle formed by the center line of the talus and the center line of the first metatarsal bone based on the feature point. It is to provide a method of predicting the type of foot, configured to determine and provide a.

Another problem to be solved by the present invention is to provide a foot type prediction method configured to predict the talus-first metatarsal angle based on the result values output by various prediction models, and to evaluate the foot type based on this It is to do.

Another problem to be solved by the present invention is a receiver configured to receive a medical image of an individual's foot, and predicts the talus region and the first metatarsal region for the medical image of the foot, and provides an angle between the talus and the first metatarsal bone based on this It is to provide a device for evaluating the type of foot, including a processor configured to be.

The problems of the present invention are not limited to the problems mentioned above, and other problems that are not mentioned will be clearly understood by those skilled in the art from the following description.

In order to solve the above-described problems, a method of predicting a foot type according to an embodiment of the present invention is provided. The present method is a method of evaluating a foot type implemented by a processor, comprising the steps of: receiving a foot medical image of an individual, a region configured to predict a talus region and a first metatarsal region based on the foot medical image Predicting the talus region and the first metatarsal region within the foot medical image using the predictive model, based on the centerline for each of the predicted talus region and the first metatarsal region, the talus-first metatarsal angle (talus-first) metatarsal angle) and assessing the type of the individual's foot based on the talus-first metatarsal angle.

According to a feature of the present invention, the evaluation method of the present invention uses a centerline prediction model configured to predict the centerline of the talus and the centerline of the first metatarsal for each of the talus region and the first metatarsal region, It may further include predicting each centerline of the metatarsal bone.

According to another feature of the present invention, the evaluation method of the present invention uses a feature point prediction model configured to predict a predetermined feature point for each of the talus region and the first metatarsal region, and feature points for each of the talus region and the first metatarsal region. It may further include predicting, and predicting, respectively, a center line of the talus and a center line of the first metatarsal bone based on the feature points.

According to another feature of the present invention, the predetermined feature points may exist in two pairs. Furthermore, the step of predicting the feature points for each of the talus region and the first metatarsal region using the feature point prediction model includes two pairs of predetermined values on the boundary line of the talus region and the boundary line of the first metatarsal region using the feature point prediction model. Predicting each of the feature points, and determining a center point of each of the predicted two pairs of feature points for each of the talus region and the first metatarsal region. Furthermore, the step of predicting the center line of the talus and the center line of the first metatarsal bone, respectively, based on the feature points, may include predicting the center line of the talus and the center line of the first metatarsal bone, respectively, based on the center point.

According to another feature of the present invention, the evaluation method of the present invention may further include receiving a standard talus image and a standard first metatarsal image including two pairs of predetermined standard feature points on the boundary line of the talus region. have. In this case, the step of predicting each of the two pairs of feature points includes a boundary line of the talus region and a distance of each of the two pairs of standard feature points in the standard talus image, and the boundary line of the first metatarsal region and two pairs of standard feature points in the standard first metatarsal image. It may include calculating each distance, and predicting, respectively, two pairs of feature points on the boundary line of the talus region and the boundary line of the first metatarsal region based on the respective distances.

According to another feature of the present invention, there may be a plurality of standard talus images and standard first metatarsal images. Further, the step of calculating each distance may include a boundary line of the talus region and a distance of each of two pairs of standard feature points in a plurality of standard talus images, and a boundary line of the first metatarsal region and two pairs of a plurality of standard first metatarsal images. It may include calculating distances of each of the standard feature points with respect to a plurality of standard talus images and a plurality of standard first metatarsal images. Furthermore, in the step of predicting each of the two pairs of feature points, based on the respective distances calculated for each of the plurality of standard talus images and the plurality of standard first metatarsal images, on the boundary line of the talus region and the boundary line of the first metatarsal region. It may include predicting each of the two pairs of feature points.

According to another feature of the present invention, the step of determining the central point of each of the two pairs of feature points for each of the talus region and the first metatarsal region may include, according to a plurality of standard talus images and a plurality of standard first metatarsal images, the talus region A central point for each of the two pairs of feature points predicted on the boundary line of and the boundary line of the first metatarsal region may be determined for each of the talus region and the first metatarsal region.

According to another feature of the present invention, the step of predicting the center line of the talus and the center line of the first metatarsal bone, respectively, based on the feature points, comprises a talus region and a first metatarsal bone according to a plurality of standard talus images and a plurality of standard first metatarsal images. Determining centroids of the center points determined for each of the talus regions and the first metatarsal regions, and based on the centroids determined for each of the talus region and the first metatarsal region, 1 It may include the step of predicting each centerline of the metatarsal bone.

According to another feature of the present invention, the evaluation method of the present invention may further include providing the predicted two pairs of feature points.

According to another feature of the present invention, the evaluation method of the present invention may further include providing an angle between the talus and the first metatarsal bone.

According to another feature of the present invention, the step of evaluating the type of the individual's foot is, when the talus-first metatarsal angle is greater than -4 degrees, it is evaluated as a concave foot, or the talus-first metatarsal angle is 4 If it is greater than the degree, it may include the step of evaluating as flat feet.

According to another feature of the present invention, the step of evaluating the type of foot of the subject may further include evaluating the severity of the flat foot of the subject. At this time, the step of evaluating the severity of the flat foot of the individual is, when the angle between the talus and the first metatarsal is 4 degrees to 15, it is evaluated as being mild flatfoot, or the angle between the talus and the first metatarsal is between 15 and 30 degrees. In the case of one, it may be evaluated as a moderate flat foot, or if the angle between the talus and the first metatarsal is greater than 30 degrees, evaluating as a severe flat foot.

According to another feature of the present invention, the region prediction model includes receiving a training foot medical image including a correct answer talus region and a correct answer first metatarsal region predetermined for the foot of a standard entity, and within the training foot medical image The model may be trained to predict the talus region and the first metatarsal region through the step of predicting the talus region and the first metatarsal region.

According to another feature of the present invention, after the step of predicting the talus region and the first metatarsal region in the training foot medical image, the correct answer talus region for each of the talus region and the first metatarsal region predicted by the region prediction model And calculating a ratio of an overlapped region in which each of the first metatarsal regions of the correct answer overlap, and evaluating a region prediction model based on the ratio of the overlapped region.

According to another feature of the present invention, the training foot medical image may include a medical image including only the correct talus region and a medical image including only the correct answer first metatarsal region in the training foot medical image.

In order to solve the above-described problem, a device for predicting a foot type according to another embodiment of the present invention is provided. The device includes a receiving unit configured to receive a medical image of an individual's foot, and a processor coupled to communicate with the receiving unit. At this time, the processor predicts the talus region and the first metatarsal region in the foot medical image using a region prediction model configured to predict the talus region and the first metatarsal region based on the foot medical image. Then, the talus-first metatarsal angle was determined based on the centerline for each of the predicted talus region and the first metatarsal region, and the type of the individual's foot based on the talus-first metatarsal angle. Is configured to evaluate.

According to a feature of the present invention, the processor uses a centerline prediction model configured to predict the centerline of the talus and the centerline of the first metatarsal for each of the talus region and the first metatarsal region, and calculates the centerline of the talus and the centerline of the first metatarsal bone. Each can be further configured to predict.

According to another feature of the present invention, the processor predicts a feature point for each of the talus region and the first metatarsal region, using a feature point prediction model configured to predict a predetermined feature point for each of the talus region and the first metatarsal region, It may be further configured to predict the center line of the talus and the center line of the first metatarsal bone, respectively, based on the feature points.

According to another feature of the present invention, the predetermined feature points exist in two pairs, and the processor uses the feature point prediction model to determine two pairs of predetermined feature points on the boundary line of the talus region and the boundary line of the first metatarsal region, respectively. It may be further configured to predict, determine the center point of each of the two predicted feature points for each of the talus region and the first metatarsal region, and predict the center line of the talus and the center line of the first metatarsal bone, respectively, based on the center point.

According to another feature of the present invention, the receiving unit may be further configured to receive a standard talus image and a standard first metatarsal image including two pairs of standard feature points determined in advance on a boundary line of the talus region. At this time, the processor calculates the distance of each of the boundary line of the talus region and the two pairs of standard feature points in the standard talus image, and the distance of each of the boundary line of the first metatarsal region and the two pairs of standard feature points in the standard first metatarsal image, respectively. It may be further configured to predict each of the two pairs of feature points on the boundary line of the talus region and the boundary line of the first metatarsal region based on the distance of.

According to another feature of the present invention, the standard talus image and the standard first metatarsal image are plural, and the processor includes a boundary line of the talus region and a distance of each of the two pairs of standard feature points in the plurality of standard talus images, and the first metatarsal region. The boundary line of and the distance of each of the two pairs of standard feature points in the plurality of standard first metatarsal images are calculated for a plurality of standard talus images and a plurality of standard first metatarsal images, and a plurality of standard talus images and a plurality of standard first metatarsal images The two pairs of feature points may be further predicted on the boundary line of the talus region and the boundary line of the first metatarsal region, based on the respective distances calculated for each metatarsal image.

According to another feature of the present invention, the processor includes a center point for each of the two pairs of feature points predicted on the boundary line of the talus region and the boundary line of the first metatarsal region according to a plurality of standard talus images and a plurality of standard first metatarsal images. Is determined for each of the talus region and the first metatarsal region, and the centroids of the central points determined for each of the talus region and the first metatarsal region according to a plurality of standard talus images and a plurality of standard first metatarsal images are talus. It may be further configured to determine each of the region and the first metatarsal region, and predict, respectively, a center line of the talus and a center line of the first metatarsal bone, based on the centroids determined for each of the talus region and the first metatarsal region.

According to another feature of the present invention, the evaluation device of the present invention may further comprise an output configured to provide the predicted two pairs of feature points.

According to another feature of the present invention, the evaluation device of the present invention may further include an output configured to provide an angle between the talus and the first metatarsal bone.

According to another feature of the present invention, the processor is evaluated as a concave foot when the angle between the talus and the first metatarsal is greater than -4 degrees, or if the angle between the talus and the first metatarsal is more than 4 degrees, the flat foot It can be further configured to evaluate as.

According to another feature of the present invention, the processor is evaluated as mild flatfoot when the talus-first metatarsal angle is 4 degrees to 15 degrees, or the talus-first metatarsal angle is 15 degrees to 30 degrees. In the case of one, it may be further configured to evaluate as a moderate flat foot, or to evaluate as a severe flat foot when the talus-first metatarsal angle is greater than 30 degrees.

The present invention provides a foot type prediction method configured to determine a core region for a foot medical image based on a predictive model and predict a talus-first metatarsal angle based on a predetermined feature point, and a device using the same. Highly, there is an effect of providing a measurement result of the talus-first metatarsal angle and an accurate evaluation result accordingly.

More specifically, the present invention divides the talus region and the first metatarsal region for the foot medical image using the region prediction model, and based on the output value of the centerline prediction model and/or the feature point prediction model, the talus-first metatarsal bone By providing each measurement information, there is an effect of providing information on diagnosis of an individual's flat feet and concave feet.

Accordingly, the present invention is a limitation and problems of a system for evaluating the type of foot based on a conventional medical image that can be measured differently according to the angle between the talus and the first metatarsal bone, the anatomy of the foot different for each individual, and the skill level of a medical practitioner. There is an effect that can overcome them.

In addition, the present invention sets the ROI such as the talus region or the first metatarsal region for the training foot medical image for the predictive model, cropping to include only the main core region, and training the predictive model. There is an effect that can be expected to improve accuracy.

Accordingly, the present invention can provide a prediction model, in particular, by evaluating the foot type with high accuracy regardless of the quality of the foot medical image input to the region prediction model. More specifically, the present invention is capable of predicting these regions by accurately segmenting the talus region or the first metatarsal region even when the resolution of the foot medical image is high or the shape to be segmented and the brightness value of the surrounding region are not clearly classified. There is an effect that the foot type can be more accurately evaluated by using the area prediction model.

The effects according to the present invention are not limited by the contents exemplified above, and more various effects are included in the present specification.

1 is an exemplary diagram showing the configuration of a device for evaluating a foot type according to an embodiment of the present invention.
2A is a diagram illustrating a procedure of a method of predicting a foot type according to an embodiment of the present invention.
2B is an exemplary diagram illustrating a procedure for predicting a talus region and a first metatarsal region using a region prediction model used in various embodiments of the present invention.
FIG. 3 exemplarily shows a procedure for predicting each between the talus and the first metatarsal bone using a centerline prediction model used in various embodiments of the present invention.
4 and 5 exemplarily illustrate a procedure of predicting each between the talus and the first metatarsal bone using a feature point prediction model used in various embodiments of the present invention.
6A and 6B illustrate medical image data of the foot for training in a region prediction model used in various embodiments of the present invention.
6C illustrates an exemplary configuration of a region prediction model used in various embodiments of the present invention.
7A to 7F illustrate evaluation results of the talus region and the first metatarsal region divided by the region prediction model used in various embodiments of the present invention.
8A to 8D illustrate the predicted talus centerline and first metatarsal centerline, and evaluation results of the talus angle and first metatarsal angle formed by these centerlines according to various embodiments of the present invention.
9A and 9B illustrate evaluation results for the talus-first metatarsal angle calculated according to various embodiments of the present disclosure.

Advantages of the invention and a method of achieving them will become apparent with reference to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in various forms different from each other, and only these embodiments make the disclosure of the present invention complete, and common knowledge in the technical field to which the present invention pertains. It is provided to completely inform the scope of the invention to the possessor, and the invention is only defined by the scope of the claims.

The shapes, sizes, ratios, angles, numbers, etc. disclosed in the drawings for explaining the embodiments of the present invention are exemplary, and thus the present invention is not limited to the illustrated matters. In addition, in describing the present invention, when it is determined that a detailed description of a related known technology may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted. When'include','have','consists of' and the like mentioned in the present specification are used, other parts may be added unless'only' is used. In the case of expressing the constituent elements in the singular, it includes the case of including the plural unless specifically stated otherwise.

In interpreting the components, even if there is no explicit description, it is interpreted as including an error range.

Each of the features of the various embodiments of the present invention may be partially or entirely combined or combined with each other, and as a person skilled in the art can fully understand, technically various interlocking and driving are possible, and each of the embodiments may be independently implemented with respect to each other. It may be possible to do it together in a related relationship.

For clarity of interpretation of the present specification, terms used in the present specification will be defined below.

As used herein, the term "individual" may mean all objects for which a flat foot is desired or a foot type is to be predicted. For example, the individual may be a suspected flatfoot individual. In this case, the individual disclosed in the present specification may be any mammal except humans, but is not limited thereto.

As used herein, the term "foot medical image" may mean a foot medical image captured by an imaging apparatus. For example, the medical image of the foot may be a medical image of the foot on the side of the object. In this case, the foot medical image may include the talus and the first metatarsal bone of the side. That is, the foot medical image may include a talus region in which the talus is present and a first metatarsal region in which the first metatarsal bone is present. Preferably, the medical image of the foot may be a radiographic image of the side of the foot, but is not limited thereto.

Meanwhile, the foot medical image may be a 2D image, a 3D image, a still image of one cut, or a moving image composed of a plurality of cuts. For example, if the foot medical image is a video composed of a plurality of cuts, the talus region and the first metatarsal region for each of the plurality of foot medical images are predicted according to the foot type prediction method according to an embodiment of the present invention, Based on this, the angle between the talus and the first metatarsal may be calculated. As a result, the present invention predicts the talus region and the first metatarsal region, and predicts the talus-first metatarsal angle at the same time as receiving the foot medical image from an imaging device such as a radiographic apparatus, thereby predicting the foot type in real time. Can provide diagnostic information for

As used herein, the term “talar region” may mean a region in which the talus is present in the foot medical image. In this case, "the boundary line of the talus region" may mean the boundary line of the talus region existing outside the talus region.

As used herein, the term "first metatarsal region" may mean a region in which the first metatarsal bone exists in the foot medical image. In this case, "the boundary line of the first metatarsal region" may mean a boundary line of the first metatarsal region existing outside the first metatarsal region.

At this time, the term used in the present specification, "talar-first metatarsal angle" may mean an angle made by the center line of the first metatarsal and the center line of the talus, and within the specification of the present application, Meary's angle and It can be used in the same sense. At this time, the type of foot, based on the talus-first metatarsal angle, may be determined as flat foot, concave foot, and normal. More specifically, an individual having an angle between the talus and the first metatarsal bone exceeding -4 degrees may be an individual having concave crutches. That is, in an individual with concave crutches, the center line of the talus may be 4 degrees or more above the center line of the first metatarsal bone. Furthermore, an individual having an angle between the talus and the first metatarsal bone exceeding 4 degrees may be an individual having a flat foot. That is, in the individual with flat feet, the center line of the talus may exist 4 degrees or more below the center line of the first metatarsal bone. Thus, a normal individual may have a talus-first metatarsal angle of 4 degrees to -4 degrees.

At this time, the angle between the talus and the first metatarsal bone may be calculated by a predictive model.

On the other hand, the "center line of the talus" that forms the angle between the talus and the first metatarsal refers to the long axis of the talus connecting the posterior end of the upper articular surface of the talus body to the mid-length of the articular surface of the talus head with respect to the side of the foot. can do. Furthermore, "the center line of the first metatarsal bone may mean the long axis of the first metatarsal bone with respect to the side of the foot.

Meanwhile, as used herein, the term "feature point" is a point that exists on the boundary line of the talus region constituting the talus region and the first metatarsal region constituting the first metatarsal region, and is the center line of the talus and the center line of the first metatarsal bone. It may be an anatomically predetermined point for prediction.

According to a feature of the present invention, the feature points may exist in a plurality of pairs for each of the talus region and the first metatarsal region.

For example, the feature points determined in advance for the determination of the center line of the talus may be a pair of feature points consisting of the highest point on the boundary line between the talus and the tibia, and the highest point on the boundary line between the talus and the calcaneus. Furthermore, the feature points may be a pair of feature points consisting of the highest point on the boundary line between the talus and the scaphoid bone, and the lowest point on the boundary line between the talus and the scaphoid bone. That is, two pairs of feature points may exist on the boundary line of the talus, and feature points present in each pair may exist in opposite directions to the talus region. As a result, center points are determined for each of the two pairs of feature points, and the center line of the talus can be determined by connecting these center points.

Further, the predetermined feature points for the determination of the first metatarsal bone are at least two points on the boundary line of the first metatarsal region in contact with the instep, with respect to the region excluding the metatarsal head of the first metatarsal bone, and the first metatarsal direction. 1 It may be a plurality of pairs of feature points composed of at least two points on the boundary line of the metatarsal region. That is, two pairs of feature points may exist on the boundary line of the first metatarsal bone, and feature points present in each pair may exist in opposite directions with respect to the first metatarsal region. As a result, center points for each of the two pairs of feature points are determined, and the center lines of the first metatarsal bone can be determined by connecting these center points.

Meanwhile, the feature points for determining the center line of the metatarsal bone and the center line of the first metatarsal bone are not limited thereto, and may be set more variously as long as the long axis of the metatarsal bone and the long axis of the first metatarsal bone are determined.

As used herein, the term "region prediction model" may be a model configured to predict a talus region and a first metatarsal region with respect to a foot medical image.

For example, the region prediction model of the present invention may be a model trained to segment the talus region and the first metatarsal region with respect to the input foot side medical image.

Meanwhile, the region prediction model of the present invention may be a model learned through two learning steps in order to improve the accuracy of prediction of the talus region and the first metatarsal region.

For example, the region prediction model of the present invention receives images obtained by reducing the size of the original training foot medical image, the correct talus region, and the mask image in which the correct answer first metatarsal region is masked. 1 can be first learned to divide each metatarsal region. After the first learning, the region prediction model of the present invention receives ROI (region of interest) images cropped around the correct talus region and the correct first metatarsal region, and the bone region and the first metatarsal region respectively Can be secondarily learned to partition.

Accordingly, the region prediction model of the present invention increases the talus region and the first metatarsal region as the core region-centered learning to activate the prediction within the correct answer region where the actual talus or the first metatarsal bone exists. Can be divided by accuracy.

Meanwhile, the region prediction model of the present invention may be a model based on a Segnet network. However, it is not limited thereto. For example, the region prediction model of the present invention is VGG-16, DCNN (Deep Convolutional Neural Network) and ResNet DNN (Deep Neural Network), CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), RBM (Restricted Boltzmann). Machine), DBN (Deep Belief Network), SSD (Single Shot Detector) model, or a prediction model based on U-net.

As used herein, the term "centerline prediction model" may be a model trained to predict the centerline of the talus and the centerline of the first metatarsal within the input talus region and the first metatarsal region. Accordingly, the angle between the talus and the first metatarsal bone may be determined by calculating the angle formed by the center lines predicted by the center line prediction model.

Meanwhile, the center line of the talus and the center line of the first metatarsal bone are not limited to being determined by the center line prediction model, and may be determined within the talus region and the first metatarsal region in a variety of ways.

As used herein, the term "feature point prediction model" may be a model trained to predict feature points that exist in two pairs on a boundary line constituting the talus region and the first metatarsal region.

According to a feature of the present invention, the feature point prediction model may predict two pairs of feature points and determine a center point for each of the two pairs of feature points. As a result, the center line of the talus and the center line of the first metatarsal bone can be determined by the center point, and finally, the angle between the talus and the first metatarsal bone, which is a mark formed by these center lines, can be determined.

Meanwhile, the feature points determined in advance for each of the talus region and the first metatarsal region are not limited to those determined by the feature point prediction model. That is, the feature points that exist in two pairs for each of the talus region and the first metatarsal region may be determined on the boundary line of the talus region and the boundary line of the first metatarsal region by various methods.

Furthermore, each of the region prediction model, the feature point prediction model, and the centerline prediction model used in various embodiments of the present invention may be interpreted as a functional component for one prediction model. For example, a single prediction model according to an embodiment of the present invention includes a region prediction module for determining a talus region and a first metatarsal region for a foot medical image, a centerline for predicting a centerline within the talus region and the first metatarsal region. It may include a prediction module and a feature point prediction module that predicts feature points within the talus region and the first metatarsal region.

Hereinafter, a device for evaluating a type of foot according to an embodiment of the present invention will be described with reference to FIG. 1.

First, a device for evaluating a type of foot according to an embodiment of the present invention will be described in detail with reference to FIG. 1. 1 shows the configuration of a device for evaluating a foot type according to an embodiment of the present invention. In this case, the medical image of the foot is described as an example, but is not limited thereto.

Referring to FIG. 1, the device for evaluating the type of foot 100 includes a receiving unit 110, an input unit 120, an output unit 130, a storage unit 140, and a processor 150.

Specifically, the receiving unit 110 may be configured to receive a medical image of an individual's foot, for example, a medical image of a side of the foot. Preferably, the medical image of the foot acquired by the receiving unit 110 may be a radiographic image of the side of the foot, but is not limited thereto.

Meanwhile, the foot medical image acquired through the receiving unit 110 may include a talus region and a first metatarsal region. Furthermore, the foot medical image may further include a predetermined feature point on the boundary of the talus region and a predetermined characteristic point on the boundary line of the first metatarsal region.

The input unit 120 may be configured to set the device 100 for evaluating the type of foot, and to receive a selection for a specific region of the foot medical image received through the above-described receiving unit 110. Meanwhile, the input unit 120 may be a keyboard, a mouse, and a touch screen panel, but is not limited thereto.

Meanwhile, the output unit 130 may visually display the foot medical image obtained from the receiving unit 110. Further, the output unit 130 visually determines the talus region and the first metatarsal region, the feature point, the center line of the talus, the center line of the first metatarsal bone, and further, the talus-first metatarsal angle determined by the processor 150 in the foot medical image. It can be configured to indicate as. Further, the output unit 130 may be configured to further output the predicted foot type based on the talus-first metatarsal angle determined by the processor 150.

The storage unit 140 may be configured to store a foot medical image of an object acquired through the receiving unit 110 and to store an instruction of the device 100 for evaluating a foot type set through the input unit 120. Further, the storage unit 140 is configured to store results classified or predicted by the processor 150 to be described later. However, it is not limited to the above, and the storage unit 140 may store various pieces of information determined by the processor 150 to predict the talus-first metatarsal angle, and further, the type of foot.

Meanwhile, the processor 150 may be a component for providing an accurate prediction result for the device 100 for evaluating the foot type. At this time, the processor 150 predicts the talus region and the first metatarsal region with respect to the foot medical image, and based on the prediction model configured to extract the center line of the talus and the center line of the first metatarsal bone, the talus-formulae, which is the angle formed by these center lines 1 Can be configured to determine the angle between metatarsal bones.

According to a feature of the present invention, the processor 150 predicts the center line of the talus and the center line of the first metatarsal bone in the talus region and the first metatarsal region by the center line prediction model, and the predicted center line of the talus and the first metatarsal bone It may be configured to calculate an angle between the talus and the first metatarsal bone by the center line.

According to another feature of the present invention, the processor 150 predicts feature points existing in a plurality of pairs on the boundary line constituting the talus region and the first metatarsal region by the feature point prediction model, and determines a central point for each of the two pairs of feature points. It may be configured to determine and predict the center line of the talus and the center line of the first metatarsal bone based on the center point, and calculate the angle between the talus and the first metatarsal bone.

According to another feature of the present invention, the processor 150 may be based on a region prediction model having improved talus region and first metatarsal region prediction accuracy through two learning steps.

For example, the processor 150 receives the original training foot medical image and images obtained by reducing the size of the mask image in which the correct talus region and the first metatarsal region are masked, and the talus region and the first metatarsal region are After first learning to segment each, ROI images cropped around the talus region and the correct answer first metatarsal region are input, and the secondly learned prediction model is used to segment each of the bone region and the first metatarsal region. Can be based.

Meanwhile, the processor 150 may be based on a region prediction model configured to segment a specific region based on an image. For example, processor 150 may be based on a Segnet network. However, it is not limited thereto, and the processor 150 is VGG-16, DCNN (Deep Convolutional Neural Network) and ResNet DNN (Deep Neural Network), CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), RBM (Restricted Boltzmann). Machine), DBN (Deep Belief Network), SSD (Single Shot Detector) model, or U-net-based prediction model.

Hereinafter, a method of predicting a foot type according to an embodiment of the present invention will be described in detail with reference to FIGS. 2A and 2D and FIGS. 3 to 5. FIG. 2A is a diagram illustrating a method of predicting foot type according to an exemplary embodiment of the present invention. It shows the procedure. 2B is an exemplary diagram illustrating a procedure for predicting a talus region and a first metatarsal region using a region prediction model used in various embodiments of the present invention. FIG. 3 exemplarily shows a procedure for predicting each between the talus and the first metatarsal bone using a centerline prediction model used in various embodiments of the present invention. 4 and 5 exemplarily illustrate a procedure of predicting each between the talus and the first metatarsal bone using a feature point prediction model used in various embodiments of the present invention.

First, referring to FIG. 2A, a procedure for evaluating a foot type according to an embodiment of the present invention is as follows. First, a foot medical image of an individual is received (S210). Next, with respect to the foot medical image, the talus region and the first metatarsal region are predicted using a region prediction model configured to predict the talus region and the first metatarsal region (S220). Next, the talus-first metatarsal angle is determined based on the centerline for each of the talus region and the first metatarsal region (S230), and the determined foot type based on the predicted talus-first metatarsal angle is evaluated ( S240).

More specifically, in the step of receiving the medical image of the foot (S210), a medical image of the foot for the suspected object with flat foot or concave foot may be received. In this case, the medical image of the foot may be a medical radiographic image of the side of the foot, but is not limited thereto.

According to an embodiment of the present invention, a plurality of foot medical images may be received in the step S210 of receiving the foot medical image. For example, in the step of receiving the foot medical image (S210), a plurality of foot medical images may be obtained as the foot medical image captured in real time according to the driving of an image diagnosis device such as a foot medical image capturing unit can be obtained. Can be obtained.

Next, in the step of predicting the talus region and the first metatarsal region (S220), the talus region and the first metatarsal region may be predicted by the region prediction model.

For example, referring to FIG. 2B together, in the step of predicting the region and the first metatarsal region (S220), when the foot lateral medical image 212 is input to the region prediction model 222, the talus region 224 and the first metatarsal region One metatarsal region 226 may be determined. Accordingly, the talus region 224 and the first metatarsal region 226 may be output as a result of predicting the talus region and the first metatarsal region (S220 ).

Next, in the step of determining the talus-first metatarsal angle (S230), the talus-first metatarsal angle is determined based on the center line of the talus and the center line of the first metatarsal bone in the talus region and the first metatarsal region. I can.

According to an embodiment of the present invention, before the step of determining the talus-first metatarsal angle (S23), the center line of the talus and the center line of the first metatarsal bone in the talus region and the first metatarsal region are Can be determined.

For example, referring to FIG. 3 together, when the talus region 224 and the first metatarsal region 226 predicted for the foot side medical image 212 are input to the centerline prediction model 312, the centerline of the talus ( 322) and the center line 324 of the first metatarsal bone may be output. As a result, in the step of determining the talus-first metatarsal angle (S230), the talus centerline 322 predicted by the centerline prediction model 312 and the centerline 324 of the first metatarsal bone respectively form the talus-system 1 The metatarsal angle 332 can be determined as '6°'. As a result, in the foot type evaluation step (S240) to be described later, the foot type may be determined as'flat foot' based on the talus-first metatarsal angle 332. Furthermore, information 342 about the type of foot, including the talus-first metatarsal angle 332 and the determined type of foot, may be provided.

According to another embodiment of the present invention, prior to the step of determining the talus-first metatarsal angle (S230), predetermined feature points in the talus region and the first metatarsal region are respectively determined by the feature point prediction model, and Based on the center line of the talus and the center line of the first metatarsal bone can be determined.

For example, further referring to FIG. 4, the talus region 224 and the first metatarsal region 226 predicted for the lateral foot medical image 212 are input to the feature point prediction model 432. At this time, a standard foot lateral medical image 412 including a predetermined standard feature point 424 on the boundary line of the standard talus region and the standard first metatarsal region 422 is input together into the feature point prediction model 432. At this time, the standard feature points 424 may exist in two pairs. Then, the boundary line of the talus region 224 and the boundary line of the first metatarsal region 226 are extracted, and the standard talus region and the standard first metatarsal region 422 of the respective boundary lines and the standard foot lateral medical image 412 The boundary lines of are matched, and based on the distance between the boundary lines of each of the two pairs of standard feature points 424 and the boundary line of the talus region 224 and the boundary line of the first metatarsal region 226, two pairs of feature points 434 are newly determined. . In this case, each of the two pairs of feature points 434 may be points at the shortest distance from each of the two pairs of standard feature points 424 on the boundary line of the talus region 224 and the boundary line of the first metatarsal region 226. Next, the center points 442 of the two pairs of feature points 434 output by the feature point prediction model 432 are determined for each pair, and the two center points 442 determined for each of the two pairs of feature points 434 ) To determine the center line 322 of the talus and the center line 324 of the first metatarsal bone. As a result, in the step of determining the talus-first metatarsal angle (S230), the talus-first metatarsal angle 332 formed by each of the predicted center line 322 of the talus and the center line 324 of the first metatarsal bone is It can be determined as '6°'. As a result, in the foot type evaluation step (S240) to be described later, the foot type may be determined as'flat foot' based on the talus-first metatarsal angle 332.

According to another embodiment of the present invention, prior to the step of determining the talus-first metatarsal angle (S230), predetermined feature points in the talus region and the first metatarsal region are each determined by the feature point prediction model, and these Based on the center line of the talus and the center line of the first metatarsal bone may be determined.

For example, referring to FIG. 5 together, the talus region 224 and the first metatarsal region 226 predicted for the foot lateral medical image 212 are input to the feature point prediction model 432. At this time, a plurality of standard foot lateral medical images 412 including a predetermined standard feature point 424 on the boundary line of the standard talus region and the standard first metatarsal region 422 are input together into the feature point prediction model 432. . At this time, the plurality of standard foot lateral medical images 412 include each of the talus region and the first metatarsal region, each of the talus region and the first metatarsal region determined in advance by the region prediction model based on an ICP (Iterative closest point) method, and 422) may be a medical image of the foot whose similarity is determined to be high after the registration is performed. Meanwhile, the standard feature points 424 may exist in two pairs for each of the plurality of standard foot lateral medical images 412. Then, based on the distance between the boundary line of each of the two pairs of standard feature points 424 for each of the plurality of standard foot lateral medical images 412 and the boundary line of the talus region 224 and the boundary line of the first metatarsal region 226, a plurality of The feature points 512 of the pair of are newly determined. In this case, each of the plurality of pairs of feature points 512 may be determined on the boundary line of the talus region 224 and the boundary line of the first metatarsal region 226. Next, a plurality of center points 522 are determined for each of the plurality of pairs of feature points 512 predicted by the feature point prediction model 432. That is, the plurality of center points 522 may be determined in the talus region 224 and the first metatarsal region 226 by the same number as the number of feature point pairs constituting the plurality of pairs of feature points 512. Next, the centroid 532 of the plurality of center points 522 is determined. As a result, two centroids 532, which are two central points for each of the talus region 224 and the first metatarsal region 226, are determined, and the center line 322 of the talus by the two centroids 532 and The centerline 324 of the first metatarsal bone may be determined, respectively.

Referring back to FIG. 2A, finally, in the step of evaluating the type of the foot (S240), the type of the individual's foot may be determined based on the angle between the talus and the first metatarsal bone.

According to another feature according to an embodiment of the present invention, in the step of evaluating the type of the individual's foot (S240), when the angle between the talus-first metatarsal bone exceeds -4 degrees, it is evaluated as a concave foot, or the talus-agent 1 If the angle between metatarsal bones exceeds 4 degrees, it can be evaluated as flat feet.

According to another embodiment of the present invention, in the step of evaluating the type of the individual's foot (S240), the evaluation of the severity of the flat foot of the individual may be further performed. More specifically, in the step of evaluating the type of the individual's foot (S240), when the talus-first metatarsal angle is 4 degrees to 15, it is evaluated as mild flatfoot, or the talus-first metatarsal angle If this is 15 degrees to 30 degrees, it is evaluated as moderate (moderate) flat feet, or if the angle between the talus-first metatarsal bone is more than 30 degrees, it can be evaluated as severe flat feet.

By the method of predicting the type of foot according to the present invention according to the various embodiments above, the measurement result of the talus-first metatarsal angle with high reproducibility based on the foot medical image obtained from the individual and the result of the accurate evaluation of the foot type accordingly Can provide.

Meanwhile, the method of predicting the type of foot according to the present invention of the present invention is not limited to the above. For example, in the foot type prediction method according to the present invention, each of the region prediction model, the centerline prediction model, and the feature point prediction model may be configured to independently predict the foot type, and are configured to predict the foot type with more various combinations. It could be.

Furthermore, in the foot type prediction method according to the present invention, each of a region prediction model, a feature point prediction model, and a centerline prediction model may be applied as a functional component for one prediction model. For example, in the foot type prediction method according to an embodiment of the present invention, a region prediction module for determining a talus region and a first metatarsal region for a foot medical image, and predicting a centerline within the talus region and the first metatarsal region. A single model including a centerline prediction module and a feature point prediction module that predicts feature points within the talus region and the first metatarsal region may be applied.

Hereinafter, a method of learning a region prediction model used in various embodiments of the present invention will be described with reference to FIGS. 6A to 6C. 6A and 6B illustrate medical image data of the foot for training in a region prediction model used in various embodiments of the present invention. 6C illustrates an exemplary configuration of a region prediction model used in various embodiments of the present invention.

In this case, the training foot medical image used for learning of the region prediction model may be a leg side radiographic image taken with respect to the side of the foot of the individual. Furthermore, the region prediction model may be trained to divide the talus region and the first metatarsal region through two learning steps. Meanwhile, the learning method of the region prediction model is not limited thereto, and the learning of the region prediction model according to the present invention may be performed in various ways based on more various foot medical images.

Referring to FIG. 6A(a), in the first learning step for segmentation of the talus region of the region prediction model of the present invention, the size of the original training foot lateral medical image having a size of 2981 x 2106 is 1/6. A 497 x 351 medical image on the side of the foot and a mask image in which the talus region is masked may be used as an image for learning.

Meanwhile, in the first-learned region prediction model, when the talus region, which is a region to be segmented, is not clearly classified as a peripheral region and an erection, or if the resolution of the training foot medical image is high, the accuracy of classification may be slightly degraded. Accordingly, the inventors of the present invention attempted to further apply an ROI image including the talus region at a high rate to secondary learning.

Referring to FIG. 6A(b), in the second learning step for segmenting the talus region of the region prediction model of the present invention, the ROI image including the talus region having a size of 600 x 600 among the lateral medical images of the foot and the talus region are masked. The resulting ROI mask image can be used as an image for learning.

The region prediction model of the present invention, which has been trained to segment the talus region through a two-step learning process, can segment the talus region with high accuracy with respect to the input new foot medical image.

Referring to FIG. 6B (a), in the first learning step for segmentation of the first metatarsal region of the region prediction model of the present invention, 1/6 of the original training foot lateral medical image having a size of 2981 x 2106 (Raw image) A medical image of the side of the foot having a size of 497 x 351 and a mask image in which the first metatarsal region is masked may be used as an image for learning.

On the other hand, in the first-learned region prediction model, when the first metatarsal region, which is a region to be segmented, is not clearly classified as a peripheral region and an erection, or if the resolution of the training foot medical image is high, the accuracy of classification may be slightly degraded. Accordingly, the inventors of the present invention attempted to further apply an ROI image including the first metatarsal region at a high rate to secondary learning.

Referring to FIG. 6B (b), in the second learning step for segmenting the first metatarsal region of the region prediction model of the present invention, an ROI image including a first metatarsal region having a size of 600 x 460 among the lateral medical images of the foot, And an ROI mask image in which the first metatarsal region is masked may be used as an image for training.

The region prediction model of the present invention, which is learned to segment the first metatarsal region through a two-step learning process, can segment the first metatarsal region with high accuracy with respect to the input new foot medical image.

That is, in the region prediction model of the present invention, in the prediction process, the ROI is set for the foot medical image, cropping to include only the core region, and pre-processing of the image such as enlargement and transformation of the foot medical image may be omitted. have. Nevertheless, the region prediction model of the present invention may be superior to other prediction models in predicting the talus region and the first metatarsal region.

Referring to FIG. 6C, a structure of a region prediction model based on a Segnet network is exemplarily disclosed. In this case, the region prediction model of the present invention may be a Segnet model initialized using VGG-16 weights. More specifically, in the region prediction model of the present invention, the talus region and the first metatarsal region, which are the core regions of the lateral medical image for training, are divided through various layers after receiving a medical image of the side of the foot for training. 1 It may be composed of a plurality of layers configured to classify the metatarsal region. In particular, the last layers of the prediction model of the present invention are one Unpool (un pooling) layer, two convolutional layers, two BN (batch normalization) layers, two ReLUs (rectifier linear units), and finally It may be composed of a plurality of layers of a softmax layer for classifying the talus region and the first metatarsal region for the foot medical image. In this case, the talus region and the first metatarsal region may be predicted in units of pixels.

Meanwhile, in the region prediction model, the filter size is 5, the number of filters is [32, 32, 64], the pooling size is 2, the number of max epochs is 120, and the momentum Is 0.9, the initial learning rate can have a parameter of 0.01. Meanwhile, the region prediction model of the present invention may be trained using a 497 x 351 sized foot lateral medical image, a 600 x 600 sized talus ROI image and a 600 x 460 sized first metatarsal ROI image.

The structure, parameters, and learning methods of the domain prediction model of the present invention are not limited thereto.

Example 1: Evaluation of the region prediction model of the present invention_Evaluation of region division

In Example 1 below, evaluation results of region division of a region prediction model used in various embodiments of the present invention will be described with reference to FIGS. 7A to 7F. At this time, the Segnet-based prediction model learned by the two-step learning step was used as the prediction model, and a total of 120 foot lateral radiographic images of 60 left foot and 60 right foot were used for this evaluation. On the other hand, in this evaluation, the similarity between the talus region or the first metatarsal region predicted by the region prediction model and the correct talus region or the first metatarsal region masked as the correct answer region was evaluated. At this time, the degree of similarity was calculated based on the overlap rate of the prediction region and the correct answer region.

7A to 7F illustrate evaluation results of the talus region and the first metatarsal region divided by the region prediction model used in various embodiments of the present invention.

First, referring to FIG. 7A, the talus region predicted by the region prediction model of the present invention for 120 foot medical images is shown. With further reference to FIGS. 7B and 7C, the average similarity coefficient for the talus region and the correct talus region predicted by the region prediction model of the present invention is 0.9653. In particular, the maximum similarity is 0.9804, and the region prediction model of the present invention is shown to predict the talus region at a similar level of about 98% to the correct answer talus region. This may mean that the region prediction model of the present invention divides the talus region with very high accuracy in the foot medical image.

Referring to FIG. 7D, a first metatarsal region predicted by the region prediction model of the present invention for a total of 120 foot medical images is shown. With further reference to FIGS. 7E and 7F, the average similarity coefficient for the first metatarsal region and the first metatarsal region predicted by the region prediction model of the present invention is 0.9524. In particular, the maximum similarity is 0.9765, and the region prediction model of the present invention is shown to predict the first metatarsal region at a level similar to that of the correct talus region, about 97.7%. This may mean that the region prediction model of the present invention divides the first metatarsal region with very high accuracy in the foot medical image.

That is, according to the results of Example 1 above, the region prediction model of the present invention, which is firstly learned with a medical image for the entire side of the foot, and secondly learned with an ROI image including only the talus region or the first metatarsal region, It appears to be predicted by segmenting the talus region and the first metatarsal region with high accuracy. Accordingly, the region prediction model of the present invention may have superior predictive performance of the core region than the prediction model learned only with the medical image of the entire side of the foot.

Example 2: Evaluation of center line prediction

In Example 2 below, the evaluation results for the prediction of the centerline of the talus and the centerline of the first metatarsal bone will be described with reference to FIGS. 8A to 8D.

At this time, the center line of the talus and the center line of the first metatarsal bone were predicted within the talus region and the first metatarsal region predicted by the Segnet-based predictive model learned by the second stage of learning. More specifically, in this evaluation, a feature point prediction model that predicts feature points that exist in two pairs on the boundary line of the talus region constituting the talus region, the boundary line of the first metatarsal region constituting the first metatarsal region, the center line of the talus, and the first metatarsal bone Evaluation of a centerline prediction model configured to predict a centerline of may be performed.

This evaluation was performed while calculating the angle difference between the predicted talus centerline and the first metatarsal centerline, the correct answer talus centerline, and the correct answer first metatarsal centerline for each of 60 left and 60 right foot images.

8A to 8D illustrate the predicted talus centerline and first metatarsal centerline, and evaluation results of the talus angle and first metatarsal angle formed by these centerlines according to various embodiments of the present invention.

First, referring to 8a, the predicted mid points of each of the plurality of pairs of feature points predicted for the talus region, and the ground truth of each of the plurality of pairs of correct answer feature points that are predetermined on the boundary line of the talus region are determined. Are shown together. At this time, the feature points of a plurality of pairs determined in advance for the determination of the center line of the talus are the highest point on the boundary line between the talus and tibia, the pair of characteristic points composed of the highest point on the boundary line between the talus and calcaneus, the highest point on the boundary line between the talus and scaphoid, It can be composed of a pair of feature points consisting of the lowest point on the boundary line of the scaphoid bone. In the prediction model of the present invention, a centroid for a center point of a plurality of feature point pairs predicted for a single foot side medical image may be determined as a final center point. As a result, centroids are determined, and as these centroids are connected, the center line of the talus in the lateral medical image of the foot can be determined.

More specifically, referring to (a) of FIG. 8B, the centerline of the talus determined for each of the 60 side images of the right foot is the difference between the centerline of the talus and the average of 1.714 degrees connecting the center points of each of the plurality of pairs of correct answer feature points. Appears to be. Further, referring to (b) of FIG. 8B, the centerline of the talus determined for the lateral image of the left foot of 60 sheets shows that there is a difference of 1.652 degrees on average from the centerline of the correct answer talus.

These results may mean that the prediction model of the present invention, in particular, the centerline prediction model predicts the centerline of the talus forming an angle between the talus and the first metatarsal at an excellent level.

Next, referring to 8c, the predicted mid points of each of the plurality of pairs of feature points predicted for the first metatarsal region, and the center points of each of the plurality of pairs of correct answer feature points that are predetermined on the boundary line of the first metatarsal region (ground truth) is shown together. At this time, the plurality of pairs of feature points predetermined for the determination of the center line of the first metatarsal bone are at least two on the boundary line of the first metatarsal region in contact with the instep, with respect to the region excluding the metatarsal head of the first metatarsal bone. The point may be a plurality of pairs of feature points composed of at least two points on the boundary line of the first metatarsal region in the direction of the sole of the foot. In the prediction model of the present invention, a centroid for a center point of a plurality of feature point pairs predicted for a single foot lateral medical image may be determined as a final center point. As a result, the centroids are determined, and as these centroids are connected, the center line of the first metatarsal bone may be determined in the lateral medical image of the foot.

More specifically, referring to FIG. 8D (a), the center line of the first metatarsal bone determined for each of the 60 side images of the right foot is the center line of the first metatarsal bone that connects the center points of each of the plurality of pairs of the correct answer feature points. There appears to be a difference of 1.257 degrees on average. Further, referring to (b) of FIG. 8D, the centerline of the first metatarsal bone determined for the left foot lateral image of 60 sheets is shown to have a difference of 1.485 degrees on average from the centerline of the correct answer first metatarsal bone.

These results may mean that the prediction model of the present invention, in particular, the centerline prediction model predicts the centerline of the first metatarsal bone forming an angle between the talus and the first metatarsal bone at an excellent level.

Example 3: Evaluation of the first metatarsal angle

In Example 3 below, the evaluation results for the prediction of the talus-first metatarsal angle will be described with reference to FIGS. 9A and 9B.

In this evaluation, the difference between the final calculated talus-first metatarsal angle and the correct talus-first metatarsal angle formed by the center line of the correct talus and the center line of the correct talus and the first metatarsal were further calculated.

9A and 9B illustrate evaluation results for the talus-first metatarsal angle calculated according to various embodiments of the present disclosure.

Referring to (a) of FIG. 9A, the talus-first metatarsal angle determined for each of the 60 lateral images of the right foot appears to have an average difference of 1.998 degrees from the correct answer talus-first metatarsal angle. In particular, further referring to (b) of FIG. 9A, it appears that the angle between the talus and the first metatarsal is predicted at a level having a difference of only 0.046 degrees from the angle between the correct talus and the first metatarsal.

Referring to (a) of FIG. 9B, the talus-first metatarsal angle determined for each of the 60 left foot lateral images shows that there is an average difference of 2.283 degrees from the correct answer talus-first metatarsal angle. In particular, further referring to (b) of FIG. 9B, it appears that the angle between the talus and the first metatarsal is predicted at a level having a difference of only 0.019 degrees from the angle between the correct talus and the first metatarsal.

That is, according to the results of Example 3 above, the prediction model of the present invention, in particular, the centerline prediction model predicts the centerline of the first metatarsal bone forming the talus-first metatarsal angle at an excellent level, and thus, with very high accuracy, the talus- It appears that the angle between the first metatarsal bones can be calculated.

According to the results of Examples 1 to 3, the foot type prediction device according to various embodiments based on various prediction models predicts the talus-first metatarsal angle with high accuracy, and the predicted talus-first metatarsal angle The type of foot can be provided on the basis of. For example, the foot type prediction device of the present invention, when it is determined that the angle between the talus and the first metatarsal bone exceeds -4 degrees by the prediction model, that is, the center line of the talus is present at least 4 degrees above the center line of the first metatarsal bone. If so, the individual's foot type can be determined as a concave foot. Furthermore, when it is determined by the prediction model that the angle between the talus and the first metatarsal is more than 4 degrees, that is, when the center line of the talus is more than 4 degrees below the center line of the first metatarsal, the foot type of the individual can be determined as flat feet. have.

Accordingly, the present invention has the effect of providing a measurement result of the talus-first metatarsal angle with high reproducibility and an evaluation result of an accurate foot type accordingly.

Accordingly, the present invention addresses the limitations and problems of the foot type evaluation system based on a conventional medical image that can be measured differently according to the anatomy of the foot, which is different for each individual in the talus-first metatarsal angle, and the skill level of a medical practitioner. There is an effect that can be overcome.

In particular, the present invention is a region capable of predicting these regions by accurately segmenting the talus region or the first metatarsal region even when the resolution of the foot medical image is high or the shape to be segmented and the brightness value of the surrounding region are not clearly classified. There is an effect that the foot type can be more accurately evaluated by using a predictive model.

Although the embodiments of the present invention have been described in more detail with reference to the accompanying drawings, the present invention is not necessarily limited to these embodiments, and various modifications may be made without departing from the spirit of the present invention. Accordingly, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but to explain the technical idea, and the scope of the technical idea of the present invention is not limited by these embodiments. Therefore, it should be understood that the embodiments described above are illustrative and non-limiting in all respects. The scope of protection of the present invention should be construed by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.

100: device for evaluating the type of foot
110: receiver
120: input
130: output
140: processor
150: storage unit
212: Lateral medical imaging of the foot
222: area prediction model
224: talus area
226: first metatarsal region
312: Centerline prediction model
322: center line of the talus
324: center line of the first metatarsal bone
332: talus-first metatarsal angle
342: Information on the type of foot
412: Standard foot lateral medical imaging
422: standard first metatarsal region
424: standard feature point
432: Feature point prediction model
434: two pairs of feature points
442: center point
512: feature points of multiple pairs
522: center point of plural
532: centroid

Claims (25)

As a method of evaluating the type of foot implemented by a processor,
Receiving a medical image of the individual's foot;
Using a region prediction model configured to predict a talus region and a first metatarsal region based on the foot medical image, predicting the talus region and the first metatarsal region within the foot medical image step;
Predicting a center line of the talus and a center line of the first metatarsal bone, respectively, using a centerline prediction model configured to predict a centerline of the talus and a centerline of the first metatarsal for each of the talus region and the first metatarsal region;
Determining a talus-first metatarsal angle based on the predicted center line of the talus and the center line of the first metatarsal bone, and
Comprising the step of evaluating the type of the individual's foot based on the talus-first metatarsal angle, foot type evaluation method. delete As a method of evaluating the type of foot implemented by a processor,
Receiving a medical image of the individual's foot;
Using a region prediction model configured to predict a talus region and a first metatarsal region based on the foot medical image, predicting the talus region and the first metatarsal region within the foot medical image step;
Predicting a feature point for each of the talus region and the first metatarsal region using a feature point prediction model configured to predict a feature point predetermined for each of the talus region and the first metatarsal region;
Predicting a center line of the talus and a center line of the first metatarsal bone, respectively, based on the feature points;
Determining a talus-first metatarsal angle based on the predicted center line of the talus and the center line of the first metatarsal bone, and
Comprising the step of evaluating the type of the individual's foot based on the talus-first metatarsal angle, foot type evaluation method. The method of claim 3,
The predetermined feature points exist in two pairs,
Using the feature point prediction model, predicting feature points for each of the talus region and the first metatarsal region,
Predicting, respectively, two pairs of predetermined feature points on the boundary line of the talus region and the boundary line of the first metatarsal region using the feature point prediction model, and
Determining a central point of each of the predicted two pairs of feature points for each of the talus region and the first metatarsal region,
Predicting the center line of the talus and the center line of the first metatarsal bone, respectively, based on the feature points,
And predicting, respectively, a center line of the talus and a center line of the first metatarsal bone based on the center point. The method of claim 4,
Receiving a standard talus image and a standard first metatarsal image including two predetermined pairs of standard feature points on a boundary line of the talus region, further comprising:
Predicting each of the two pairs of feature points,
Calculating a boundary line of the talus region and a distance of each of two pairs of standard feature points in the standard talus image, and a distance of each of the boundary line of the first metatarsal region and two pairs of standard feature points in the standard first metatarsal image, and
And predicting each of the two pairs of feature points on a boundary line of the talus region and a boundary line of the first metatarsal region based on the respective distances. The method of claim 5,
The standard talus image and the standard first metatarsal image,
Plural,
The step of calculating each of the distances,
The boundary line of the talus region and the distance of each of the two pairs of standard feature points in the plurality of standard talus images, and the distance of each of the two pairs of standard feature points in the boundary line of the first metatarsal region and the plurality of standard first metatarsal images And calculating a plurality of the standard talus images and a plurality of the standard first metatarsal images,
Predicting each of the two pairs of feature points,
Predicting two pairs of feature points on the boundary line of the talus region and the boundary line of the first metatarsal region, respectively, based on the respective distances calculated for each of the plurality of standard talus images and the plurality of standard first metatarsal images. A method of assessing the type of foot, comprising the steps of. The method of claim 6,
Determining a central point of each of the two pairs of feature points for each of the talus region and the first metatarsal region,
According to the plurality of standard talus images and the plurality of standard first metatarsal images, a center point for each of the two pairs of feature points predicted on the boundary line of the talus region and the boundary line of the first metatarsal region is defined as the talus region and the Determining for each of the first metatarsal regions,
Predicting the center line of the talus and the center line of the first metatarsal bone, respectively, based on the feature points,
Centroids of the central points determined for each of the talus region and the first metatarsal region according to the plurality of standard talus images and the plurality of standard first metatarsal images are obtained for each of the talus region and the first metatarsal region. Determining, and
And predicting, respectively, a center line of the talus and a center line of the first metatarsal bone based on the centroid determined for each of the talus region and the first metatarsal region. The method of claim 4,
The method of evaluating foot type further comprising the step of providing the predicted feature points of the two pairs. As a method of evaluating the type of foot implemented by a processor,
Receiving a medical image of the individual's foot;
Using a region prediction model configured to predict a talus region and a first metatarsal region based on the foot medical image, predicting the talus region and the first metatarsal region within the foot medical image step;
Determining a talus-first metatarsal angle based on the predicted centerline for each of the talus region and the first metatarsal region;
Evaluating the type of the individual's foot based on the talus-first metatarsal angle, and
Comprising the step of providing the talus-first metatarsal angle, foot type evaluation method. As a method of evaluating the type of foot implemented by a processor,
Receiving a medical image of the individual's foot;
Using a region prediction model configured to predict a talus region and a first metatarsal region based on the foot medical image, predicting the talus region and the first metatarsal region within the foot medical image step;
Determining a talus-first metatarsal angle based on the predicted centerline for each of the talus region and the first metatarsal region, and
Including the step of evaluating the type of the individual's foot based on the talus-first metatarsal angle,
Evaluating the type of the individual's foot,
If the angle between the talus and the first metatarsal bone is less than -4 degrees, it is evaluated as a concave foot, or
If the talus-the first metatarsal angle is greater than 4 degrees, comprising the step of evaluating as flat feet, foot type evaluation method. The method of claim 1,
Evaluating the type of the individual's foot,
Further comprising the step of assessing the severity of the flatfoot of the subject,
The step of evaluating the severity of the flatfoot of the individual,
If the talus-first metatarsal angle is 4 to 15 degrees, it is evaluated as mild flatfoot, or
If the angle between the talus and the first metatarsal is 15 degrees to 30 degrees, it is evaluated as a moderate flat foot, or
If the angle between the talus-first metatarsal bone is greater than 30 degrees, the method of evaluating the type of foot comprising the step of evaluating as a severe flat foot. As a method of evaluating the type of foot implemented by a processor,
Receiving a medical image of the individual's foot;
Using a region prediction model configured to predict a talus region and a first metatarsal region based on the foot medical image, predicting the talus region and the first metatarsal region within the foot medical image step;
Determining a talus-first metatarsal angle based on the predicted centerline for each of the talus region and the first metatarsal region, and
Including the step of evaluating the type of the individual's foot based on the talus-first metatarsal angle,
The region prediction model,
Receiving a training foot medical image including a predetermined correct answer talus region and a correct answer first metatarsal region with respect to the foot of the standard individual, and
A method for evaluating a foot type, which is a model trained to predict the talus region and the first metatarsal region through the step of predicting the talus region and the first metatarsal region within the training foot medical image. The method of claim 12,
After the step of predicting the talus region and the first metatarsal region in the training foot medical image,
Calculating a ratio of an overlapping area in which each of the correct answer talus region and the correct answer first metatarsal region overlap with respect to each of the talus region and the first metatarsal region predicted by the region prediction model, and
The method further comprising evaluating the area prediction model based on the ratio of the overlapping area. The method of claim 12,
The training foot medical image,
A method for evaluating a foot type, including a medical image including only the correct answer talus region and a medical image including only the correct answer first metatarsal region in the training foot medical image. A receiving unit configured to receive a medical image of the foot of the individual, and
A processor connected to communicate with the receiving unit,
The processor,
Using a region prediction model configured to predict a talus region and a first metatarsal region based on the foot medical image, the talus region and the first metatarsal region are predicted in the foot medical image, ,
Using a centerline prediction model configured to predict a centerline of the talus and a centerline of the first metatarsal for each of the talus region and the first metatarsal region, respectively, predicting the centerline of the talus and the centerline of the first metatarsal bone,
Determine the talus-first metatarsal angle based on the predicted center line of the talus and the center line of the first metatarsal,
The device for evaluating the type of the foot, configured to evaluate the type of the individual's foot based on the talus-first metatarsal angle. delete A receiving unit configured to receive a medical image of the foot of the individual, and
A processor connected to communicate with the receiving unit,
The processor,
Using a region prediction model configured to predict a talus region and a first metatarsal region based on the foot medical image, the talus region and the first metatarsal region are predicted in the foot medical image, ,
Using a feature point prediction model configured to predict a predetermined feature point for each of the talus region and the first metatarsal region, predicting feature points for each of the talus region and the first metatarsal region,
Predicting the center line of the talus and the center line of the first metatarsal bone, respectively, based on the feature points,
Determine the talus-first metatarsal angle based on the predicted center line of the talus and the center line of the first metatarsal,
The device for evaluating the type of the foot, configured to evaluate the type of the individual's foot based on the talus-first metatarsal angle. The method of claim 17,
The predetermined feature points exist in two pairs,
The processor,
Using the feature point prediction model, two pairs of predetermined feature points are respectively predicted on the boundary line of the talus region and the boundary line of the first metatarsal region, and the center points of each of the predicted two pairs of feature points are determined as the talus region and the first metatarsal region. 1 Determine for each metatarsal region,
A device for evaluating the type of foot, further configured to predict each of the center line of the talus and the center line of the first metatarsal bone based on the center point The method of claim 18,
The receiving unit is further configured to receive a standard talus image and a standard first metatarsal image including two predetermined pairs of standard feature points on a boundary line of the talus region,
The processor,
A boundary line of the talus region and a distance of each of two pairs of standard feature points in the standard talus image, and a distance of each of the boundary line of the first metatarsal region and two pairs of standard feature points in the standard first metatarsal image are calculated, and each of the The device for evaluating a foot type, further configured to predict each of the two pairs of feature points on a boundary line of the talus region and a boundary line of the first metatarsal region based on a distance of. The method of claim 19,
The standard talus image and the standard first metatarsal image are plural,
The processor,
The boundary line of the talus region and the distance of each of the two pairs of standard feature points in the plurality of standard talus images, and the distance of each of the two pairs of standard feature points in the boundary line of the first metatarsal region and the plurality of standard first metatarsal images , Based on the respective distances calculated for the plurality of standard talus images and the plurality of standard first metatarsal images, and calculated for each of the plurality of standard talus images and the plurality of standard first metatarsal images, the A device for evaluating a foot type, further configured to predict each of two pairs of feature points on a boundary line of a talus region and a boundary line of the first metatarsal region. The method of claim 20,
The processor,
According to the plurality of standard talus images and the plurality of standard first metatarsal images, a center point for each of the two pairs of feature points predicted on the boundary line of the talus region and the boundary line of the first metatarsal region is defined as the talus region and the Determine for each of the first metatarsal regions,
Centroids of the central points determined for each of the talus region and the first metatarsal region according to the plurality of standard talus images and the plurality of standard first metatarsal images are obtained for each of the talus region and the first metatarsal region. Decide,
A device for evaluating a foot type, further configured to predict a center line of the talus and a center line of the first metatarsal bone, respectively, based on the centroid determined for each of the talus region and the first metatarsal region. The method of claim 18,
And an output configured to provide the predicted two pairs of feature points. A receiving unit configured to receive a medical image of the foot of the individual,
Output, and
A processor connected to communicate with the receiving unit and the output unit,
The processor, using a region prediction model configured to predict a talus region and a first metatarsal region based on the foot medical image, the talus region and the first metatarsal bone in the foot medical image Predict the area,
Determine the talus-first metatarsal angle based on the predicted center line for each of the talus region and the first metatarsal region,
Evaluate the type of the individual's foot based on the talus-first metatarsal angle,
The device for evaluating the type of foot, configured to provide the talus-first metatarsal angle through an output unit. A receiving unit configured to receive a medical image of the foot of the individual, and
A processor connected to communicate with the receiving unit,
The processor,
Using a region prediction model configured to predict a talus region and a first metatarsal region based on the foot medical image, the talus region and the first metatarsal region are predicted in the foot medical image, ,
Determine the talus-first metatarsal angle based on the predicted center line for each of the talus region and the first metatarsal region,
Configured to evaluate the type of the individual's foot based on the talus-first metatarsal angle,
The processor,
If the angle between the talus and the first metatarsal is less than -4 degrees, it is evaluated as a concave foot, or
When the talus-first metatarsal angle is greater than 4 degrees, the device for evaluating the type of foot is further configured to be evaluated as a flat foot. The method of claim 15,
The processor,
If the talus-first metatarsal angle is 4 to 15 degrees, it is evaluated as mild flatfoot, or
If the angle between the talus and the first metatarsal is 15 to 30 degrees, it is evaluated as moderate flat feet, or if the angle between the talus and the first metatarsal is more than 30 degrees, it is evaluated as severe flat feet. A device for evaluating the type of foot, further configured to.

Images