WO2019022307A1

WO2019022307A1 - Method for creating three-dimensional skeleton model of patient

Info

Publication number: WO2019022307A1
Application number: PCT/KR2017/012936
Authority: WO
Inventors: 윤기범; 이제희; 박문석
Original assignee: 서울대학교산학협력단; 서울대학교병원
Priority date: 2017-07-25
Filing date: 2017-11-15
Publication date: 2019-01-31
Also published as: KR101921988B1

Abstract

The present invention provides a method for creating a three-dimensional skeleton model of a patient from a first image and a second image of a skeleton corresponding to a specific part of the patient with reference to a statistical model, the method comprising the steps of: (a) by a computing device, calculating (1.k)-th and (2.k)-th relative position information indicating relative position information of photographing positions of the first and second images with respect to a position of the skeleton of the patient, with reference to the statistical model; and (b) by the computing device, arranging the statistical model, the first image, and the second image with reference to the (1.k)-th and (2.k)-th relative position information, and correcting a shape of the statistical model with reference to the first image and the second image, thereby generating a k-th correction model.

Description

How to create a three-dimensional skeleton model of a patient

The present invention relates to a method for generating a three-dimensional skeleton model of a patient, and more particularly, to a method for generating a three-dimensional skeleton model of a patient by referring to a statistical model generated by analyzing skeleton data corresponding to a specific region of a plurality of reference persons, A method for generating a three-dimensional skeleton model of a patient from a first image and a second image of a skeleton corresponding to a specific region (the first image and the second image are obtained at different photographing positions) ) Relative to the position of the skeleton of the patient with reference to the statistical model, the first relative position information indicating the relative position information of the photographing position of the first image with respect to the position of the skeleton of the patient, 2. The method of claim 1, wherein the computing device is configured to calculate the relative position information of the second image, The first image and the second image are arranged with reference to the 1.k relative position information and the 2.k relative position information, And a method of generating a kth correction model by correcting the shape of the statistical model.

Conventionally, CT equipment has been mainly used for three-dimensional diagnosis of the human skeleton. However, CT equipment is limited to radiation doses that are several tens of times higher than simple two-dimensional radiography, and there is a potential risk, especially since children are vulnerable to radiation. On the other hand, conventional three-dimensional imaging apparatus such as EOS can provide three-dimensional stereoscopic images by simultaneously photographing the front and side of the patient by using two imaging apparatuses arranged at 90 degrees. However, the cost of such a device is as high as 1 billion KRW, and there is a problem in that a separate space is required because two radiographic devices must be disposed. In addition, since the EOS is designed assuming that the patient is standing still, there is a limitation in that it is difficult to use patients who are unable to stand up due to the pathology.

Accordingly, it is an object of the present invention to solve all the problems described above.

Another object of the present invention is to enable generation of a three-dimensional skeleton model of a patient by using only existing radiation equipment without purchasing a new expensive equipment.

The present invention also relates to a statistical model generated by analyzing a plurality of skeleton data and a method capable of generating a three-dimensional skeleton model using two radiological images of a patient photographed at a free photographing position independent of the photographing angle For other purposes.

In order to accomplish the above object, a representative structure of the present invention is as follows.

According to one aspect of the present invention, a statistical model generated by analyzing skeleton data corresponding to a specific region of a plurality of reference persons is referred to, and a first image of a skeleton corresponding to the specific region of the patient and a second image A method for generating a three-dimensional skeleton model of a patient from an image, wherein the first image and the second image are acquired at different imaging locations, the method comprising: (a) 1.k relative positional information indicating relative positional information of the imaging position of the first image with respect to the position of the skeleton of the patient and relative positional information of the imaging position of the second image relative to the position of the skeleton of the patient Calculating second relative position information indicating the second relative position information; And (b) the computing device arranges the statistical model, the first image, and the second image with reference to the 1.k relative position information and the 2.k relative position information, and the first And generating a k th correction model by correcting the shape of the statistical model with reference to the image and the second image.

In addition, there is further provided another method, apparatus, system for implementing the invention and a computer readable recording medium for recording a computer program for executing the method.

According to the present invention, it is possible to generate a three-dimensional skeleton model of a patient by using only existing radiation equipment without purchasing a new expensive equipment.

In addition, a three-dimensional skeleton model can be generated using the statistical model generated by analyzing a plurality of skeleton data and two radiation images of a patient photographed at a free photographing position independent of the photographing angle.

FIG. 1 is an exemplary diagram for explaining a process of calculating 1.k relative position information and 2.k relative position information using a method of generating a three-dimensional skeleton model of a patient according to an embodiment of the present invention; FIG.

2 illustrates a process of calculating the (k + 1) -th relative position information and the (2k + 1) -th relative position information using a method of generating a three-dimensional skeleton model of a patient according to an embodiment of the present invention For example,

FIG. 3 is an exemplary view for explaining a method of generating a three-dimensional skeleton model of a patient according to an embodiment of the present invention;

4 is an exemplary diagram for explaining a method of correcting a shape of a statistical model with reference to a first image and a second image according to an embodiment of the present invention;

The following detailed description of the invention refers to the accompanying drawings, which illustrate, by way of illustration, specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that the various embodiments of the present invention are different, but need not be mutually exclusive. For example, certain features, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the invention in connection with an embodiment.

It is also to be understood that the position or arrangement of the individual components within each disclosed embodiment may be varied without departing from the spirit and scope of the invention. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present invention is to be limited only by the appended claims, along with the full scope of equivalents to which such claims are entitled, if properly explained. In the drawings, like reference numerals refer to the same or similar functions throughout the several views.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings, so that those skilled in the art can easily carry out the present invention.

1 is an exemplary view for explaining a process of calculating 1.k relative position information and 2.k relative position information using a method of generating a three-dimensional skeleton model of a patient according to an embodiment of the present invention; Respectively. Here, FIG. 1A is a diagram illustrating modeling of an embodiment of the present invention in a three-dimensional space, and FIG. 1B is a simplified diagram for explaining an embodiment of the present invention.

More specifically, the present invention relates to a method and apparatus for analyzing a skeleton from a first image and a second image of a skeleton corresponding to a specific region of a patient, with reference to a statistical model generated by analyzing skeleton data corresponding to a specific region of a plurality of reference persons, Dimensional skeleton model of the three-dimensional skeleton model.

Here, the statistical model is generated with reference to each average value calculated for each variable with respect to a plurality of skeleton data, in a state where each skeleton data corresponding to a specific region of a plurality of reference persons includes at least one variable .

By way of example, the length of the skeleton corresponding to the specific region, the diameter of the cross section, the shape and position of the one end, the angle of the bend, and the like may be variables. The variables constituting the plurality of skeleton data are analyzed, A three-dimensional statistical model can be constructed by calculating an average value for each of the variables.

In addition, by constructing various statistical model databases such as a statistical model for a plurality of reference subjects, a statistical model for a reference age of a specific range of reference persons, and a statistical model for a specific gender or a specific range key, When creating the 3D skeleton model, the statistical model of the specific group to which the patient belongs can be referred to.

Next, the first image and the second image can be obtained at different photographing positions, and can be photographed at a free photographing position regardless of the photographing angle. For example, the first image may be an image taken when the patient is lying down properly, and the second image may be an image taken when the patient lies sideways.

Further, the first image and the second image can be obtained by extracting the skeleton of the patient from the photographed radiographic image, respectively, or by radiographing the radiographic image with a predetermined camera. Illustratively, a first display and a second image may be obtained by photographing a predetermined display screen displaying a radiological image using a user terminal including a camera such as a tablet.

Meanwhile, a computing device (e.g., a control computer, a control server, a control terminal, etc.) for performing a method of generating a three-dimensional skeleton model of a patient according to an embodiment of the present invention, Program) may include first 1.k relative position information indicating the relative positional information of the imaging position of the first image with respect to the position of the skeleton of the patient with reference to the statistical model, 2.k relative position information indicating the relative position information of the photographing position of the two images can be calculated.

Illustratively, referring to FIGS. 1A and 1B, when the image on the left side of the statistical model is referred to as the first image and the image on the right side of the statistical model is referred to as the second image, the computing device according to the present invention calculates Thus, the relative position information of each of the first image and the second image can be calculated.

That is, when the shape of at least a part of the skeleton of the patient shown in each of the first image and the second image is found to coincide with at least a part of the shape of the statistical model within a predetermined range from a point at which the statistical model can be observed, , It is possible to calculate the first and the second relative position information and the 2.k relative position information by predicting the angle and the distance of each of the first image and the second image with respect to the position of the skeleton of the patient.

As a concrete example, when a patient's skeleton shown in the first image is projected on a statistical model, a viewpoint at which the shape of the patient skeleton and the shape of the statistical model satisfy a predetermined condition (for example, 90% match) (? ₀ ?) Between the imaginary straight line passing through the statistical model and the normal vector of the plane of the first image and the relative distance data of the first image with respect to the statistical model The coordinate data of each point constituting the skeleton), and the like. By performing the same process for the second image, it is possible to calculate the angle data (? ₀ ?) And the second image It is possible to calculate the 2.k relative position information including the relative distance data and the like.

Further, in calculating the 1.k relative position information and the 2.k relative position information, the first image of the position of the skeleton of the patient by using the PnP (Perspective-n-Point) algorithm with reference to the statistical model And the angle and distance of each of the second images, thereby calculating the first 1.k relative position information and the 2. k relative position information, but not limited thereto.

Next, with reference to FIG. 4, a process of generating a k-th correction model using a method of generating a three-dimensional skeleton model of a patient according to an embodiment of the present invention will be described.

Specifically, the computing device arranges the statistical model, the first image and the second image with reference to the 1.k relative position information and the 2.k relative position information, and with reference to the first image and the second image The kth correction model can be generated by correcting the shape of the statistical model.

That is, the statistical model is arranged at the position of the patient skeleton when the first image and the second image are photographed according to the 1.k relative position information and the 2.k relative position information, and based on this, With the image placed, the shape of the statistical model can be corrected with reference to the shape of the patient skeleton shown in the first image and the second image.

Illustratively, assuming that the image shown on the left side of FIG. 4 is either the first image or the second image, the outline of the statistical model projected on the first image and the second image is the first image and the second image The shape of the model can be optimized by modifying the 3D mesh of the statistical model to match the outline of the patient skeleton shown in Fig.

Here, in correcting the shape of the statistical model, least squares methods may be applied with reference to the first image and the second image. For reference, in the error function of Fig. 4, c is the current shape parameter, b is the vector between the two contours (red arrow), and Q is the deformable direction of the projected contour, thereby optimizing c.

Next, with reference to FIG. 2, a method of generating a three-dimensional skeleton model of a patient according to an embodiment of the present invention is used to generate first 1.k + 1 relative position information and 2.k + 1 relative position information And a k + 1 correction model is generated.

After computing the kth correction model by correcting the shape of the statistical model, the computing device calculates (i) the relative position information of the photographing position of the first image with respect to the position of the skeleton of the patient with reference to the kth correction model (K + 1) -th relative position information indicating relative position information of the first imaging position of the second image with respect to the position of the skeleton of the patient, and (ii) The first image and the second image are arranged with reference to the 1.k + 1 relative position information and the 2.k + 1 relative position information, and the kth correction is performed with reference to the first image and the second image, The process of generating the k + 1 correction model by correcting the shape of the model can be performed.

That is, a kth correction model is generated by correcting the shape of the statistical model with reference to the first image and the second image so that the shape of the model is closer to the shape of the patient skeleton, K + 1 correction model by correcting the shape with reference to the first image and the second image.

Specifically, by calculating the time when the shape of the patient skeleton and the shape of the k-th correction model satisfy a predetermined condition (for example, 90% match) when the skeleton of the patient shown in the first image is projected on the k-th correction model , Angle data (? ₁ ?) Formed by a virtual straight line passing through the kth correction model and the normal vector of the plane of the first image, and relative distance data of the first image with respect to the kth correction model (e.g., the 1.k + 1 will be able to calculate the relative position information, by performing the same process, also in the second image data, the angle (β ₁ ˚), or the like coordinate data) of the respective points constituting the skeleton of the patient on And relative distance data of the second image, and the like.

On the other hand, the computing device compares the kth correction model or the (k + 1) th correction model with the first image and the second image to determine a predetermined condition (for example, the contour of the patient skeleton appearing in the first image and the second image, (K + 1) -th correction model or the (k + 1) -th correction model is determined as the three-dimensional skeleton model of the patient. If the model does not satisfy the condition, (ii) repeating the process.

That is, according to an embodiment of the present invention, as shown in FIG. 3, a first image (first image), a second image (second image), and a statistical model (third image) , Thereby generating a three-dimensional model (fourth figure) of the patient's skeleton by repeating (i) the process and (ii) process.

Through such a process, it is possible to create a three-dimensional skeleton model of a patient by using only the existing radiation equipment without purchasing expensive equipment. According to the present invention, a freely taken image is used without limiting the position or posture of the patient, and even if the exact photographing position of the photographed image is unknown, the photographing position of the image can be calculated using the statistical model, The skeleton model of the patient can be generated by correcting the statistical model with reference to the image.

On the other hand, the first image and the second image each include the contour information and the center line information of the patient skeleton, and the computing device compares the pixel data information, the contour information, and the center line information of each of the first image and the second image, The k-th correction model can be generated by correcting the shape of the model.

In the determination of the outline information, in a state where an original image (original image corresponding to the first image and the second image) photographed so that a specific region of the patient appears on at least a part thereof is displayed on a predetermined display, At least a part of the skeleton corresponding to a specific part of the original image is selected as the first selection area and at least a part of the area around the skeleton is selected as the second selection area, 2 outline information indicating the boundary of the skeleton corresponding to the specific region can be determined with reference to the selection region.

Illustratively, when an original image is displayed on a user terminal such as a tablet and the user touches at least a part of a region corresponding to a particular skeleton to which a three-dimensional skeleton model is to be generated, When the user touches at least some areas around a specific skeleton, the corresponding part is displayed as blue (second selection area). When the selection is completed, the red area and the blue area, which are partially displayed, A process of dividing and displaying the boundaries of specific skeletons may be performed.

As described above, since a method for generating a three-dimensional skeleton model of a patient according to the present invention can be applied to a mobile terminal, there is no need to provide a separate equipment and space, It can be easily used in small hospitals.

The embodiments of the present invention described above can be implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program commands, data files, data structures, and the like, alone or in combination. The program instructions recorded on the computer-readable recording medium may be those specially designed and constructed for the present invention or may be those known and used by those skilled in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules for performing the processing according to the present invention, and vice versa.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, Those skilled in the art will appreciate that various modifications, additions and substitutions are possible, without departing from the scope and spirit of the invention as disclosed in the accompanying claims.

Therefore, the spirit of the present invention should not be construed as being limited to the above-described embodiments, and all of the equivalents or equivalents of the claims, as well as the following claims, I will say.

Claims

A first image and a second image of a skeleton corresponding to the specific region of the patient, with reference to a statistical model generated by analyzing skeleton data corresponding to a specific region of a plurality of reference subjects, The second image being obtained at different imaging locations, the method comprising:

(a) computing means for calculating a relative positional information of the first image relative to the position of the skeleton of the patient with reference to the statistical model, Calculating second relative positional information indicating relative positional information of a photographing position of the second image with respect to the position; And

(b) the computing device arranges the statistical model, the first image and the second image with reference to the 1.k relative position information and the 2.k relative position information, and the first image And generating a k-th correction model by correcting the shape of the statistical model with reference to the second image;

&Lt; / RTI >
The method according to claim 1,

After the step (b), the computing device includes: (i) a first positional information storage unit for storing relative position information of the first image with respect to the position of the skeleton of the patient with reference to the kth correction model; (k + 1) -th relative position information and relative position information of a photographing position of the second image with respect to the position of the skeleton of the patient; and (ii) The first image and the second image are arranged with reference to the 1. k + 1 relative position information and the 2. k + 1 relative position information, and the first image and the second image, And a process of generating a (k + 1) -th correction model by correcting the shape of the k-th correction model with reference to the k-th correction model.
3. The method of claim 2,

The computing device compares the kth correction model or the k + 1 correction model with the first image and the second image and if the predetermined condition is satisfied, the kth correction model or the k + Determining a model as a skeleton model of the patient,

Wherein the step (i) and the step (ii) are repeatedly performed until the predetermined condition is satisfied.
The method according to claim 1,

Wherein each of the first image and the second image is obtained by extracting the skeleton of the patient from the radiographic image taken or by photographing the radiological image with a predetermined camera.
The method according to claim 1,

Wherein the first image and the second image each include contour information of the skeleton of the patient, centerline information,

The computing device generates the kth correction model by correcting the shape of the statistical model with reference to the pixel data information, the contour information, and the center line information of each of the first image and the second image, How to.
6. The method of claim 5,

In determining the outline information,

Wherein the original image photographed so that the specific region of the patient appears in at least a part thereof, the original image representing an original corresponding to the first image and the second image, When at least a partial region of the skeleton corresponding to the specific region on the original image is selected as a first selection region and at least a region around the skeleton is selected as a second selection region, And the contour information indicating the boundary of the skeleton corresponding to the specific region is determined with reference to the contour information.
The method according to claim 1,

In a state where each of the skeleton data corresponding to the specific region of the plurality of reference persons includes at least one variable,

Wherein the statistical model is generated with reference to each average value calculated for each of the plurality of the skeleton data.
The method according to claim 1,

Wherein the computing device calculates at least a part of the shape of the statistical model from a point at which the shape of at least a part of the skeleton of the patient exhibited in the first image and the second image can observe the statistical model, And estimating an angle and a distance of each of the first image and the second image with respect to the position of the skeleton of the patient with reference to a point of time when the second relative position information and the second . k. < / RTI >
The method according to claim 1,

The computing device estimates the angle and the distance of each of the first image and the second image with respect to the position of the skeleton of the patient through a Perspective-n-Point (PnP) algorithm with reference to the statistical model, The first relative position information and the second relative position information.
The method according to claim 1,

Wherein the computing device corrects the shape of the statistical model by applying least squares methods with reference to the first image and the second image.