KR102271614B1 - 3D anatomical landmark detection method and apparatus - Google Patents

Abstract

The present invention relates to a method and apparatus for detecting a three-dimensional reference point, the method according to the present invention comprises the steps of performing volume rendering on three-dimensional image data to generate a two-dimensional projection image, and using a learned neural network model, two-dimensional projection Detecting a feature point corresponding to a reference point from an image, extracting voxel values of voxels having the same two-dimensional coordinate value as the feature point from 3D image data, extracting with respect to an axial direction passing through the feature point and perpendicular to the two-dimensional projection image obtaining gradient values of the obtained voxel values, and detecting, as a position of a reference point, a position on an axis at which the voxel values and the gradient values satisfy a predetermined condition.

Description

3D anatomical landmark detection method and apparatus

The present invention relates to a method and apparatus for detecting a 3D anatomical reference point, and more particularly, to a method and apparatus capable of overcoming the complexity of processing 3D image data and accurately detecting a 3D anatomical reference point.

Since X-ray image-based cephalometry was introduced by Broadbent, it has been widely used for orthodontic diagnosis and treatment planning, and has now become an essential element in dental treatment. In the craniomaxillofacial structure analysis, head measurement analysis using an anterior-posterior relationship was mainly conducted, and head measurement analysis using two-dimensional frontal and lateral X-ray images was mainly performed. Head measurement analysis using X-ray images requires a process of finding anatomical landmarks. Finding a reference point requires a highly skilled specialist, but most of it is a time-intensive and repetitive task. To overcome this, an image processing-based automation or semi-automation algorithm has been proposed.

Recently, it is rapidly changing to head measurement analysis using 3D computed tomography (CT). The three-dimensional volume image through computed tomography can observe the interior of complex anatomical structures in a desired direction. In addition, important parts can be observed in detail without distortion, magnification and overlap of the image. In addition, it is being used as a very useful diagnostic data as it is possible to distinguish between tissues with a small difference in physical density through high contrast spatial resolution. Like the 2D cephalometric analysis, it is essential to find a landmark in the 3D cephalometric analysis.

However, unlike the 2D cephalometric analysis, since the geometrical complexity of the reference point in the 3D space and the number of points to be handled increases, the complexity of the operation increases.

Recent deep-learning-based image processing algorithms can be viewed as creating nonlinear filters that can extract or classify meaningful information from images through data-based learning. Because it creates a suitable nonlinear filter through learning according to the characteristics of the data, it shows higher performance than the previous model-based image processing algorithm, and the field is expanding to the medical imaging field.

However, deep learning-based models that process 3D data have difficulties in increasing parameters, computing resources, and computational complexity. To overcome this limitation, data resampling and a method of localizing the head measurement points using a three-dimensional convolutional neural network (CNN) model have been proposed, but the error of localization in three dimensions is not suitable for clinical use. shows non-performing performance.

US Patent No. 8,363,918 (Registration Date: January 29, 2013)

Accordingly, the technical problem to be solved by the present invention is to provide a method and an apparatus capable of detecting a 3D reference point from 3D image data.

The three-dimensional reference point detection method according to the present invention for solving the above technical problem includes the steps of performing volume rendering on three-dimensional image data to generate a two-dimensional projection image, using a learned neural network model, detecting a feature point corresponding to a reference point in the two-dimensional projection image; extracting voxel values of voxels having the same two-dimensional coordinate value as the feature point from the three-dimensional image data; obtaining gradient values of the extracted voxel values in an axial direction perpendicular to the two-dimensional projection image, and detecting a position on the axis at which the voxel values and the gradient values satisfy a predetermined condition as the position of the reference point including the steps of

The 2D projection image may be generated by performing volume rendering on the 3D image data through a transfer function for visualizing the bone.

The generating of the 2D projection image may include a preprocessing step of designating voxel values of voxels located in a direction opposite to the ROI with respect to a reference plane in the 3D image data as transparent values, and the preprocessed 3D image data. The method may include generating the 2D projected image by performing volume rendering on the image data.

The predetermined condition may be differently determined according to characteristics of the reference point.

The predetermined condition is that when the reference point is outside the bone, the voxel value of the reference point estimated position is within a HU (Hounsfield Unit) value range corresponding to the bone, and the gradient value of the reference point estimated location and the reference point on the axis The sum of the gradient values of the positions before or after the estimated position may be set to be equal to or greater than a predetermined reference.

The neural network model may be trained to detect feature points through reinforcement learning using learning data formed of a two-dimensional projection image obtained by performing ray tracing volume rendering on three-dimensional image data.

The computer-readable recording medium according to the present invention for solving the above technical problem is, by using a set of instructions for generating a two-dimensional projection image by performing volume rendering on three-dimensional image data, and a learned neural network model, the two-dimensional A command set for detecting a feature point corresponding to a reference point in a projection image, a command set for extracting voxel values of voxels having the same two-dimensional coordinate value as the feature point from the 3D image data, to the two-dimensional projection image after passing the feature point A set of instructions for obtaining gradient values of the extracted voxel values with respect to a vertical axis direction, and a set of instructions for detecting a position on the axis at which the voxel values and the gradient values satisfy a predetermined condition as the position of the reference point include

A three-dimensional reference point detection apparatus according to the present invention for solving the above technical problem is a two-dimensional projection image generator for generating a two-dimensional projection image by performing volume rendering on three-dimensional image data, using a learned neural network model , a feature point detector for detecting a feature point corresponding to a reference point in the two-dimensional projection image, a voxel value extractor for extracting voxel values of voxels having the same two-dimensional coordinate value as the feature point from the 3D image data, passing the feature point a gradient value calculator for obtaining gradient values of the extracted voxel values in an axial direction perpendicular to the two-dimensional projection image, and a position on the axis at which the voxel values and the gradient values satisfy a predetermined condition as the reference point It includes a reference point detection unit for detecting the position of.

According to the present invention, there is an advantage in that the 3D reference point can be accurately detected while overcoming the complexity of 3D image data processing.

1 is a diagram showing the configuration of a three-dimensional reference point detection system according to an embodiment of the present invention.
2 illustrates a two-dimensional projection image generated through volume rendering.
3 illustrates a two-dimensional projection image in a view direction when the head of the object is viewed from above.
4 illustrates a 2D projection image generated through volume rendering of preprocessed 3D image data.
5 shows a two-dimensional projection image in which a feature point corresponding to a Nasion point according to the present invention is detected.
FIG. 6 is a schematic diagram for explaining extraction of voxel values corresponding to the Nasion feature points detected in FIG. 5 .
FIG. 7 is a voxel value profile graph of a line buffer in which the voxel values extracted in FIG. 6 are stored.
8 is a view showing both the voxel value profile data and the gradient value profile data obtained according to the present invention.
9 is a flowchart illustrating a method for detecting a three-dimensional reference point according to the present invention.

Then, with reference to the accompanying drawings, embodiments of the present invention will be described in detail so that those of ordinary skill in the art can easily implement them.

1 is a diagram showing the configuration of a three-dimensional reference point detection system according to an embodiment of the present invention.

Referring to FIG. 1 , the 3D reference point detection system according to the present invention may include an image acquisition device 200 , an image storage device 300 , and a 3D reference point detection device 100 .

The image acquisition device 200 is a device for acquiring three-dimensional (3D) image data of an object. Here, the 3D image data is data obtained in a type such as Computed Tomography (CT), Cone Beam Computed Tomography (CBCT), Magnetic Resonance Imaging (MRI), Positron Emmission Tomography (PET), or Single-Photon Emmission Computed Tomography (SPECT). can be The object may be a body part such as a skull.

The image storage device 300 may store 3D image data acquired by the image acquisition device 200 or provided from an external medical image information provider including an external medical institution or research institution.

The 3D reference point detection apparatus 100 may detect a 3D landmark point from 3D image data provided from the image acquisition device 200 or the image storage device 300 .

To this end, the 3D reference point detection apparatus 100 includes a data receiving unit 110 , a 2D projected image generation unit 120 , a machine learning unit 130 , a feature point detection unit 140 , a voxel value extraction unit 150 , and a gradient value. It may include a calculation unit 160 , a reference point detection unit 170 , a storage unit 180 , a display unit 185 , and a control unit 190 .

According to an embodiment, the machine learning unit 130 may not be included in the 3D reference point detection apparatus 100 but may be constructed as a separate device. The three-dimensional reference point detection apparatus 100 may receive and use a feature point detection algorithm learned from a separate device.

The data receiver 110 may receive 3D image data from the image acquisition device 200 or the image storage device 300 .

The 2D projection image generator 120 may perform volume rendering on 3D image data to generate a 2D projection image.

Volume rendering is a method of displaying 3D voxel data by projecting it into a 2D projection image. For example, the ray tracing volume rendering method considers a virtual ray in the projection direction and accumulates values along the ray in consideration of color and transparency according to preset voxel values, so that 3D data can be visualized on a 2D screen. .

Here, the voxel value may be a HU value in the case of CT. Of course, in the case of 3D image data having a modality other than CT, a value indicating signal strength according to the modality may be used as a voxel value.

When volume rendering is performed by applying a transfer function for visualizing bones to 3D image data, a 2D projection image as illustrated in FIG. 2 may be obtained.

2 illustrates a two-dimensional projection image generated through volume rendering.

In an embodiment according to the present invention, the two-dimensional projection image is a view from the front (Anterior View), a view from the back (Posterior View), a view from above (Superior View), a view from below (Inferior View), a view from the left (Left View) and a view viewed from the right (Right View) may be used. Of course, when a reference point of a body part other than the head is to be detected, it may be a two-dimensional projection image of the corresponding body part. Also, the view direction can be applied in various ways.

Meanwhile, according to an embodiment, the 2D projection image generator 120 may perform a predetermined pre-processing on the 3D image data. In addition, the 2D projection image generator 120 may perform volume rendering on the preprocessed 3D image data to generate a desired 2D projection image. For example, the 2D projection image generator 120 may designate a voxel value located in a direction opposite to the body region of interest based on a constant reference plane in the 3D image data as a value to be transparently processed. Thereafter, the 2D projection image generator 120 transparently processes the non-interested region and then applies a transfer function for visualizing the ROI to 3D image data to perform volume rendering, thereby performing 2D projection on a desired body part. You can create an image.

Here, the reference plane may be determined as a sagittal plane, a coronal plane, a horizontal plane, or a midsagittal plane. Of course, other reference planes may be applied.

The machine learning unit 130 may train a neural network model that detects two-dimensional feature points in a two-dimensional projected image through supervised learning or reinforcement learning. Here, the two-dimensional feature point is a point corresponding to the three-dimensional reference point in the two-dimensional projection image.

Hereinafter, for convenience of description, a two-dimensional feature point is simply referred to as a feature point, and a three-dimensional reference point is simply referred to as a reference point.

The machine learning unit 130 may train a neural network model by using a plurality of two-dimensional projection images labeled with feature points by a specialist, a clinician, or the like as training data. Here, the neural network model may be in the form of a machine learning algorithm such as a convolutional neural network (CNN). Convolutional neural networks can be implemented with deep CNN models such as AlexNet, VGGNet, GoogLeNet, ResNet, etc.

A method in which the neural network model detects two-dimensional feature points in a two-dimensional projection image through reinforcement learning in the machine learning unit 130 will be described in more detail.

Reinforcement learning is a technology typically applied to Google's AlphaGo. In order to receive the best reward in a specific state, there is a lot of information about what policy to take and how to proceed to the next state. It refers to a method of learning so that the optimal behavior can be selected through learning.

Specifically, a computer program targeted for reinforcement learning is also called an agent. The agent establishes a policy that expresses the action to be taken by the agent in a given state. The goal of reinforcement learning is to train agents to formulate policies that will receive maximum rewards.

In this embodiment, a value obtained by subtracting the distance between the currently estimated point (P _i) and the characteristic point (P _t) in the distance between the previous estimation point (P _i-1) and the characteristic point (P _t) as illustrated in equation [1 An example of reinforcement learning using as a reward is shown. That is, reinforcement learning can be performed in such a way that the reward increases as the distance between the estimation point and the feature point decreases.

[Equation 1]

R = D(P _i-1 , P _t ) - D(P _i , P _t )

where R is the reward, D(P _i-1 , P _t ) is the distance between the previous estimation point (P _i-1 ) and the feature point (P _t ), and D(P _i , P _t ) is the current estimation point (P _{i )} ) and the feature point (P _t ).

Of course, the machine learning unit 130 may train the neural network model to detect the feature points in the two-dimensional projected image by other learning methods other than those exemplified here.

Each neural network model may be trained for each view direction from which the machine learning unit 130 can extract feature points well.

3 illustrates a two-dimensional projection image in the direction of the patient's head viewed from above.

As illustrated in FIG. 3 , the Bregma point is most clearly seen in the direction of the view when the head is viewed from above (Superior View). Therefore, if the reference point is a bregma point, the neural network model can be trained by composing the training data in the two-dimensional projection image labeled with the feature point in Superior View.

On the other hand, in the case of the Nasion point, the patient's head can be seen well in the direction of the Anterior View. Therefore, when the reference point is the Nasion point, the neural network model can be trained by composing the training data in the two-dimensional projection image labeled with the feature point in the Anterior View.

As such, a reference point located on the outside of the head, such as a bregma point or a Nasion point, can be learned using a two-dimensional projection image in the view direction when the head is viewed from the outside.

On the other hand, in the case of the mandibular foramen point (point F) located on the inside of the jawbone, the neural network model is learned using the 2D projection image obtained by performing volume rendering after preprocessing the 3D image data first. can do.

4 illustrates a 2D projection image generated through volume rendering of preprocessed 3D image data.

FIG. 4(a) shows the right mandibular foramen point (right F; after performing pre-processing of designating the voxel value of the left part of the head as '-1024' corresponding to air in the 3D image data to see the RF) , shows a two-dimensional projection image obtained by performing volume rendering through a transfer function corresponding to a bone.

Fig. 4(b) shows the bone after pre-processing in which the voxel value of the left part of the head is designated as '-1024' corresponding to air in the 3D image data to view the left mandibular foramen point (left F; LF). It shows a two-dimensional projection image obtained by performing volume rendering through a corresponding transfer function.

As described above, it is possible to learn by using a two-dimensional projection image generated in a view direction that can be easily confirmed according to the position of the detection target reference point. If necessary, it is also possible to learn using a two-dimensional projection image generated after transparent processing of the region opposite to the region of interest with respect to the reference plane. Of course, in addition to the method described here, it is possible to learn to acquire various feature points from a two-dimensional projection image obtained from three-dimensional image data by various methods.

Referring back to FIG. 1 , the feature point detector 140 may detect a feature point corresponding to a reference point in the two-dimensional projection image using the learned neural network model.

5 shows a two-dimensional projection image in which a feature point corresponding to a Nasion point according to the present invention is detected.

As illustrated in FIG. 5 , when a feature point corresponding to the Nasion point is detected in the 2D projection image in the Anterior View direction (ie, the vertical axis direction of the 2D projection image), the 2D coordinate value of the Nasion point (X _Na , Z _Na ) can be obtained However, the depth information of the Nasion point, that is, the position on the Y-axis, cannot be obtained yet.

FIG. 6 is a schematic diagram for explaining extraction of voxel values corresponding to the Nasion feature points detected in FIG. 5 .

Referring to FIG. 6 , voxels _{passing through the Nasion feature point (X Na} , Z _Na ) and positioned on the axis Y perpendicular to the 2D projection image of FIG. 5 may be extracted. The voxels positioned on the axis Y have the same two-dimensional coordinate values (X _Na , Z _Na ).

The voxel value extractor 150 may extract voxel values of voxels having the same two-dimensional coordinate value as the feature point detected by the feature point detector 140 from the 3D image data. For example, the voxel value extractor 150 may extract voxel values of voxels located on the axis Y from the 3D image data stored in the storage unit 180 and temporarily store them in a voxel line buffer. In the voxel line buffer, voxel values are arranged at regular intervals in the Y-axis direction.

The reference points that are mainly found in cephalometric analysis, such as Nasion points, are mainly points outside the bone. Therefore, it is enough to find out the first position corresponding to the bone on the other axis (eg, the Y axis in the above embodiment). Of course, if the reference point is located inside the bone, the last position corresponding to the bone may be found. Specific conditions for detecting the position of the reference point on the other axis may vary according to the characteristics of the reference point.

FIG. 7 is a voxel value profile graph of a line buffer in which the voxel values extracted in FIG. 6 are stored.

In FIG. 7 , the vertical axis indicates the HU value, and the horizontal axis indicates the position of the voxel on the axis (Y).

The voxel value profile of the line buffer illustrated in FIG. 7 shows that, while moving from left to right, it meets the outside of the bone at approximately the 80th position, and the HU value sharply increases and the inside of the bone appears again. Since the current reference points are mainly points outside the bone, the first position corresponding to the bone in the voxel value profile of the other axis (e.g., the Y axis) rather than the two axes (X axis, Z axis) in which the two-dimensional coordinate value is estimated. to find out However, since characteristics of each image data are different, it may be difficult to simply estimate a position corresponding to a bone only with the voxel value profile illustrated in FIG. 7 .

At the interface between bone and skin tissue, skin and air, etc., the amount of change in the voxel value increases. Therefore, in order to increase accuracy, a gradient profile composed of a gradient value obtained by taking a gradient to a voxel value stored in the line buffer may be considered together.

Voxel values of voxels spaced apart from each other on a specific axis are stored in the line buffer. Here, the gradient operation may be to obtain a difference between the voxel values of the current position and the next position on the line buffer.

The gradient value calculator 160 may calculate gradient values corresponding to changes in voxel values in the axial direction (the Y direction in FIG. 6 ) perpendicular to the 2D projection image in which the feature point is detected. The gradient values calculated by the gradient value calculator 160 may be temporarily stored in the gradient line buffer.

The reference point detector 170 may detect the reference point using the voxel value profile and the gradient profile. The reference point detector 170 may detect the position of the reference point on the other axis that satisfies a predetermined condition by using the voxel value profile and the gradient value profile.

8 is a view showing both the voxel value profile data and the gradient value profile data obtained according to the present invention.

Referring to FIG. 8 , a graph Linebuf L represents a voxel value profile extracted from each position of the Y-axis in FIG. 6 . And the graph (Gradient) (G) shows the gradient value profile calculated at each position of the Y-axis in FIG. 6 .

Looking at the section A corresponding to the skin tissue, the voxel value (HU value) rapidly increases from the approximately 60th position on the Y-axis, but the HU value is less than '0'.

Meanwhile, looking at the section (B) corresponding to the bone, the HU value rapidly increases from the 75th position on the Y-axis, and the HU value is greater than '0'.

Therefore, as a condition for detecting the position of the reference point outside the bone, the HU value is greater than '0', and the change amount increases as the HU value increases rapidly in the outer part of the bone. can be set to

In addition, in order to prevent erroneous detection due to noise or object characteristics, the gradient value condition may be set to a case in which the sum of the gradient values of the current position and the neighboring position is equal to or greater than a predetermined criterion. For example, the sum of the gradient value of the N-th position of the Y-axis and the gradient value of the N+1-th position (or the N-1th position) may be set as the first position on the Y-axis equal to or greater than a predetermined reference.

8 illustrates that the 77th Y-axis position is detected as the Y-axis position of the reference point under the condition that the HU value is positive and the sum of the gradient values of the current position and the next position is '300' or more.

Again, it should be understood that the conditions for the voxel value and the gradient value for detecting the position of the reference point on the other axis may be set differently depending on the characteristics of the reference point.

The storage unit 180 may store various data and programs related to the operation of the 3D reference point detection apparatus 100 . The storage unit 180 includes a voxel line buffer storing voxel values of voxels located in the other axis as described above, and a gradient line buffer in which gradient values obtained by taking gradients from the voxel values stored in the voxel line buffer in the other axis direction are temporarily stored. and the like.

The storage unit 180 includes magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magneto-optical disks such as floppy disks. It may be implemented as a memory device such as magneto-optical media, ROM, RAM, flash memory, and the like.

The display unit 185 may display various types of information related to the operation of the 3D reference point detection apparatus 100 on the screen. The display unit 185 may be implemented as a display device such as an LCD display device, an LED display device, or the like.

The controller 190 may control the overall operation of the 3D reference point detection apparatus 100 .

9 is a flowchart illustrating a method for detecting a three-dimensional reference point according to the present invention.

Referring to FIG. 9 , first, the 2D projection image generator 120 may perform a predetermined pre-processing on 3D image data obtained by photographing a body part such as a patient's head ( S900 ). Step S900 may be omitted depending on the embodiment. In step S900, the 2D projection image generator 120 may perform preprocessing of designating voxel values of voxels located in the opposite direction of the ROI with respect to a constant reference plane in the 3D image data as values to be transparently processed. have.

The 2D projection image generator 120 may perform volume rendering on 3D image data to generate a 2D projection image (S910). In operation S910 , the 2D projection image generator 120 may generate a desired 2D projected image by performing volume rendering by applying a transfer function for visualizing an ROI to 3D image data. For example, if volume rendering is performed by applying a transfer function for visualizing bones to 3D image data, a 2D projection image as illustrated in FIG. 2 may be obtained.

Next, the feature point detector 140 may detect a feature point corresponding to the reference point in the two-dimensional projection image using the pre-trained neural network model ( S920 ). In step S920, a two-dimensional coordinate value may be obtained from among the three-dimensional coordinate values of the reference point. When the two-dimensional projection image is in the XZ plane, the X coordinate value and the Z coordinate value of the reference point can be obtained.

Thereafter, the voxel value extractor 150 may extract voxel values of voxels having the same two-dimensional coordinate value as the feature point detected by the feature point detector 140 from the 3D image data (S930).

Next, the gradient value calculator 160 calculates gradient values corresponding to the amount of change in voxel values in the axial direction perpendicular to the two-dimensional projection image in which the feature point is detected (the Y-axis direction when the two-dimensional projection image is in the XZ plane). It can be calculated (S940).

Finally, the reference point detector 170 determines the position of the reference point on the remaining axes (eg, the Y axis when the X and Z coordinate values of the reference point are determined) based on the voxel values obtained in step S930 and the gradient values obtained in step S940 . can be detected (S950). In operation S950 , the position of the reference point may be determined as a position on the Y axis that satisfies a predetermined condition according to the characteristics of the reference point.

The embodiments described above may be implemented by a hardware component, a software component, and/or a combination of a hardware component and a software component. For example, the apparatus, methods and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate (FPGA) array), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For convenience of understanding, although one processing device is sometimes described as being used, one of ordinary skill in the art will recognize that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that can include For example, the processing device may include a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

The software may comprise a computer program, code, instructions, or a combination of one or more thereof, which configures a processing device to operate as desired or is independently or collectively processed You can command the device. The software and/or data may be any kind of machine, component, physical device, virtual equipment, computer storage medium or device, to be interpreted by or to provide instructions or data to the processing device. may be permanently or temporarily embody in The software may be distributed over networked computer systems, and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and carry out program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with reference to the limited drawings, those skilled in the art may apply various technical modifications and variations based on the above. For example, the described techniques are performed in a different order than the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result.

100: 3D reference point detection device
110: data receiving unit 120: two-dimensional projection image generating unit
130: machine learning unit 140: feature point detection unit
150: voxel value extraction unit 160: gradient value calculation unit
170: reference point detection unit 180: storage unit
185: display unit 190: control unit
200: image acquisition device
300: video storage device

Claims (27)

generating a two-dimensional projection image by performing volume rendering on the three-dimensional image data in the two-dimensional projection image generator;
detecting a feature point corresponding to a reference point in the two-dimensional projection image using the neural network model learned by the feature point detector;
extracting, by a voxel value extracting unit, voxel values of voxels having the same two-dimensional coordinate value as the feature point from the 3D image data;
obtaining gradient values of the extracted voxel values with respect to an axial direction passing through the feature point and perpendicular to the two-dimensional projection image in a gradient value calculator; and
A three-dimensional reference point detection method in which a reference point detection unit detects, as a location of the reference point, a position on the axis where the voxel values and the gradient values satisfy a predetermined condition. In claim 1,
A 3D reference point detection method for generating the 2D projection image by performing volume rendering on the 3D image data through a transfer function for visualizing the bone in the 2D projection image generator. In claim 2,
The step of generating the two-dimensional projection image,
a pre-processing step of designating voxel values of voxels located in a direction opposite to the region of interest with respect to the reference plane in the 3D image data as values to be transparent;
generating the 2D projected image by performing volume rendering on the preprocessed 3D image data
A three-dimensional reference point detection method comprising a. In claim 3,
The reference plane is a three-dimensional reference point detection method determined by a sagittal plane, a coronal plane, a horizontal plane, or a midsagittal plane. In claim 3,
The 3D image data is
A three-dimensional reference point detection method that is any one of Computed Tomography (CT), Cone Beam Computed Tomography (CBCT), Magnetic Resonance Imaging (MRI), Positron Emmission Tomography (PET), and Single-Photon Emmission Computed Tomography (SPECT). In claim 3,
The predetermined condition is
A three-dimensional reference point detection method that is differently determined according to the characteristics of the reference point. In claim 3,
The neural network model is
A three-dimensional reference point detection method learned to detect feature points through reinforcement learning using learning data consisting of a two-dimensional projection image obtained by performing ray tracing volume rendering on three-dimensional image data. In claim 3,
The learned neural network model is,
A three-dimensional reference point detection method comprising a plurality of neural network models trained to detect feature points for each view direction in which a two-dimensional projection image is generated. In claim 3,
The neural network model is
3D reference point detection in reinforcement learning using the value obtained by subtracting the distance between the _{current estimation point (P i} ) and the feature _{point (P t} ) from the distance between the previous estimation point (P _i-1 ) and the feature _{point (P t ) as compensation} Way. In claim 3,
The 3D image data is
A three-dimensional reference point detection method obtained by photographing a patient's skull. 11. In claim 10,
The two-dimensional projection image is,
The patient's head viewed from the front (Anterior View), the back view (Posterior View), the top view (Superior View), the bottom view (Inferior View), the left view (Left View) and the right view ( Right View) is one of the three-dimensional reference point detection method. In claim 3,
The voxel value is a three-dimensional reference point detection method that is a HU (Hounsfield Unit) value. In claim 12,
The predetermined condition is
When the reference point is outside the bone,
The voxel value of the estimated position of the reference point is within the HU value range corresponding to the bone,
A three-dimensional reference point detection method, wherein the sum of the gradient value of the reference point estimated position and the gradient value of the position before or after the reference point estimated position on the axis is set to be equal to or greater than a predetermined reference point. A set of instructions for generating a two-dimensional projection image by performing volume rendering on three-dimensional image data;
A set of instructions for detecting a feature point corresponding to a reference point in the two-dimensional projection image using the learned neural network model;
a command set for extracting voxel values of voxels having the same two-dimensional coordinate value as the feature point from the 3D image data;
a set of instructions for obtaining gradient values of the extracted voxel values with respect to an axial direction passing through the feature point and perpendicular to the two-dimensional projection image, and
A set of instructions for detecting a position on the axis at which the voxel values and the gradient values satisfy a predetermined condition as the position of the reference point
A computer-readable recording medium comprising a. A two-dimensional projection image generator for generating a two-dimensional projection image by performing volume rendering on three-dimensional image data;
A feature point detection unit that detects a feature point corresponding to a reference point in the two-dimensional projection image using the learned neural network model;
a voxel value extractor configured to extract voxel values of voxels having the same two-dimensional coordinate value as the feature point from the 3D image data;
a gradient value calculator for obtaining gradient values of the extracted voxel values in an axial direction passing through the feature point and perpendicular to the two-dimensional projection image; and
a reference point detection unit configured to detect a position on the axis at which the voxel values and the gradient values satisfy a predetermined condition as the position of the reference point
A three-dimensional reference point detection device comprising a. 16. In claim 15,
The two-dimensional projection image generating unit,
A three-dimensional reference point detection apparatus for generating the two-dimensional projection image by performing volume rendering on the three-dimensional image data through a transfer function for visualizing a bone. 17. In claim 16,
The two-dimensional projection image generating unit,
In the 3D image data, a preprocessing of designating voxel values of voxels located in a direction opposite to the region of interest with respect to a reference plane as values to be transparently processed, and volume rendering of the preprocessed 3D image data are performed to the 2D image data. A three-dimensional reference point detection device for generating a projected image. In claim 17,
The reference plane is a three-dimensional reference point detection device that is defined as a sagittal plane, a coronal plane, a horizontal plane, or a midsagittal plane. In claim 17,
The 3D image data is
A three-dimensional reference point detection device that is any one of CT, CBCT, MRI, PET, and SPECT. In claim 17,
The predetermined condition is
A three-dimensional reference point detection apparatus that is differently determined according to the characteristics of the reference point. In claim 17,
The neural network model is
A three-dimensional reference point detection apparatus trained to detect a feature point through reinforcement learning using learning data consisting of a two-dimensional projection image obtained by performing volume rendering on three-dimensional image data. In claim 17,
The learned neural network model is,
A three-dimensional reference point detection apparatus comprising a plurality of neural network models trained to detect feature points for each view direction in which a two-dimensional projection image is generated. In claim 17,
The neural network model is
3D reference point detection in reinforcement learning using the value obtained by subtracting the distance between the _{current estimation point (P i} ) and the feature _{point (P t} ) from the distance between the previous estimation point (P _i-1 ) and the feature _{point (P t ) as compensation} Device. In claim 17,
The 3D image data is
A three-dimensional reference point detection device obtained by photographing a patient's head. 25. In claim 24,
The two-dimensional projection image is,
A three-dimensional reference point detection device that is one of a front view, a rear view, a top view, a bottom view, a left view, and a right view of the patient's head. In claim 17,
The voxel value is a three-dimensional reference point detection device that is a HU value. 27. In claim 26,
The predetermined condition is
When the reference point is outside the bone,
The voxel value of the estimated position of the reference point is within the HU value range corresponding to the bone,
and a sum of the gradient value of the reference point estimated position and the gradient value of the position before or after the estimated reference point position on the axis is set to be equal to or greater than a predetermined reference point.

Images