JP7263324B2 - Method and program for generating 3D brain map - Google Patents

Description

The present invention relates to a method and program for generating a three-dimensional brain map.

An MRI system is a device that obtains an image of a tomographic region of a target body by expressing the intensity of MR (Magnetic Resonance) signals with respect to RF (Radio Frequency) signals generated in a magnetic field of a specific strength by light and dark contrast. For example, if the object is placed in a strong magnetic field and momentarily irradiated with an RF signal that resonates only a specific nucleus (for example, a hydrogen nucleus) and then interrupted, MR can be obtained from the specific nucleus. A signal is emitted and an MRI system can receive this MR signal to acquire an MR image. An MR signal means an RF signal emitted from an object. The magnitude of the MR signal can be determined by the concentration of a given atom (eg, hydrogen) contained in the object, relaxation time T1, relaxation time T2, and flow such as blood flow.

MRI systems include features that distinguish them from other imaging devices. Unlike imaging devices such as CT, where image acquisition depends on the orientation of the detecting hardware, MRI systems can acquire 2D or 3D volumetric images directed at any point. . In addition, unlike CT, X-ray, PET, and SPECT, the MRI system does not expose the subject and the examiner to radiation, and is capable of acquiring images with high soft tissue contrast and detecting abnormal tissue. It is possible to acquire neurological images, intravascular images, musculoskeletal images, and oncological images, for which clear delineation of is important.

Transcranial Magnetic Stimulation (TMS) is a non-invasive therapeutic method for the nervous system, and has the advantage of treating nervous system diseases without drugs or invasive treatments. TMS can use changes in magnetic fields to provide electrical stimulation to a subject.

In general, TMS has been performed by giving electrical stimulation to a clinically or empirically known stimulation point, or by allowing a user to move the stimulation point little by little to determine the stimulation point. Therefore, there is a problem that it is difficult to reflect the type of coil used for the treatment and the difference in body structure between individuals, and it is difficult to directly confirm the effect of the treatment.

In addition, an electroencephalogram (EEG), which can measure electrical activity due to brain activity of a subject, and a method of treating brain diseases by electrical stimulation are widely used. However, as with TMS, EEG and electrical stimulation are also required to develop a guide method that reflects the shape of the head, which varies from person to person.

The present invention has been made in view of the above circumstances, and an object thereof is to provide a method and program for generating a three-dimensional brain map.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those of ordinary skill in the art from the following description.

According to one aspect of the present invention, there is provided a method for generating a 3D brain map for solving the above-described problems. generating a three-dimensional brain image of the subject including the plurality of regions using the obtained brain MRI image; and based on properties of each of the plurality of regions included in the three-dimensional brain image, and generating a 3D brain map of the subject that can simulate a transmission process of electrical stimulation to the brain of the subject, wherein the segmentation is learned using a plurality of processed brain MRI images. obtaining a segmented brain MRI image of the subject by inputting the brain MRI image of the subject into the model.

Further, the processed brain MRI image is an image in which each of a plurality of regions included in the processed brain MRI image is labeled, and the learned model receives an input of the brain MRI image. It may be a model that outputs a segmented brain MRI image.

Further, the step of generating the 3D brain map of the target comprises a plurality of meshes capable of simulating the transmission process of electrical stimulation to the brain of the target using the 3D brain image of the target. generating a rendered three-dimensional stereoscopic image.

Also, generating the 3D brain map of the object includes acquiring physical characteristics of each of the plurality of regions for simulating current flow due to electrical stimulation to the brain of the object, The physical properties may include at least one of isotropic electrical conductivity and anisotropic electrical conductivity of each of the plurality of regions.

Further, the step of obtaining the physical characteristics includes obtaining a conduction tensor image of the subject's brain from the subject's brain MRI image; and obtaining an anisotropic electrical conductivity.

Also, the brain MRI image of the subject includes a diffusion tensor image, and the step of acquiring the physical properties includes determining the anisotropic electrical conductivity of each of the plurality of regions using the diffusion tensor image of the subject. It may include the step of obtaining.

Further, the method for generating a three-dimensional brain map is characterized in that when a specific electrical stimulus is applied to one point on the head of the subject using the three-dimensional brain map, the specific electrical stimulus is applied to the subject. may further include simulating states propagated from the brain of the .

Also, the method for generating the 3D brain map includes the steps of: obtaining a stimulation target point to which electrical stimulation is applied in the brain of the subject; and applying electrical stimulation to the stimulation target point using the 3D brain map. obtaining a location for applying electrical stimulation to the subject's head to apply.

Further, the step of obtaining the position to apply the electrical stimulation includes obtaining a recommended route for the electrical stimulation to be transmitted from the subject's scalp to the stimulation target point using the 3D brain map. and obtaining a position for applying electrical stimulation to the head of the target from the recommended path.

A computer program according to one aspect of the present invention for solving the above-described problems is a program stored in a computer-readable recording medium, which is hardware, and for executing a method for generating a three-dimensional brain map by the computer. be.

Other specifics of the invention are contained in the detailed description and drawings.

According to the disclosed embodiment, segmentation of a brain MRI image is performed using a model learned in advance, thereby achieving an effect of automatically segmenting a brain MRI image in a short period of time.

As a result, anyone in the medical field can acquire a three-dimensional brain image of the subject in a short period of time, and furthermore, it is possible to provide simulation results that allow visual confirmation of the effect of electrical stimulation on the subject's brain.

In addition, by utilizing magnetic vector potential information according to the type of coil used for transcranial magnetic stimulation, it is possible to calculate and provide stimulation points that can give optimal stimulation to the stimulation target points, thereby improving the effect of the treatment. It has the effect of being able to

Further, by using head modeling and MRI modeling to guide the position of the electrical stimulation pad, it is possible to guide the position of the electrical stimulation pad in consideration of the structures of the head and brain that differ from person to person.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those of ordinary skill in the art from the following description.

1 is a flowchart illustrating a method of generating a 3D brain map according to one embodiment; 4 is a flowchart illustrating a method of generating a 3D brain map of a target object and performing a simulation according to one embodiment; FIG. 10 is a diagram showing the results of segmentation of brain MRI images; FIG. 2 illustrates an example of a noise removal method based on connected components; FIG. 10 is a diagram illustrating an example of a post-processing scheme using hole rejection; FIG. 2 is a diagram showing an example of a three-dimensional brain image generated from a brain MRI image of a subject; It is a figure which shows an example of a diffusion tensor image. It is a figure which shows an example of a simulation result. FIG. 4 is a flowchart illustrating a TMS stimulus navigation method according to one embodiment; FIG. It is a figure which shows an example of the TMS treatment method. FIG. 4 is a diagram showing the relationship between a magnetic field and an electric field applied to the brain of a subject; FIG. 5 is a diagram showing information visualizing magnetic vector potentials according to types of treatment coils. It is a figure which shows an example of the method of calculating the position and direction of a coil. FIG. 4 is a diagram showing an example of visualizing a state in which an electrical stimulation induced from a magnetic field of a treatment coil is propagated from the brain of a subject; 4 is a flowchart illustrating a pad guiding method according to one embodiment; FIG. 10 is a diagram showing results of simulating electrical stimulation results according to one embodiment; FIG. 3 illustrates one embodiment of a method for aligning images; FIG. 2 shows an example of a 3D scan model acquired using a depth camera; FIG. 5 is a diagram showing an example of guiding a position for attaching a pad to the photographed head of a subject photographed by a computer device to which a depth camera module is connected; Fig. 2 shows a portable computing device and a depth camera module connected thereto;

Advantages and features of the present invention, as well as the manner in which they are achieved, will become apparent with reference to the embodiments described in detail below in conjunction with the accompanying drawings. The present invention may, however, be embodied in various different forms and should not be construed as limited to the embodiments set forth below. However, the Examples are provided so that this disclosure of the present invention will be complete and will fully comprehend the scope of the present invention by those of ordinary skill in the art to which the present invention pertains. defined only by the scope of the

The terminology used herein is for the purpose of describing examples and is not intended to be limiting of the invention. In this specification, the singular also includes the plural unless specifically stated otherwise. As used herein, "comprises" and/or "comprising" does not exclude the presence or addition of one or more other elements besides the stated elements. Like reference numbers refer to like elements throughout the specification, and "and/or" includes each and every combination of one or more of the referenced elements. Of course, even though "first," "second," etc. are used to describe various components, these components are not limited by these terms. These terms are only used to distinguish one component from another. Therefore, it is natural that the first component referred to below can also be the second component within the technical concept of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used herein have the meaning commonly understood by one of ordinary skill in the art to which this invention belongs. Also, terms defined in commonly used dictionaries should not be ideally or unduly interpreted unless explicitly defined otherwise.

As used herein, the term "unit" or "module" refers to a hardware component such as software, FPGA, or ASIC, where the "unit" or "module" performs a specific role. However, the terms "unit" or "module" are not meant to be limited to software or hardware. A "portion" or "module" may be configured to reside on an addressable storage medium and may be configured to be implemented as one or more processors. Thus, by way of example, a "part" or "module" refers to components such as software components, object-oriented software components, class components and task components, processes, functions, attributes, procedures, subroutines, Includes segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays and variables. Functionality provided within components and "sections" or "modules" may be combined with fewer components and "sections" or "modules", or may be combined with additional components and "sections" or "modules", etc. can be further separated into

As used herein, an "object" may include a person or animal, or a part of a person or animal. For example, the subject can include organs such as the liver, heart, uterus, brain, breast, abdomen, or blood vessels. A "object" can also include a phantom. By phantom is meant a material having a volume that closely resembles the density and effective atomic number of an organism, and can include sphere phantoms with properties similar to the body.

Also, as used herein, a "user" is a medical professional such as a doctor, a nurse, a clinical laboratory technician, a medical imaging specialist, and can be, but is not limited to, a technician who repairs medical equipment. not.

Further, the term "magnetic resonance image (MR image)" as used herein means an image of an object acquired using the principle of nuclear magnetic resonance.

Embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

FIG. 1 is a flowchart illustrating a method for generating a 3D brain map according to one embodiment.

The method illustrated in FIG. 1 is a diagram illustrating steps performed by a computer in chronological order. Computer is used herein as a generic term for computer devices that include at least one processor.

At step S110, the computer acquires a brain MRI image of the subject.

In one embodiment, the computer is a workstation connected to the MRI image acquisition device and capable of acquiring brain MRI images of the subject directly from the MRI image acquisition device.

The computer can also acquire brain MRI images of the subject from an external server or other computer.

In the disclosed embodiment, a brain MRI image of a subject means an MRI image of the head including the brain of the subject. That is, the brain MRI image of the subject means an MRI image including not only the brain of the subject but also the skull and scalp of the subject.

At step S120, the computer segments the brain MRI image obtained at step S110 into multiple regions.

In one embodiment, the computer segments the brain MRI image obtained in step S110 by region. For example, the computer can segment the brain MRI image obtained in step S110 into white matter, gray matter, cerebrospinal fluid, skull and scalp, but the types of segmentation of the brain MRI image are not limited thereto.

In one embodiment, the computer obtains segmented brain MRI images of the subject by inputting the subject's brain MRI images into a model trained using a plurality of processed brain MRI images.

In one example, the processed brain MRI image is an image in which each of the plurality of regions included in the brain MRI image is labeled. Also, the learned model is a model that receives an input of a brain MRI image and outputs a segmented brain MRI image.

In one embodiment, a trained model means a model trained using machine learning, and in particular, a model trained using deep learning.

In one embodiment, the learned model may be a model including one or more Batch Normalization layer, Activation layer and Convolution layer, but is not limited to this. not.

In one embodiment, the trained model comprises a horizontal pipeline composed of multiple blocks that extract high-level features from the low-level features of the MRI image, and aggregates the extracted features from the horizontal pipeline to perform the segmentation. It can also be configured to include a vertical pipeline to allow relatively poor quality MRI segmentation.

Referring to FIG. 3, a segmentation result 300(b) of a brain MRI image 300(a) is shown.

In one embodiment, the computer performs post-processing on the segmentation results.

In one embodiment, the computer performs Connected Component-based Noise Rejection. Connected component-based denoising methods are used to improve the results of segmentation performed using a Convolution Neural Network (CNN).

Referring to FIG. 4, an example of a noise removal method based on connected components is shown.

The computer obtains an enhanced segmentation image 410 by removing the remaining components 402 excluding the connected component, which is the largest chunk in the segmentation image 400 .

In one embodiment, the computer performs Hole Rejection. Hole rejection is used to remove holes, which are one of the errors in segmentation based on convolutional neural networks.

Referring to FIG. 5, an example post-processing scheme using hole rejection is shown.

The computer removes at least a portion of the holes 502 contained in segmentation image 500 to obtain enhanced segmentation image 510 .

At step S130, the computer generates a three-dimensional brain image of the subject including the segmented regions using the brain MRI image of the subject segmented at step S120.

Referring to FIG. 6, a three-dimensional brain image 600 generated from brain MRI images of a subject is shown.

FIG. 6 shows an example of the result of generating a 3D brain image 610 of the segmented object from the 2D brain MRI image of the segmented object.

In step S140, the computer performs 3-dimensional imaging of the object, which can simulate the transmission process of electrical stimulation to the brain of the object, based on the properties of each of the plurality of regions included in the 3D brain image generated in step S130. Generate a dimensional brain map.

A specific method of generating a three-dimensional brain map of an object and performing a simulation using the generated brain map will be described later with reference to FIG.

FIG. 2 is a flow chart illustrating a method of generating a 3D brain map of a target and performing a simulation according to one embodiment.

The method shown in FIG. 2 corresponds to one embodiment of the method shown in FIG. Therefore, the content described in connection with FIG. 1 also applies to the method shown in FIG. 2, even if the content is omitted in connection with FIG.

In step S210, the computer generates a 3D stereoscopic image composed of a plurality of meshes capable of simulating the transmission process of electrical stimulation to the brain of the target using the 3D brain image of the target.

In one embodiment, a computer generates a three-dimensional stereoscopic image made up of a plurality of surface meshes containing triangles or squares.

In one embodiment, a computer generates a three-dimensional stereoscopic image composed of a plurality of volumetric meshes containing tetrahedrons or hexahedrons.

The type of lattice that forms the 3D stereoscopic image can be set differently depending on the application of the simulation.

In step S220, the computer acquires physical properties of each of the plurality of regions for simulating current flow due to electrical stimulation to the brain of the subject.

In one embodiment, the physical properties obtained in step S220 include at least one of isotropic electrical conductivity and anisotropic electrical conductivity of each of the plurality of segmented regions. .

In one embodiment, the isotropic electrical conductivity can be obtained by assigning an experimentally known electrical conductivity to each segmented region.

For example, the electrical conductivity known for each region of the brain is shown in Table 1 below.

Anisotropic electrical conductivity is a realization of the anisotropy of the white matter fibers in the white matter of the brain.

In one embodiment, the anisotropic electrical conductivity is obtained from a conduction tensor image for the subject's brain.

For example, the computer acquires a conduction tensor image of the subject's brain from a brain MRI image of the subject, and acquires the anisotropic electrical conductivity of each of the segmented regions using the acquired conduction tensor image.

In another embodiment, the brain MRI image of the subject includes a diffusion tensor image, and the computer obtains the anisotropic electrical conductivity of each of the regions segmented using the diffusion tensor image of the subject.

Referring to FIG. 7, an example diffusion tensor image 700 is shown.

Since the eigenvectors of the diffusion tensor image are known to match the eigenvectors of the conduction tensor, the computer can obtain the anisotropic electrical conductivity depending on the orientation of the nerve fibers contained in the diffusion tensor image. For example, the direction of the nerve fibers has high electrical conductivity and the direction perpendicular to the nerve fibers has low electrical conductivity.

In step S230, the computer uses the 3D brain map to determine a state in which a specific electrical stimulus is propagated from the brain of the subject when a specific electrical stimulus is applied to one point on the head of the subject. to simulate.

In one embodiment, the computer uses the grid image obtained in step S210 and the physical properties obtained in step S220 to simulate the state in which electrical stimulation propagates from the subject's brain.

Referring to FIG. 8, an example of simulation results is shown.

The electrical stimulation that can be applied to the subject's head can include at least one of a magnetic field, an electric field, and an electric current, and when the magnetic field is applied to the subject's head, the subject's brain is induced by the magnetic field. current can be propagated.

In one embodiment, the computer acquires stimulation target points for electrical stimulation in the subject's brain. The computer uses the subject's three-dimensional brain map to acquire locations for applying electrical stimulation to the subject's head in order to apply electrical stimulation to stimulation target points.

For example, the computer acquires a recommended route for the electrical stimulation to be transmitted from the scalp of the subject to the stimulation target point using the 3D brain map of the subject, and conducts the electrical stimulation from the recommended path to the head of the subject. Acquire a stimulating position.

A method of calculating and providing the position and direction for the computer to apply electrical stimulation to the brain of the subject will be described below.

FIG. 9 is a flowchart illustrating a Transcranial Magnetic Stimulation (TMS) stimulation navigation method according to one embodiment.

The TMS stimulus navigation method shown in FIG. 9 is a timeline of computer-implemented steps.

Referring to FIG. 10, an example of a TMS treatment method is shown.

TMS is a treatment method in which a treatment coil 1000 is placed adjacent to one side of the head of the subject 10, and an electric field induced in the brain of the subject 10 by a magnetic field generated by the coil 1000 is used to stimulate a specific part of the brain. is.

Since the strength and shape of the magnetic field generated around the coil 1000 are different depending on the shape of the treatment coil 1000, the manner in which the electric signal is propagated should also be different depending on the shape of the head and brain of the subject 10. FIG.

Therefore, according to the disclosed embodiment, stimulation points are calculated and provided according to the type of the coil 1000, and simulation results are provided according to the shape of the head and brain of the subject 10. FIG.

In step S910, the computer acquires a stimulation target point for applying electrical stimulation to the brain of the subject.

The stimulation target point is selected clinically or rationally depending on the disease to be treated. In one embodiment, stimulation target points are indicated using a 3D brain image or 3D brain map of the subject generated by the disclosed embodiments.

At step S920, the computer acquires information about the spatial distribution of the Magnetic Vector Potential of the TMS treatment coil.

In one embodiment, the information about the spatial distribution includes information obtained by visualizing the magnetic vector potential using a magnetic dipole due to the shape of the treatment coil.

Referring to FIG. 12, information (1210 and 1260) that visualizes the magnetic vector potential according to the types of treatment coils (1200 and 1250) is shown.

At step S930, the computer obtains one or more parameters for obtaining optimal stimulation conditions for the stimulation target points obtained at step S910 from the spatial distribution obtained at step S920.

In one embodiment, the optimum stimulation condition for the stimulation target point means a condition that maximizes the strength of the magnetic field applied to the stimulation target point by the treatment coil.

Referring to FIG. 11, the relationship between the magnetic field and the electric field applied to the brain of the subject is shown.

Referring to the simulation image 1100(a) of FIG. 11, the image visualizes the magnitude of the magnetic field applied to the brain of the subject, the magnitude of the gradient (potential), and the magnitude of the electric field induced by the magnetic field. each shown. The magnitude of the electric field applied to the subject's brain can be calculated by adding the magnetic field applied to the subject's brain and the gradient.

Referring to graph 1100(b) of FIG. 11, the correlation between the magnetic field applied to the subject's brain and the electric field induced by the magnetic field is shown.

According to the graph 1100(b), it can be seen that as a stronger magnetic field is applied to the brain of the subject, a stronger electric field is induced in the brain of the subject.

Therefore, it can be seen that the optimum stimulation condition for the stimulation target point is to maximize the strength of the magnetic field applied to the stimulation target point by the treatment coil.

In one embodiment, the parameters obtained by the computer include the Optimal Point with the highest magnetic vector potential value in the spatial distribution of the magnetic vector potential induced by the coil.

In addition, the parameters acquired by the computer are the Optimal Vector, which is the normal vector that minimizes the product of the gradient at the optimum point among the normal vectors starting from the optimum point. )including.

Referring to FIG. 12, the optimum points (1212 and 1262) and optimum vectors (1214 and 1264) for the magnetic vector potentials (1210 and 1250) respectively are shown.

The optimum point (x, y, z) and the optimum vector v are calculated by Equations 1 and 2 below.

In Equation 1, f means the magnetic vector potential map, and by Equation 1, the position (x, y, z) having the largest value in the magnetic vector potential map f is calculated as the optimum point.

は最適点を定義する際に用いられたｆを最適点である

で微分した値であり、ｖ（ｘ、ｙ、ｚ）は（ｘ、ｙ、ｚ）方向への法線ベクトルを意味する。 In Equation 2,

is the optimal point f used in defining the optimal point

, and v(x, y, z) means a normal vector in the (x, y, z) direction.

At step S940, the computer uses the parameters obtained at step S930 to calculate the position and orientation of the coil that satisfies the optimum stimulation condition for the stimulation target point obtained at step S910.

In one embodiment, calculating the position and orientation of the coil includes calculating the position and orientation of the coil that causes the stimulation target point to be closest to the optimum vector direction from the optimum point.

Referring to FIG. 13, an example method for calculating the position and orientation of the coil is shown.

Once the subject 10 and the stimulation target point (S, 12) of the subject 10 are obtained, the computer determines a point 14 on the scalp that is closest to the stimulation target point 12 .

At this time, let D be the distance between the stimulation target point 12 and a point 14 on the scalp closest to the stimulation target point 12, and let K be a vector with the point 14 as the starting point and the stimulation target point 12 as the end point. Also, the thickness of the coil 1310 is assumed to be 2P.

The computer generates and applies a matrix such as Equation 3 below that aligns vector K 1320 with the optimum vector 1312 of coil 1310 .

Accordingly, the position of the coil is calculated as shown in Equation 4 below.

In step S950, the computer causes the electrical stimulation induced by the magnetic field of the treatment coil to be applied to the brain of the subject when the treatment coil is positioned in the position calculated in step S940 and in the direction calculated in step S950. Simulate the state propagated from .

In one embodiment, a computer performs a simulation using a 3D brain map generated by the method shown in FIGS.

For example, the computer can acquire a brain MRI image of the subject and simulate the transmission process of electrical stimulation to the brain of the subject based on the properties of each of a plurality of regions included in the acquired brain MRI image. A 3D brain map of the subject can be generated.

The computer uses the generated three-dimensional brain map to simulate the state in which electrical stimulation by the coil is propagated from the subject's brain.

Also, the 3D brain map may include a 3D stereoscopic image composed of a plurality of grids capable of simulating the transmission process of electrical stimulation to the brain of the subject.

In one embodiment, the computer visualizes, using a three-dimensional stereoscopic image, how electrical stimulation induced by the magnetic field of the treatment coil propagates from the subject's brain.

Referring to FIG. 14, there is shown an example of visualizing a state in which electrical stimulation induced by the magnetic field of the treatment coil is propagated from the subject's brain.

In the disclosed embodiment, the computer is connected to a robotic arm device equipped with TMS treatment coils. The robotic arm device includes a mechanical device that can move the TMS treatment coil to a computer-specified position.

By moving the TMS treatment coil to a position specified by the computer according to the disclosed embodiment, the robot arm device can automatically treat the patient using the TMS coil according to the calculation result of the computer.

FIG. 15 is a flowchart illustrating a pad guiding method according to one embodiment.

In the disclosed embodiment, the pads include brain stimulation pads. For example, brain stimulation pads can include, but are not limited to, electrical stimulation pads and ultrasonic stimulation pads. The pads also include EEG pads. However, the types of pads according to the disclosed embodiments are not limited to the examples given above.

At step S1510, the computer acquires a 3D scan model including the head of the object using a depth camera.

The depth camera may include a triangulation type 3D laser scanner, a depth camera using a structured ray pattern, a depth camera using a TOF (Time Of Flight) method using a reflection time difference of infrared rays, and the like. possible, but the type is not limited to this.

Depth cameras are used to reflect depth information in images to obtain a 3D scan model.

In one embodiment, the subject, i.e., the patient, is seated in a round chair with no backrest, and the user, i.e., the doctor, uses a temporary fixation device such as a triangular table to view the depth camera from the height of the patient's face to the patient's face. Position it so that you can see it clearly.

The doctor initiates a scan with a depth camera and slowly rotates the patient once to obtain a three-dimensional scan model containing the patient's head.

In one embodiment, the depth camera can be placed in an auto-rotatable stationary module to acquire a 3D scan model by rotating around the patient in which the depth camera is centrally located.

On the other hand, according to the disclosed embodiments, a depth camera module can be connected to a portable computing device (e.g., smart phone, tablet PC, etc.) so that 3D scanning can be performed without a separate expensive device. A 3D scan model can be acquired by fixing a computer device to which a depth camera module is connected using a readily available temporary fixation device such as a triangular table, and then seating the patient on a stool or the like and then rotating it.

Referring to FIG. 20, there is shown a portable computing device 2000 and a depth camera module 2010 connected thereto.

Also referring to FIG. 18, an example of a 3D scan model 1800 acquired using a depth camera is shown.

In one embodiment, the computer generates a three-dimensional model containing the subject's head using the range images collected using the depth camera, and aligns the images taken at different starting points with each other and then merges them. to reconstruct a three-dimensional model of the object. For example, a computer collects 3D data in the form of a point cloud from range images collected using a depth camera to reconstruct a model. However, the method of generating the three-dimensional model is not limited.

At step S1520, the computer acquires a 3D brain MRI model of the subject.

In one embodiment, acquiring a three-dimensional brain MRI model of the subject includes acquiring a brain MRI image of the subject and the properties of each of the plurality of regions included in the subject's brain MRI image. and generating a 3D brain map of the subject, which can simulate the transmission process of electrical stimulation to the brain of the subject.

Also, generating the 3D brain map of the target includes generating a 3D stereoscopic image composed of a plurality of meshes capable of simulating a transmission process of electrical stimulation to the brain of the target.

The method for obtaining the 3D brain MRI model of the object by the computer in step S1520 can use the 3D brain map generation method described with reference to FIGS.

At step S1530, the computer aligns the 3D scan model including the subject's head with the subject's brain MRI model.

Referring to FIG. 17, one embodiment of a method for aligning images is shown. Referring to the image 1700 shown in FIG. 17, the brain MRI image of the subject and the modeled image of the brain structure of the subject are superimposed.

In the image 1700, the lower three images correspond to an example in which the brain MRI photograph and the modeled image of the brain structure are not matched. Also, in the image 1700, the upper three images correspond to an example in which brain MRI photographs and brain structure modeling images are matched.

The computer uses a brain MRI model to calculate the changes that occur in the subject's brain due to electrical or ultrasonic stimulation of the pads depending on where the pads are applied. The computer also uses a three-dimensional scan model including the subject's head to calculate where the pads should actually be applied.

Accordingly, the computer calculates the positions at which the subject's head should be padded by performing a match to the subject's brain MRI model with a three-dimensional scan model that includes the subject's head, thereby calculating the positions in the subject's brain. You can calculate the changes that occur. Similarly, the computer can use the aligned results to calculate and provide the results of the positions at which the subject's head should be padded to cause a particular change in the subject's brain.

In one embodiment, the step of computer matching includes calculating facial features of the scan model and the brain MRI model; including matching.

Since the scan model including the subject's head and the subject's brain MRI model are in different modalities, it is difficult to match them. Therefore, the computer can match both models using the subject's facial characteristics.

In one embodiment, calculating facial features of a scan model including the subject's head includes acquiring a color image and a depth image including the subject's head, and using the color image including the subject's head. The method includes calculating facial features of the object, and calculating three-dimensional locations of the facial features of the object using a depth image including the head of the object.

Referring to FIG. 18, an example of matching a scan model 1800 including a subject's head and a subject's brain MRI model 1810 to produce a matched model 1820 is shown.

In step S1540, the computer obtains an image of the head of the object using the depth camera.

For example, a doctor can directly hold a temporarily fixed depth camera and move it while viewing the patient's head.

At step S1550, the computer matches a location on the model that was aligned with a location on the image taken at step S1540.

For example, if the computer is using a depth camera to image a point on the subject's head, the computer performs a calculation as to where the imaged point falls on the aligned model.

In one embodiment, the computer displays an image that matches the captured image with the aligned model to guide the location of a pad on the subject's head.

Referring to FIG. 19, a computer device 1900 to which a depth camera module is connected captures a head 1910 of a subject, and the computer device 1900 is positioned to attach a pad 1930 to the head 1910 of the captured subject. An image that guides 1920 is displayed.

In one embodiment, the computing device 1900 determines the positions to apply the pads 1930 on the aligned model and displays the positions 1920 corresponding to the determined positions in the captured image.

Also, the computer device 1900 recognizes the pad 1930 in the captured image and guides the movement direction of the recognized pad 1930 .

Further, computing device 1900 determines whether recognized pad 1930 has been applied to determined location 1920 .

In one embodiment, pad 1930 is marked or otherwise marked with at least one marker. For example, at least one of a specific figure, color, and two-dimensional code is attached or displayed on the pad 1930, and the computer device 1900 recognizes the pad 1930 using a marker attached or displayed on the pad 1930. and track the movement of pad 1930 .

For example, when the doctor photographs the patient's head while changing the position using the computer device 1900 or a depth camera connected to the computer device 1900, the position of the patient's head displayed on the computer device 1900 is also changed, Similarly, the position of pad 1930 recognized by computing device 1900 is also changed. In this case, the computing device 1900 tracks the pad 1930 even when the computing device 1900 moves, and guides the physician to apply the pad 1930 to the correct location on the patient's head.

In one embodiment, the computing device 1900 recognizes the pad 1930 in the captured image and guides the movement direction of the recognized pad. For example, computing device 1900 displays the direction of movement of pad 1930 so that pad 1930 can be applied to determined position 1920 .

Computing device 1900 also determines whether recognized pad 1930 has been applied to determined location 1920 . For example, the computing device 1900 determines whether the final recognized position of the pad 1930 corresponds to the determined position 1920, and if the determined position 1920 and the position of the pad 1930 are different, A notification can be provided requesting that the pad 1930 be repositioned.

In one embodiment, the computing device 1900 recognizes the pad 1930 attached to the head of the subject in the captured image and determines the position of the recognized pad 1930 . Computing device 1900 obtains the position on the matched model corresponding to the determined pad 1930 position.

For example, when performing EEG electroencephalography, the EEG pads are placed in a consistent position or in any position regardless of the shape and structure of the user's head. In this case, it is difficult to know from which part of the subject's brain the brain waves acquired by the EEG pad are received.

Thus, according to the disclosed embodiments, the computing device 1900 can image a subject's head with one or more EEG pads attached to it, and determine the locations of the one or more EEG pads recognized in the captured image. Earn.

The computing device 1900 obtains the positions on the matched model of the subject corresponding to the obtained positions of the EEG pads, and the electroencephalograms obtained with the EEG pads attached to the subject's head are obtained from the subject's brain. It is possible to specifically determine from which part the message was received. For example, computing device 1900 can utilize the disclosed embodiments to analyze the source of brain waves received at each EEG pad.

FIG. 16 is a diagram showing results of simulating electrical stimulation results according to one embodiment.

Referring to FIG. 16, there is shown an embodiment of a 3D model of a subject's head 1600 and a pad 1610 at a position on the 3D model.

When the pad 1610 is attached to one position of the three-dimensional model of the subject's head 1600, the computer simulates the result of electrical stimulation transmitted from the subject's brain 1650 by the pad 1610. FIG.

In one embodiment, the computer obtains a 3D brain map for the subject's brain 1650 and uses the 3D brain map to determine the location of the pad 1610 applied to the subject's head.

In one embodiment, determining the position of the pad 1610 includes obtaining a purpose for using the pad 1610 and electrically stimulating the brain 1650 of the subject according to the location where the pad 1610 is applied to the head 1600 of the subject. and determining the position of the pad 1610 using the obtained objective and the simulation result.

For example, when the computer intends to apply a specific stimulation to the brain 1650 of the subject, the computer can use the simulation results to determine the position of the pad 1610 where the specific stimulation is applied to the brain 1650 of the subject.

The computer matches the position of the pad 1610 determined by the embodiment shown in FIG. 16 to a point on the subject's head captured using the depth camera and displays an image guiding the pad to the matched position. can.

The method or algorithm steps described in connection with the embodiments of the present invention may be implemented directly in hardware, implemented in software modules executed by hardware, or a combination thereof. The software module includes RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), Flash Memory, hard disk, removable disk, CD-ROM, Or it may reside on any form of computer readable recording medium that is well known in the art to which this invention pertains.

Although the embodiments of the present invention have been described above with reference to the accompanying drawings, those skilled in the art to which the present invention pertains will appreciate that the present invention may be modified in other ways without changing its technical idea or essential features. It will be understood that it can be implemented in the specific form of Accordingly, the above-described embodiments are to be considered in all respects as illustrative and not restrictive.

Claims (10)

obtaining a brain MRI image of a subject;
segmenting the brain MRI image into a plurality of regions;
generating a three-dimensional brain image of the subject including the plurality of regions using the segmented brain MRI image;
generating a 3D brain map of the subject that can simulate a transmission process of electrical stimulation to the brain of the subject based on the properties of each of a plurality of regions included in the 3D brain image;
including
The segmenting step includes:
acquiring a segmented brain MRI image of the subject by inputting the brain MRI image of the subject into a trained model obtained by machine learning using a plurality of processed brain MRI images;
The trained model is segmented by brain region based on the structure of the brain when the brain MRI image is input by performing learning based on deep learning using the plurality of processed brain MRI images. A learning model constructed to output brain MRI images,
The step of generating a 3D brain map of the object includes: using a segmented brain MRI image of the object obtained by inputting the brain MRI image of the object into the trained model; outputting a 3D brain map image of the subject segmented by brain regions based on
The segmenting step includes:
removing holes included in the segmented brain MRI image of the subject obtained by inputting the brain MRI image of the subject into the trained model;
A method for generating a three-dimensional brain map. The processed brain MRI image is an image in which each of a plurality of regions included in the processed brain MRI image is labeled;
The trained model is
2. The method of generating a three-dimensional brain map according to claim 1, wherein the model is a model that receives an input of a brain MRI image and outputs a segmented brain MRI image. The step of generating a 3D brain map of the subject includes:
generating a 3D stereoscopic image composed of a plurality of meshes capable of simulating a transmission process of electrical stimulation to the brain of the target using the 3D brain image of the target. 3. The method of generating a three-dimensional brain map according to claim 1. The step of generating a 3D brain map of the subject includes:
acquiring physical characteristics of each of the plurality of regions for simulating current flow due to electrical stimulation to the subject's brain;
2. The method of claim 1, wherein the physical properties include at least one of isotropic electrical conductivity and anisotropic electrical conductivity of each of the plurality of regions. 3D brain map generation method. Obtaining the physical property comprises:
obtaining a conduction tensor image of the subject's brain from the subject's brain MRI image;
obtaining the anisotropic electrical conductivity of each of the plurality of regions using the conduction tensor image;
5. The method of generating a three-dimensional brain map according to claim 4, comprising: the brain MRI image of the subject comprises a diffusion tensor image;
Obtaining the physical property comprises:
obtaining the anisotropic electrical conductivity of each of the plurality of regions using a diffusion tensor image of the object;
5. The method of generating a three-dimensional brain map according to claim 4, comprising: using the three-dimensional brain map to simulate a state in which the specific electrical stimulus is transmitted from the brain of the subject when the specific electrical stimulus is applied to one point on the head of the subject; The method of claim 1, further comprising: obtaining a stimulation target point for applying electrical stimulation to the brain of the subject;
using the 3D brain map to obtain a position for applying electrical stimulation to the subject's head in order to apply electrical stimulation to the stimulation target point;
The method of claim 1, further comprising: Acquiring the location for applying the electrical stimulation comprises:
obtaining a recommended route for electrical stimulation to be delivered from the subject's scalp to the stimulation target point using the 3D brain map;
obtaining a position for applying electrical stimulation to the head of the target from the recommended path;
9. The method of generating a three-dimensional brain map according to claim 8, comprising: A computer program stored in a computer-readable recording medium, which is hardware, for executing the method according to any one of claims 1 to 9 on the computer.

Images