KR20210080244A - Device and method for modeling three-dimensional organ by image segmentation - Google Patents

Abstract

The present invention relates to a method for three-dimensional modeling of an organ through image segmentation performed by an apparatus. The method for three-dimensional modeling of an organ through image segmentation comprises the steps of: receiving medical image data for a specific body organ of an object; setting a region of interest for the body organ based on the medical image data; forming one or more blocks corresponding to the region of interest; setting a segmentation algorithm for each of the blocks; generating first image data of each of the three-dimensional modeling of the parts included in each block based on the algorithm set in each of the blocks; and merging each of the first image data to generate second image data in a three-dimensional form for the entire body.

Description

Device and method for 3D modeling of organs through image segmentation {DEVICE AND METHOD FOR MODELING THREE-DIMENSIONAL ORGAN BY IMAGE SEGMENTATION}

The present invention relates to an apparatus and method for 3D modeling of an organ through image segmentation.

The living organ imitation technology is a technology that imitates internal organs such as the heart, lungs and liver. In vivo organ simulating technology is attracting attention as a technology that can compensate for the shortcomings of existing clinical trials that rely on cultured cells or animal experiments. Through biological organ simulating technology, internal organs of the human body are produced similarly to the real thing, and the surrounding environment is reproduced to implement a system in which the actual organs operate and conduct clinical trials to reflect safety as well as the physiological environment in which actual human organs operate more accurately. It has the advantage of being able to solve the ethical problems, cost, time, and inaccuracy of results that arise from animal experiments.

The organ model is a 3D shape made similar to the actual shape of an organ, and modeling and rendering techniques are required for precise simulation. Modeling refers to the act of creating a 3D model in the virtual space inside a computer using computer graphics, and mainly uses 3D graphic tools. The modeled 3D model is stored as data, not as a real object. Rendering refers to the process of creating an image from a model using a computer program.

Image segmentation refers to a process of dividing a digital image into a plurality of pixel or voxel sets. The purpose of segmentation is to simplify or transform the representation of the image into something more meaningful and easier to interpret. Image segmentation is used to find objects and boundaries in an image. The result of segmentation is a set of regions collectively including the entire image or a set of contours extracted from the image.

Registered Patent Publication No. 10-1986571, 2019.05.31

An object of the present invention is to provide an apparatus and method for 3D modeling of an organ through image segmentation.

More specifically, it is to provide a method and a program for modeling a three-dimensional organ shape that minimizes the manual work area and post-processing work required in the modeling process of a living body and enables rapid processing using a computer program.

In addition, the problem to be solved by the present invention is a method and program for modeling a three-dimensional organ shape that can be accurately modeled so as to minimize the difference between parts of an organ in the biological organ modeling process, and to accurately model even the detailed parts of an organ to be similar to an actual organ. is to provide

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

A three-dimensional modeling method of an organ through image segmentation according to an embodiment of the present invention is a method performed by an apparatus, comprising: receiving medical image data for a specific body organ of an object; based on the medical image data , setting a region of interest for the body organ, forming one or more blocks corresponding to the region of interest, each of the blocks including a portion of the body organ corresponding to the region of interest, the block Setting a segmentation algorithm for each block, based on the algorithm set for each block, generating each first image data obtained by 3D modeling the parts included in each of the blocks, and each of the first images and generating second image data in a three-dimensional form for the entire body by merging the data.

In some embodiments of the present invention, the setting of the region of interest includes recognizing the type of the body organ based on a first model based on deep learning, and automatically generating the region of interest corresponding to the type of the body organ. may be doing

In some embodiments of the present invention, the merging may be performed through image registration, and the image registration may be performed through at least one of a feature element matching technique and a template-based matching technique.

In some embodiments of the present invention, the generating of the first image data includes: if there is an overlapping area between the blocks, the body part included in the overlapping area is selected based on a segmentation algorithm set for each of the blocks. It can be characterized by exploring.

In some embodiments of the present invention, the setting of the region of interest includes dividing the first region into a first region corresponding to a blood region of the body organ and a second region corresponding to a muscle region of the body organ; The method may further include setting a region and a second region as the region of interest for the body organ, wherein the dividing may include: based on the medical image data, a contrast value of the body organ is equal to or greater than a preset first value In this case, the first region corresponding to the blood region may be identified, and if the contrast value is less than a preset second value, the blood region may be identified and divided into a second region corresponding to the muscle region.

An apparatus for 3D modeling of an organ through image segmentation according to another aspect of the present invention for solving the above-mentioned problems is a collection unit that acquires medical image data and an interest in the body organs based on the medical image data Set a region, form one or more blocks corresponding to the region of interest, set a segmentation algorithm for each of the blocks, and based on the algorithm set in each of the blocks, 3D parts included in each of the blocks Each of the modeled first image data is generated, and the respective first image data is merged to generate the second image data in a three-dimensional form for the entire body, wherein each of the blocks corresponds to the region of interest. and parts of the body organs that

In some embodiments of the present invention, when an overlapping region exists between the blocks, the body part included in the overlapping region may be searched based on a segmentation algorithm set in each of the blocks.

In some embodiments of the present invention, setting the region of interest includes recognizing the type of the body organ based on a first model based on deep learning and automatically generating the region of interest corresponding to the type of the body organ. it could be

In some embodiments of the present invention, the image registration may be performed through at least one of a feature element matching technique and a template-based matching technique.

The program for 3D modeling of an organ through image segmentation according to another aspect of the present invention for solving the above-mentioned problems is combined with a computer which is hardware, and is stored in a medium in order to execute the 3D modeling method of the organ through image segmentation. is saved

Other specific details of the invention are included in the detailed description and drawings.

According to the present invention, by designating one or more blocks for regions of interest constituting an organ and applying an optimized algorithm to each block, manual work and post-processing can be reduced, and segmentation can be performed simultaneously on a plurality of regions of interest. This allows for faster organ modeling work.

In addition, according to the present invention, it is possible to perform segmentation by applying a differentiated algorithm to each block arranged in each of the regions of interest constituting the organ, so that precise and accurate organ modeling work is possible even for complex organs.

Effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a block diagram of an apparatus for 3D modeling of an organ through image segmentation according to an embodiment of the present invention.
2 is a flowchart schematically illustrating a process of a three-dimensional modeling method of an organ through image segmentation according to an embodiment of the present invention.
3 is an exemplary diagram illustrating a basic screen of a program for modeling a three-dimensional organ shape according to an embodiment of the present invention.
4 is an exemplary diagram illustrating output by forming one or more blocks including a region of interest according to an embodiment of the present invention.
5 is an exemplary diagram of performing three-dimensional modeling on the entire photographed image of the heart, which is a target organ.
6 is an exemplary diagram illustrating generation of respective first image data obtained by 3-D modeling parts included in each block based on a segmentation algorithm set for each block according to an embodiment of the present invention.
7 is an exemplary diagram illustrating generation of second image data in a three-dimensional form for the entire cardiac organ by merging respective first image data.

Advantages and features of the present invention and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only these embodiments allow the disclosure of the present invention to be complete, and those of ordinary skill in the art to which the present invention pertains. It is provided to fully understand the scope of the present invention to those skilled in the art, and the present invention is only defined by the scope of the claims.

The terminology used herein is for the purpose of describing the embodiments and is not intended to limit the present invention. As used herein, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, “comprises” and/or “comprising” does not exclude the presence or addition of one or more other components in addition to the stated components. Like reference numerals refer to like elements throughout, and "and/or" includes each and every combination of one or more of the recited elements. Although "first", "second", etc. are used to describe various elements, these elements are not limited by these terms, of course. These terms are only used to distinguish one component from another. Therefore, it goes without saying that the first component mentioned below may be the second component within the spirit of the present invention.

The word "exemplary" is used herein in the sense of "used as an illustration or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

Also, as used herein, the term “unit” refers to a hardware element such as software, FPGA, or ASIC, and “unit” performs certain roles. However, "part" is not meant to be limited to software or hardware. A “unit” may be configured to reside on an addressable storage medium and may be configured to refresh one or more processors. Thus, by way of example, “part” refers to elements such as software elements, object-oriented software elements, class elements, and task elements, and processes, functions, properties, procedures, subroutines, and programs. It includes segments of code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables. The functionality provided within elements and “parts” may be combined into a smaller number of elements and “parts” or further separated into additional elements and “parts”.

In addition, all “units” of the present specification may be controlled by at least one processor, and at least one processor may perform operations performed by the “units” of the present disclosure.

Embodiments of the present disclosure may be described in terms of a function or a block performing a function. Blocks, which may be referred to as 'parts' or 'modules', etc. in the present disclosure include logic gates, integrated circuits, microprocessors, microcontrollers, memories, passive electronic components, active electronic components, optical components, hardwired circuits, and the like. It may be physically implemented by analog or digital circuitry, such as, and optionally driven by firmware and software.

Embodiments of the present disclosure may be implemented using at least one software program running on at least one hardware device and may perform a network management function to control an element.

Spatially relative terms "below", "beneath", "lower", "above", "upper", etc. It can be used to easily describe the correlation between a component and other components. A spatially relative term should be understood as a term that includes different directions of components during use or operation in addition to the directions shown in the drawings. For example, when a component shown in the drawing is turned over, a component described as “beneath” or “beneath” of another component is placed “above” the other component. can get Accordingly, the exemplary term “below” may include both directions below and above. Components may also be oriented in other orientations, and thus spatially relative terms may be interpreted according to orientation.

Unless otherwise defined, all terms (including technical and scientific terms) used herein will have the meaning commonly understood by those of ordinary skill in the art to which this invention belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless specifically defined explicitly.

As used herein, "organ" includes, but is not limited to, heart, stomach, liver, lung, lymph node, tooth, eye, thyroid, ovary, skin, brain, and the like. Part or all of any organ that makes up the body of a particular subject, eg, a human or animal.

As used herein, “segmentation” refers to an operation of finding an object and a boundary in an image. For example, the segmentation may include, but is not limited to, an operation of finding a target part and a boundary by adjusting a brightness value and processing it to be visually distinguished.

In the present specification, 'medical image data' includes magnetic resonance imaging (MRI), computerized tomography (CT), positron emission tomography (PET), and ultrasonography, The present invention is not limited thereto, and includes all image data obtained by photographing a patient's body for medical purposes.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of an apparatus for 3D modeling of an organ through image segmentation according to an embodiment of the present invention.

According to an embodiment of the present invention, the apparatus 100 for modeling an organ through image segmentation may include a collection unit 110 , a memory 120 , and a processor 130 .

The three-dimensional modeling apparatus 100 of the institution through image segmentation is, for example, a computer, an Ultra Mobile PC (UMPC), a workstation, a net-book, a Personal Digital Assistants (PDA), a portable computer, As one of electronic devices such as a web tablet, a wireless phone, a mobile phone, a smart phone, and a portable multimedia player (PMP), the installation and It may include any executable electronic device. The electronic device may perform overall service operations such as, for example, configuration of a service screen, data input, data transmission/reception, data storage, and the like, under the control of the application.

The apparatus 100 for 3D modeling of an institution through image segmentation may acquire medical image data through the communication unit 110 . Here, the medical image data may include an MRI or CT image.

The memory 120 of the present invention is a local storage medium capable of storing medical image data and the first image data and the second image data extracted by the processor 130 . If necessary, the processor 130 may use data stored in the memory 120 . In addition, the memory 120 of the present invention may store instructions, programs, or applications for the processor 130 to operate.

In addition, the memory 120 of the present invention must remain data even if the power supplied to the 3D modeling apparatus of the institution through image segmentation is cut off, and is a writable non-volatile memory (Writable Rom) to reflect the changes. can be provided. That is, the memory 120 may be provided with either a flash memory, an EPROM, or an EEPROM. In the present invention, it is described that all instruction information is stored in one memory 120 for convenience of explanation, but the present invention is not limited thereto. can be provided

According to an embodiment of the present invention, the processor 130 selects 3 specific organs included in the medical image data based on the medical image data acquired through the above-described communication unit 110 or stored in the memory 120 . dimensional modeling. This will be described in detail later.

On the other hand, in an embodiment of the present invention, the processor 130 acquires the second image data by merging a plurality of first image data generated through three-dimensional modeling of the body organs included in each block. can In this case, the second image data may be data including the entire region of interest for an organ included in the medical image data.

The embodiments described for the apparatus 100 for 3D modeling of an organ through image segmentation are applicable to at least some or all of the three-dimensional modeling methods of an organ through image segmentation, and conversely, 3D organ through image segmentation. At least some or all of the embodiments described for the modeling method may be applied to the embodiments of the 3D modeling apparatus 100 of an organ through image segmentation. In addition, the 3D modeling method of an organ through image segmentation according to the disclosed embodiments is performed by the 3D modeling apparatus 100 of an organ through image segmentation disclosed herein, and the embodiment is not limited, and various forms It can be performed by the electronic device of

Hereinafter, a three-dimensional modeling process of an organ through image segmentation according to the present invention will be described in detail with reference to FIGS. 2 to 7 . All operations of the device 100 described below may be equally performed by the processor 130 that controls the overall operation of the device 100 .

2 is a flowchart schematically illustrating a process of a three-dimensional modeling method of an organ through image segmentation according to an embodiment of the present invention. 3 is an exemplary diagram illustrating a basic screen of a program for modeling a three-dimensional organ shape according to an embodiment of the present invention.

Referring to FIG. 3 , the basic screen may include a layer displaying the overall shape of an object's body and a layer displaying images of each of three planes (axial plane, coronal plane, and sagittal plane).

First, referring to FIG. 2 , the processor 130 receives medical image data for a specific body organ of an object ( S210 ).

Here, the medical image data includes computed tomography (CT) imaging data or magnetic resonance imaging (MRI) imaging data. However, the present invention is not limited thereto.

The CT scan data is data obtained by taking a cross-section of the inside of a human body using a computed tomography (CT) method using a CT scanner.

MRI imaging data is data captured by magnetic resonance imaging (MRI), that is, by using a magnetic field generated by magnetic force, any advanced medical device in the living body or an imaging method made by the machine.

CT scan data or MRI scan data are used as basic data in the process of 3D modeling of body organs. That is, the organ shape is 3D modeled by performing segmentation and rendering operations based on the input CT imaging data or MRI imaging data.

Referring back to FIG. 2 , based on the medical image data received through the collection unit 110 of the device, the processor 130 sets the region of interest for the body organ ( S220 ).

A region of interest (ROI) is an image processing method that sets the range of an organ, organ, tissue, etc. or region to be measured. The region of interest may be set as a two-dimensional region or a three-dimensional region (Volume of Interest; VOI).

In step S220, when a region of interest for a body organ is set, the processor 130 forms and outputs one or more blocks corresponding to the region of interest (S230). In this case, in an embodiment of the present invention, each of the blocks includes a portion of the body organ corresponding to the region of interest.

4 is an exemplary diagram illustrating output by forming one or more blocks including a region of interest according to an embodiment of the present invention.

Step S230 is a step in which the processor 130 forms a block corresponding to each of the one or more ROIs set in step S220 to visually display the ROI.

Referring to FIG. 4 , according to an embodiment of the present invention, a block may be configured in a cube (cuboid) shape. However, the present invention is not limited thereto, and it may be configured in a shape including a curve (eg, a shape in which only some surfaces of a spherical or rectangular parallelepiped are curved, etc.). 4 shows a state in which four blocks 200a, 200b, 200c, and 200d are output by receiving CT scan data of the heart, setting a plurality of ROIs, and forming four blocks 200a, 200b, 200c, and 200d for each ROI. Although the drawings show blocks in the form of four cubes for convenience, the shape and number of blocks are not limited.

For example, assuming that the set ROI is a human heart, one or more cube-shaped blocks corresponding to the heart are output to the user's device. At this time, each block of the cube shape includes the whole or a part of the heart organ. Specifically, each block may be formed to include the left atrium, left ventricle, right atrium, right ventricle, aorta, pulmonary artery, vein, aortic valve, pulmonary valve, three-thousand plate, mitral valve, coronary artery, and the like of the heart, respectively.

According to an embodiment of the present invention, the device may provide the user with an interface capable of adjusting the size and position of the block to include the region of interest. Meanwhile, the number of blocks is not limited, and a plurality of blocks may be formed for each of a plurality of regions of interest. In addition, the size, angle, and position of each of the plurality of blocks formed may be different.

Referring to FIG. 4 , each block may be sized based on axes for three planes (axial plane, coronal plane, and sagittal plane). In addition, as described above, it is possible to move and rotate the blocks if there is an overlapping area between blocks or if necessary.

In another embodiment, blocks may be created and arranged so that there is an area overlapping other blocks. The parts included in most organs are not configured to be completely separated based on one side, but are composed of various shapes and combinations. Accordingly, when a block is formed for each region set as a region of interest, an overlapping region may occur. Since the blocks of the present invention can be overlapped, it is possible to more accurately set the region of interest for each region and form the block. In addition, it is possible to more accurately and precisely grasp a tissue (eg, a heart valve) positioned at the boundary of each region.

In another embodiment, the block may be configured to be rotatable. That is, the block may be configured to rotate about a specific axis according to the manipulation of the system. Through this, a region of interest can be more accurately set for each part of an organ having a complex shape.

In this case, according to an embodiment of the present invention, blocks may be automatically generated for each region of interest using artificial intelligence technology. An input may be received for regions mainly set as a region of interest according to each organ. Alternatively, a type of body organ is identified based on medical image data through machine learning and deep learning technology, an area of interest corresponding to the type of body organ is automatically set, and the interest Blocks for an area can be automatically formed and displayed.

Specifically, when the medical image data is input to the first deep learning-based model, the body organs of the object included in the medical image data are identified, the region of interest is set, and a plurality of blocks are formed according to the region of interest. You will be able to print If the target organ is the heart, the region of interest for the region such as the left atrium, left ventricle, right atrium, right ventricle, aorta, pulmonary artery, vein, aortic valve, pulmonary valve, three-thousand plate, mitral valve, and coronary artery is automatically set and the corresponding Blocks for regions of interest can be automatically formed and displayed.

Meanwhile, in an embodiment of the present invention, when the body organ is a heart organ, the processor 130 may extract location information of a valve of the heart organ and set a region of interest based on the location information of the valve.

Specifically, the processor 130 identifies a tricuspid, a pulmonary valve, a mitral valve, and an aortic valve of a heart organ, and extracts location information of each valve. Then, based on the position information of each valve, four regions of interest are set. In this case, the four regions of interest may be set to correspond to the left atrium, left ventricle, right atrium, and right ventricle, respectively.

Meanwhile, although not clearly shown in the drawings, according to an embodiment of the present invention, in setting the region of interest, the processor 130 places the body organ into a first region corresponding to a blood-glomerular region and a muscle region of the body organ. The second region may be divided into corresponding second regions, and the first region and the second region may be set as regions of interest for the body organs, respectively. In this case, the dividing of the first region and the second region includes identifying the first region corresponding to the blood region when the contrast value of the body organ is greater than or equal to a preset first value based on the medical image data, , when the contrast value is less than a preset second value, the muscle region may be identified and divided into a second region corresponding to the muscle region. In this case, the first value may be set to be higher than the second value. However, the present invention is not limited thereto, and the first value and the second value may be set to the same value.

For example, assuming that the body organ is the heart and the medical image data is CT scan data of the heart, the blood (Lumen) region in the heart is taken with a contrast medium injected for CT scan. . Accordingly, the blood region in the medical image data is relatively brighter than the muscle region, ie, has a higher contrast value. Accordingly, when the contrast value is equal to or greater than the first preset value, it is identified as a blood region of the heart organ, and when the contrast value is equal to or greater than the preset second value, it is identified and divided as a muscle region of the heart organ. In addition, the divided blood region (the first region) and the muscle region (the second region) may be set as regions of interest of the heart organ, respectively. In this case, according to an embodiment of the present invention, the muscle region, which is the second region of the cardiac organ, may be reset to four regions of interest based on the valve position information as described above.

Referring back to FIG. 2 , after step S230 , the device sets a segmentation algorithm for each block ( S240 ).

The segmentation algorithm refers to an algorithm that adjusts a specific variable (eg, Hounsfield un (HU), contrast value, brightness value, etc.) of photographed data based on a set reference value. The segmentation algorithm is used to search for or highlight the boundary of a part of the body included in the block for each ROI.

Meanwhile, in the generating of the first image data according to an embodiment of the present invention, the body organs included in each block may be simultaneously 3D modeled based on a segmentation algorithm set in the block. That is, according to the present invention, a three-dimensional modeling environment optimized for each block is set by setting variables, contrast values, etc. of a segmentation algorithm applied to a part of a body organ included in each block, and then a body organ included in each block. By performing 3D modeling of a part of the body at the same time, accurate and rapid modeling of whole body organs is possible.

Also, according to an embodiment of the present invention, a segmentation algorithm may be set differently and applied according to each block. That is, when a plurality of blocks are formed, the blocks perform a segmentation operation on the region of interest included in the block based on the segmentation algorithm set for each block. Through this, it is possible to apply the segmentation algorithm optimized for each part even for the same organ, so that the 3D organ shape can be modeled more quickly and accurately.

For example, if it is assumed that the organ of the object is a human heart, the shape and shape of the heart are changed according to the systolic and diastolic phases of the heart. Also, the contraction and relaxation of the heart changes the arrangement of blood and the amount of blood flow in the heart. In addition, depending on the movement of the contrast medium administered to the human body so that a specific tissue or blood vessel can be seen clearly during an examination or procedure, the brightness may be different for each part of the heart.

5 is an exemplary diagram of performing three-dimensional modeling on the entire photographed image of the heart, which is a target organ.

Referring to FIG. 5 , since one segmentation algorithm is applied to the entire photographed image, it can be confirmed that the correct segmentation operation is not performed for each part of the heart.

That is, when one algorithm is applied to three-dimensionally model the entire heart, it is difficult to accurately model the entire heart. In order to solve this problem, the prior art extracts a portion of blood within the heart, and divides the heart by region, that is, left atrium, left ventricle, right atrium, right ventricle, aorta, pulmonary artery, vein, aortic valve, pulmonary artery valve, tricuspid valve, mitral valve, coronary artery After separating the back into the back, three-dimensional modeling was performed. In particular, manual and post-processing were required for 3D modeling of difficult-to-identify areas in medical image data. Such a conventional open art has a problem in that the quality of the results is different depending on the individual's proficiency, and above all, it takes a lot of time.

In order to solve this problem, the present invention provides for each part of an organ (for example, in the case of heart, left atrium, left ventricle, right atrium, right ventricle, aorta, pulmonary artery, vein, aortic valve, pulmonary valve, tricuspid valve, mitral valve, coronary artery, etc. ) Each block is formed, and a segmentation algorithm applied to each block is set to perform a segmentation operation on a plurality of parts at the same time. This has the effect of accurately and quickly modeling the three-dimensional shape of the organ.

Meanwhile, in an embodiment of the present invention, in the generating of the first image data, when an overlapping region exists between the blocks, the body part included in the overlapping region is based on a segmentation algorithm set for each of the blocks. It can be characterized by exploring

Specifically, when an overlapping region exists between blocks generated by the device, a target region is searched for by applying a segmentation algorithm set to each block. In addition, it is possible to accurately analyze the parts located in the area where the blocks overlap by using a Boolean operation on the data obtained from each block. That is, by applying a Boolean operator (eg, AND, OR, NOT, XOR, etc.) to the data obtained from each block, it is possible to logically analyze and search for parts located in the overlapping area.

As a specific example, when the target organ is a heart, it is possible to accurately model a valve existing in a region where blocks formed in the atrium and the ventricle overlap. As another specific example, if the boundary of the region of interest consists of muscles of different thickness for each region, a block is formed so that the boundary composed of the corresponding muscle is located in the overlapping region of the block, and an algorithm specialized for each block is applied to the same as the actual organ. can be modeled.

Referring back to FIG. 2 , the apparatus generates respective first image data obtained by three-dimensionally modeling parts included in each block based on an algorithm set in each block ( S250 ).

At this time, in order to generate the first image data, a volume rendering process and a surface rendering process are performed on a part of a body organ included in each block. Since the above-described rendering technique is a known technique, a detailed description thereof will be omitted.

6 is an exemplary diagram illustrating generation of respective first image data obtained by 3-D modeling parts included in each block based on a segmentation algorithm set for each block according to an embodiment of the present invention.

6 illustrates a state in which a segmentation operation is simultaneously performed on each of the four blocks 200a, 200b, 200c, and 200d formed in a plurality of ROIs of a target organ.

For each block, a segmentation algorithm optimized for imaging data for a region of interest included in the block is set. Segmentation algorithms set in blocks may be the same or different.

Each block is segmented based on a set segmentation algorithm. That is, a variable or a reference value to be adjusted according to a set segmentation algorithm may be applied differently. Accordingly, it is possible to perform a 3D organ shape modeling work faster and more accurately by performing segmentation work at the same time by applying a differentiated algorithm according to the site even in the same organ, without the need to separate each area and work separately.

Referring to FIG. 2 , when a plurality of first image data is generated by three-dimensionally modeling parts of body organs included in each block, each of the first image data is merged to form a 3D shape of the entire body organ Generates second image data of (S260).

7 is an exemplary diagram illustrating generation of second image data in a three-dimensional form for the entire cardiac organ by merging respective first image data.

6 and 7 , the processor 140 merges the four first image data generated for each of the four blocks 200a, 200b, 200c, and 200d to form a second 3D shape of the entire cardiac organ. Create video data.

Specifically, the processor 120 generates second image data by merging a plurality of second image data. Specifically, the processor 140 may generate the second image data through image registration.

The image registration refers to a technique for obtaining a cross-sectional shape of a region of interest from among the first image data and moving it to one reference coordinate to overlap it.

According to an embodiment of the present invention, image registration is a feature element matching technique for extracting and matching major feature points of an image or a template-based matching technique for determining a region with the highest similarity by comparing a predetermined region in an image with a specified template (Template) -based registration).

The feature element matching technique may consist of four steps: feature extraction, feature matching, transformation model estimation, and image registration.

In addition, as the feature element matching method, an intensity-based matching method such as CC (Cross-Correlation), MI (Mutual Information) or LSM (Least-Squares Matching) and SIFT (Scale-Invariant Feature Transform) ) and feature-based matching methods such as Speeded Up Robust Features (SURF).

According to an embodiment of the present invention, when a feature-based registration technique is used as the image registration method, the axes of the first image data are aligned. In an embodiment, the axis of the image data refers to x, y, and z axes in a three-dimensional space.

In addition, by extracting and matching a plurality of feature points from a plurality of first image data, the size and position of each image may be adjusted. In an embodiment, the feature point includes a specific point in which a position in a three-dimensional space does not change according to a change in a patient's state (eg, breathing).

In an embodiment, the feature point extraction may be implemented by an artificial intelligence algorithm including machine learning or deep learning.

When the plurality of feature points are extracted, the size and position of each of the first image data are matched based on the distance or location between the plurality of feature points, and then merged to generate the second image data.

Meanwhile, the three-dimensional modeling data of the organ generated according to the embodiment of the present invention described above may be materialized and output according to a three-dimensional printer or a three-dimensional printing technique. The output results in this way can be used for surgical training and surgical simulation.

In the present invention, various embodiments via the processor 130 may be implemented using a machine learning model. As an example, the deep neural network (DNN) of the present invention may include a system or network that constructs one or more layers in one or more computers and performs a determination based on a plurality of data.

The deep neural network may be implemented as a set of layers including a convolutional pooling layer, a locally-connected layer, and a fully-connected layer.

The convolutional pooling layer or local connection layer may be configured to extract features in an image.

The fully connected layer may determine a correlation between features of an image.

As another example, the overall structure of the deep neural network of the present invention may be formed in a form in which a local access layer is connected to a convolutional pooling layer, and a fully connected layer is formed in the local access layer. The deep neural network may include various judgment criteria (ie, parameters), and may add new judgment criteria (ie, parameters) through input image analysis.

The deep neural network according to embodiments of the present invention is a structure called a convolutional neural network suitable for image analysis, and a feature extraction layer that learns by itself the feature with the greatest discriminative power from given image data. ) and a prediction layer that learns a prediction model to obtain the highest prediction performance based on the extracted features may be configured in an integrated structure.

The feature extraction layer spatially integrates a convolution layer that creates a feature map by applying a plurality of filters to each region of the image and a feature map that is invariant to changes in position or rotation. It can be formed in a structure that alternately repeats the pooling layer for extracting . Through this, various level features can be extracted from low-level features such as points, lines, and planes to complex and meaningful high-level features.

The convolutional layer obtains a feature map by taking a nonlinear activation function on the dot product of the filter and the local receptive field for each patch of the input image. In comparison, CNNs are characterized by using filters with sparse connectivity and shared weights. This connection structure reduces the number of parameters to be learned, makes learning through the backpropagation algorithm efficient, and consequently improves prediction performance.

The integration layer (Pooling Layer or Sub-sampling Layer) generates a new feature map by using local information of the feature map obtained from the previous convolutional layer. In general, the newly created feature map by the integration layer is reduced to a smaller size than the original feature map. Representative integration methods include Max Pooling, which selects the maximum value of the corresponding region in the feature map, and the corresponding feature map in the feature map. There is an average pooling method that calculates the average value of a region. In general, the feature map of the integrated layer may be less affected by the position of an arbitrary structure or pattern existing in the input image than the feature map of the previous layer. That is, the integration layer can extract features more robust to regional changes, such as noise or distortion, from the input image or previous feature map, and these features can play an important role in classification performance. Another role of the integration layer is to reflect the features of a wider area as you go up to the upper learning layer in the deep structure. As the feature extraction layers are piled up, the lower layers reflect local features and move up to the upper layers. More and more abstract features can be generated that reflect the features of the entire image.

As such, the features finally extracted through iteration of the convolutional layer and the integration layer are fully connected to the classification model such as multi-layer perception (MLP) or support vector machine (SVM). -connected layer) and can be used for classification model training and prediction.

Also, according to an embodiment of the present invention, learning data for machine learning may be generated based on the U-Net-dhSgement model. Here, the U-Net-dhSgement model is based on an end-to-end fully connected convolutional network (FCN), and contracts an expansive path and a symmetric path. It could be a model that created a U-shaped architecture with skip connections for each level by setting it to .

In addition, the training data for the machine learning model may be composed of medical image data, first image data, and second image data.

Accordingly, the processor 140 may perform the three-dimensional modeling of the organ through image segmentation, as described above with reference to FIGS. 2 and 7 , using the machine learning model learned through the training data.

Various embodiments of the present invention may include a storage medium (eg, memory) readable by a machine (eg, an apparatus 100 for 3D modeling of an organ through image segmentation or a computer). It may be implemented as software including one or more instructions stored in the . For example, the processor (eg, the processor 140 ) of the device may call at least one of the one or more instructions stored from the storage medium and execute it. This enables the device to be operated to perform at least one function according to the at least one instruction called. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory storage medium' is a tangible device and only means that it does not include a signal (eg, electromagnetic wave), and this term means that data is semi-permanently stored in the storage medium. and temporary storage. For example, the 'non-transitory storage medium' may include a buffer in which data is temporarily stored.

According to one embodiment, the method according to various embodiments disclosed in the present specification may be provided as included in a computer program product. Computer program products may be traded between sellers and buyers as merchandise. The computer program product is distributed in the form of a machine-readable storage medium (eg compact disc read only memory (CD-ROM)), or via an application store (eg Play Store™) or on two user devices. It can be distributed (eg downloaded or uploaded) directly or online between devices (eg smartphones). In the case of online distribution, at least a portion of a computer program product (eg, a downloadable app) is stored at least in a machine-readable storage medium such as a memory of a manufacturer's server, a server of an application store, or a relay server. The present invention may be temporarily stored or temporarily created. The embodiments of the present invention have been described above with reference to the accompanying drawings, but those of ordinary skill in the art to which the present invention pertains will appreciate the technical spirit or essential features of the present invention. It will be understood that the invention may be embodied in other specific forms without modification. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

100: 3D modeling device of an organ through image segmentation
110: collection unit
120: memory
130: processor

Claims (10)

A method performed by an apparatus comprising:
receiving at least one or more medical image data for a specific body organ of a subject;
setting a region of interest for the body organ based on the medical image data;
forming one or more blocks corresponding to the region of interest, each block including a portion of the body organ corresponding to the region of interest;
setting a segmentation algorithm for each of the blocks;
generating respective first image data obtained by three-dimensionally modeling parts included in each of the blocks based on an algorithm set in each of the blocks; and
Comprising the step of merging each of the first image data to generate a three-dimensional form of second image data for the entire body organ,
A three-dimensional modeling method of organs through image segmentation. The method of claim 1,
The step of setting the region of interest comprises:
Based on the deep learning-based first model, by recognizing the type of the body organ, automatically generating a region of interest corresponding to the type of the body organ,
A three-dimensional modeling method of organs through image segmentation. The method of claim 1,
The merging is performed through image registration,
The image registration is performed through at least one of a feature element matching technique and a template-based matching technique,
3D modeling method of organs through image segmentation The method of claim 1,
The step of generating the first image data comprises:
When an overlapping region exists between the blocks, the body part included in the overlapping region is identified based on a segmentation algorithm set in each of the blocks.
3D modeling method of organs through image segmentation The method of claim 1,
The step of setting the region of interest comprises:
dividing a first region corresponding to a blood region of the body organ and a second region corresponding to a muscle region of the body organ;
The method further comprising the step of setting the first region and the second region as regions of interest for the body organs,
The dividing step is
Based on the medical image data, when the contrast value of the body organ is greater than or equal to a preset first value, it is identified as a first region corresponding to the blood region, and when the contrast value is less than a preset second value, it is applied to the muscle region. which is to be identified and divided into a corresponding second region,
A three-dimensional modeling method of organs through image segmentation. a collection unit for acquiring at least one or more medical image data; and
processor; including;
The processor is
Based on the medical image data, a region of interest for the body organ is set, one or more blocks corresponding to the region of interest are formed, a segmentation algorithm is set for each of the blocks, and an algorithm set for each of the blocks is set. Based on the three-dimensional modeling of the parts included in each block, each of the first image data is generated, and the respective first image data is merged to form the second image data in a three-dimensional form for the entire body. create, but
Each of the blocks comprises a portion of the body organ corresponding to the region of interest,
A 3D modeling device for organs through image segmentation. 7. The method of claim 6,
When an overlapping region exists between the blocks, the body part included in the overlapping region is identified based on a segmentation algorithm set for each of the blocks.
A 3D modeling device for organs through image segmentation. 7. The method of claim 6,
Setting the region of interest is
The processor recognizes the type of the body organ based on the first model based on deep learning based on the medical image data, and automatically generates a region of interest corresponding to the type of the body organ.
A 3D modeling device for organs through image segmentation. 7. The method of claim 6,
The image registration is performed through at least one of a feature element matching technique and a template-based matching technique,
A 3D modeling device for organs through image segmentation. A computer program stored in a computer-readable recording medium to perform a three-dimensional modeling method of an organ through image segmentation of any one of claims 1 to 5 in combination with a computer.

Images