KR20220085076A - A method of creating a shape for reconstruct the defectvie area of wound using artificial intelligence - Google Patents

Abstract

The present specification discloses a method of generating a cover of the affected area in a three-dimensional curved shape using artificial intelligence. As a method of generating a cover of a three-dimensional curved affected part using artificial intelligence according to the present specification, after removing the points inside the affected part boundary from the point cloud of the affected part shape, the curvature equation is calculated through polynomial regression analysis, and this After creating the affected area as a point cloud, the shape of the affected area can be created through the mesh process at the boundary point.

Description

A method of creating a shape to reconstruct a defect in the affected area using artificial intelligence

The present invention relates to a method for generating a cover for an affected area using artificial intelligence, and more particularly, to a method for generating a shape for reconstructing a defect in an affected area using artificial intelligence.

The content described in this section merely provides background information for the embodiments described herein and does not necessarily constitute prior art.

3D bioprinting, an application and development field of 3D printing technology, is based on 3D printing technology, and is based on an extracellular matrix (ECM) such as collagen or bio-ink (bio-ink) that mimics it. ink) is combined with cells and other biomaterials to create a desired shape. Currently, various methods for 3D bioprinting are being developed according to the desired purpose and biological environment, and various bio-inks are also being studied.

Such 3D bioprinting constructs ECM or bio-ink in a desired shape, and culturing cells necessary for ECM or bio-ink to produce a living organ or tissue having the same function as in reality. One of the most important issues in 3D bioprinting is to ensure that cells and biomaterials, which are materials, can be safely stored and used as much as possible so that cells can continue to function without dying.

One of the important factors influencing the quality of prints in 3D bioprinting is how well the print matches the affected area. For this, the process of accurately photographing the three-dimensional shape of the affected area and 3D modeling it should be treated as an important process.

Republic of Korea Patent Publication 　10-1828345 (2018.02.06)

An object of the present specification is to provide a method for generating a cover of a three-dimensional curved affected area using artificial intelligence.

The present specification is not limited to the above-mentioned problems, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

The 3D data generation method for the affected area cover according to the present specification for solving the above-mentioned problems is a method of generating a cover of the affected area using 3D data for the affected area, the processor (a) 3D consisting of a point cloud for the deployed affected area storing data; (b) removing the point cloud located inside based on the boundary of the affected part among the point clouds constituting the 3D data; (c) dividing the 3D data according to a preset interval based on a first axis among three axes serving as a reference of the three-dimensional coordinates; (d) calculating a curvature equation through regression analysis for the 3D data divided in step (c) (hereinafter, 'slicing 3D data'); (e) calculating the coordinate values for the third axis (hereinafter, 'cover point cloud') by substituting the coordinate values for the first and second axes of the point cloud included in the slicing 3D data into the curve equation; And (f) removing the point outside the boundary of the affected part of the cover point cloud and combining with the point cloud for the developed affected part; may include.

According to an embodiment of the present specification, the step (a) includes: (a-1) calculating inclination values for the first axis and the second axis from the stored 3D data; (a-2) calculating a vector sum of the calculated gradient values; and (a-3) rotating the stored 3D data in the second axis direction by the inclination that the vector sum makes with the second axis.

According to an embodiment of the present specification, the step (c) may be a step of dividing the 3D data according to 1/n intervals (n is a natural number) of the average point interval of the point clouds constituting the 3D data.

According to an embodiment of the present specification, the curve equation of the affected part boundary may be in the form of a quadratic function.

According to an embodiment of the present specification, (g) calculating the maximum value with respect to the third axis boundary points of the point cloud constituting the 3D data of the developed affected part; (h) substituting all the values for the third axis of the boundary points with the maximum value; and (i) filling the value of the third axis of the point cloud with the maximum value up to a point extended by a preset offset in the first and second axis directions of the boundary points, and starting from the point after the offset, the second value of the point cloud The method may further include a step of filling the values for the 3 axis with 0.

The 3D data generating method for the cover 3D data on the affected area according to the present specification can be implemented in the form of a computer program recorded on a computer-readable recording medium written to perform each step of the method for generating cover 3D data.

Other specific details of the invention are included in the detailed description and drawings.

According to the present specification, it is possible to output a 3D model for covering the affected area that can fill the defective area of the affected area. Through this, further infection and relief of the affected area is possible.

Effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is an overall flow diagram of creating a 3D lesion cover according to the present disclosure.
Figure 2 is a more detailed flow chart of the production process of the 3D wound cover according to the present specification.
3 is an artificial intelligence learning method for recognizing an affected area boundary according to an embodiment of the present specification.
4 is an artificial intelligence learning method for recognizing an affected area boundary according to the present specification.
5 is a reference diagram of a technique that can be used during learning for recognizing the border of the affected area.
6 is a reference diagram of post-processing in the artificial intelligence learning method according to the present specification.
7 is a flowchart of a method for recognizing an affected area boundary according to another embodiment of the present specification.
8 is a reference diagram related to a method for recognizing an affected area boundary according to the present specification.
9 is a reference diagram of a method for recognizing an affected area boundary according to another embodiment of the present specification.
10 is a reference view of the 3D modeled lesion after scanning the lesion.
11 is a 3D data development method of the affected part according to an embodiment of the present specification.
12 is a reference diagram for various orders of a curve equation.
13 is a reference diagram for the development of the affected area.
14 is a 3D data development method of the affected part according to an embodiment of the present specification.
15 is a flowchart of a 3D data generating method for the affected part cover according to an embodiment of the present specification.
16 is an exemplary view of 3D data of the affected area cover.
17 is a flowchart of a method for generating 3D data for an affected area cover according to another embodiment of the present specification.

Advantages and features of the invention disclosed herein, and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present specification is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only the present embodiments allow the disclosure of the present specification to be complete, and those of ordinary skill in the art to which this specification belongs. It is provided to fully inform those skilled in the art (hereinafter, 'those skilled in the art') the scope of the present specification, and the scope of the present specification is only defined by the scope of the claims.

The terminology used herein is for the purpose of describing the embodiments and is not intended to limit the scope of the present specification. In this specification, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, “comprises” and/or “comprising” does not exclude the presence or addition of one or more other components in addition to the stated components.

Like reference numerals refer to like elements throughout, and "and/or" includes each and every combination of one or more of the recited elements. Although "first", "second", etc. are used to describe various elements, these elements are not limited by these terms, of course. These terms are only used to distinguish one component from another. Accordingly, it goes without saying that the first component mentioned below may be the second component within the spirit of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used herein will have the meaning commonly understood by those of ordinary skill in the art to which this specification belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless specifically defined explicitly.

Spatially relative terms "below", "beneath", "lower", "above", "upper", etc. It can be used to easily describe the correlation between a component and other components. Spatially relative terms should be understood as terms including different directions of components during use or operation in addition to the directions shown in the drawings. For example, when a component shown in a drawing is turned over, a component described as "beneath" or "beneath" of another component may be placed "above" of the other component. can Accordingly, the exemplary term “below” may include both directions below and above. Components may also be oriented in other orientations, and thus spatially relative terms may be interpreted according to orientation. Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is an overall flow diagram of creating a 3D lesion cover according to the present disclosure.

Referring to FIG. 1 , the patient's affected area is photographed in the hospital, and the photographed data is transmitted to the 3D affected area cover producer through the server. The producer uses 3D data taken of the affected area to produce a 3D cover for the affected area.

Figure 2 is a more detailed flow chart of the production process of the 3D wound cover according to the present specification.

Referring to FIG. 2 , the affected area is first photographed. Since the photographing is for the purpose of generating a 3D model, the affected area may be photographed using a device capable of generating 3D data, such as a 3D scanner or an RGB-D camera.

Next, a process of recognizing the boundary of the affected area within the photographed image is required. In diabetic patients, necrosis may progress from the skin surface to the inside. In order to produce a 3D affected area cover that fills the affected area with diabetic foot necrosis, it is necessary to accurately recognize the boundary of the affected area.

Next, the captured image is converted into 3D data. In this case, the converted 3D data may also include data on the boundary of the previously identified affected area.

Next, a 3D model is created only for the affected area. The 3D model of the affected part may consist of point cloud data expressed as three-dimensional coordinates for each point.

Next, using the 3D model of the affected area, it is possible to generate a 3D model for the cover of the affected area. Since the 3D model for the cover of the affected area is a shape that fills the affected area, the 3D model for the affected area and the 3D model for the cover of the affected area have a shape corresponding to each other.

Next, the 3D bio-printer uses the 3D model for the cover for the affected area to output the cover for the affected area as a bio-material. The lesion cover is attached to the lesion to help fill the necrotic area with new tissue.

On the other hand, in this specification, the detailed description of the present invention will proceed with diabetic foot necrosis as an example. However, the invention described herein is not a technology applied only to diabetic foot necrosis, and the scope of the present invention is not limited by the examples described herein. Hereinafter, each step will be described in more detail.

[Recognition of the border of the affected area]

In the image obtained by photographing the affected area, the affected area and parts other than the affected area are photographed together. At this time, in order to accurately distinguish the affected area from the non-affected area, it is necessary to recognize the boundary of the affected area. As a method for recognizing the border of the affected area, there may be a method in which a person sets the border of the affected area, a method of automatically recognizing the border through artificial intelligence learning, and a method of calculating the border using 3D data. Among them, the method of setting the boundary of the affected part by a person is a method in which an expert such as a doctor inputs boundary information using equipment such as a mouse and a touch pen. Therefore, I will explain in more detail the remaining methods except for the method of setting the border of the affected area by a person.

<Recognition of affected area through artificial intelligence learning>

3 is an artificial intelligence learning method for recognizing an affected area boundary according to an embodiment of the present specification.

Referring to FIG. 3 , a schematic flow of an artificial intelligence learning method according to the present specification can be confirmed. Prepare images for learning of various patient's affected areas taken using RGB-D cameras. An artificial neural network such as deep learning is trained using an image for learning and learning, and the learned artificial neural network outputs the result of recognizing the boundary of the affected part first. In this case, the boundary of the primarily recognized affected area may include an error. These errors are corrected by an expert such as a doctor, and reinforcement learning can be performed by using the data recognized as the final boundary again as a learning image.

In such supervised learning or supervised reinforcement learning, the performance of an artificial neural network can be improved as there is more data for learning. However, unlike general object recognition that can easily obtain tens of millions of image data, it is very difficult to secure a large amount of image data (RGB-D image data) for image data of the affected area. For example, in the case of diabetic foot necrosis image data, the condition of the affected area is very different for each patient, and even the largest open data set is only about 2,000 to 4,000. Therefore, the artificial intelligence learning method according to the present specification suggests a method in which more effective learning can be achieved using a small amount of learning images.

4 is an artificial intelligence learning method for recognizing an affected area boundary according to the present specification.

Referring to Figure 4, it is possible to store a plurality of image data of the affected area first in step S10. The affected part image data may be 2D image data, or 3D image data.

In the next step S11, each affected part image data may be pre-processed (pre-processing). As an example of the pre-processing, measurement information separation or data augmentation techniques may be included.

Separation of measurement information means separating the information necessary to determine the size of the affected area from the affected area image data. In the case of photographing the affected area in a hospital, there are many cases where a ruler or a marker is photographed together with the affected area to know the size of the affected area. Such information helps to determine the size of the lesion in the image data, but does not help much in determining the boundaries of the lesion. Therefore, only the area remaining after cutting the area related to the measurement tool from the image data of the affected area can be used as image data for training. At this time, the cut measuring tool area may be separately stored as measurement information for determining the size of the affected part (eg, if the number of pixels corresponding to 1 cm in the photo is 1000, record as 0.001).

Data augmentation refers to a method of increasing limited data to training data. Color randomization, Adjust image intensity value, enhanced contrast using histogram equalization, contrast-limited adaptive histogram equalization, Blurring (Gaussian/Median), Affine transformation, Brightness randomization, etc. may be used as data augmentation techniques.

In the next step S12, the artificial neural network can be trained using the training image data. Since the artificial intelligence learning method according to the present specification is for recognizing an affected part boundary, an artificial neural network may be configured based on a Faster R-CNN network specialized for classification. However, going beyond the level of displaying a bounding box around the affected area, RoI Align, Mask R-CNN, Feature Pyramid Network, TTA & Model ensemble techniques can be added/changed to accurately recognize the complex area boundary. have.

5 is a reference diagram of a technique that can be used during learning for recognizing the border of the affected area.

Referring to Figure 5 (a), the RoI pooling method of the existing Faster R-CNN is a problem that the area of the object recognized while discarding the decimal point on the feature map that is reduced in size through the CNN network and calculating it as an integer is misaligned. have. The artificial intelligence learning method according to the present specification can recognize an accurate object region even on a feature map reduced to a decimal unit by using bilinear interpolation for precise segmentation.

Referring to Figure 5 (b), while utilizing Faster R-CNN with good object classification and detection performance, a Mask R-CNN structure with a separate Mask network is added to be used for segmentation of the detailed edges of the affected area. can The AI learning method according to the present specification performs classification and maintains Faster R-CNN that generates a bounding box around an object as it is, and can configure a mask network branch separately.

Referring to (c) of Figure 5, it is difficult to predict what size the affected area will exist in one image, and in some cases, the entire picture is close-up, and in some cases, the affected area appears very small by shooting from a distance. do. For this reason, the CNN network structure in which the size of the resulting feature map becomes smaller than that of the original photo is likely to give difficulty in recognizing small affected areas. Therefore, the artificial intelligence learning method according to the present specification can utilize the feature pyramid network technique for extracting features by up-sampling feature maps of various sizes.

Referring to (d) of FIG. 5 , when prediction is performed on a new picture of an affected area using the previously trained model, different results may appear even for the same affected area depending on the angle, direction, size, etc. of the newly taken picture. have. In order to prevent this, the artificial intelligence learning method according to the present specification performs prediction by inputting each image into an artificial neural network by variously changing the angle, direction, and size of the image before inputting the learning image to the model. All results can be integrated through Soft NMS to obtain the final prediction result.

Referring to FIG. 5(e), various variants exist also in Faster R-CNN. The artificial intelligence learning method according to the present specification performs separate learning using three representative variants (Cascade R-CNN, Deformable Convolution v2, Prime Sample Attention), and then the final prediction result is derived through the weighted boxes fusion technique. can

Meanwhile, all of the learning methods described with reference to FIG. 5 are limited to examples, and the artificial intelligence learning method according to the present specification is not limited to the examples.

Referring back to FIG. 4 , post-processing may be performed on the result output by the artificial neural network in step S13. The post-processing may be an operation of connecting the boundary of the affected part recognized by the artificial neural network with a smoother curve.

6 is a reference diagram of post-processing in the artificial intelligence learning method according to the present specification.

Referring to FIG. 6 , a contour may be recognized from the segmentation recognition result (a). And using the Douglas-Peucker algorithm, it is possible to extract the minimum number of vertices that can form a contour (b). All extracted vertices can be connected to a smooth curve without sharp parts by connecting them with the Catmull-Rom spline algorithm (c). In this case, a person may directly move the vertices.

In addition, most of the affected part photo datasets contain only the intact affected part. However, during actual surgery, an operation called debridement is frequently performed to remove damaged or infected tissue, which leads to bleeding. Such bleeding is easy to produce inaccurate results in predicting the rim of the affected area with a machine learning algorithm, and the following post-processing can be performed to correct this. Bleeding area recognition can measure the bleeding area by giving different weights to the CIELAB color space and the CMYK color space and combining them. (A(x,y): A channel of CIELAB, M(x,y): M channel of CMYK)

Edge detection is performed on the bleeding area to extract the boundary, and the point where the boundary of the affected area and the boundary of the bleeding area diverge is found. The gradient for the affected area can be calculated, and the coefficient matrix of the polynomial regression curvature equation can be obtained according to the gradient around the point. By substituting the curvature equation, the affected area boundary for the corresponding section can be modified.

The boundary of the affected area can be recognized in the image taken through the artificial neural network trained through the artificial intelligence learning method according to the present specification described above.

<Recognition of the border of the affected area using the center point of the affected area>

The method of recognizing the border of the affected area through artificial intelligence described above can be said to be a method using only RGB data in the image of the affected area. However, since the image data taken from the affected area may be RGB-D data, a method for recognizing the boundary of the affected area will be described using the depth information of the affected area. The method to be described below is a method using the characteristic that the inclination of the skin and the affected area changes rapidly at the boundary of the affected area.

7 is a flowchart of a method for recognizing an affected area boundary according to another embodiment of the present specification.

8 is a reference diagram related to a method for recognizing an affected area boundary according to the present specification.

Referring to FIG. 7 , it is possible to store 3D image data captured by the affected part in step S20 first. Since the 3D image data has been described above, a repetitive description will be omitted.

In the next step S21, a point located inside the affected part in the image data may be set as a reference point. The reference point is a coordinate used for calculations to determine the boundary of the affected area, and the closer to the center of the affected area, the more preferable the reference point.

For example, in a situation where the boundary of the affected part is not yet known, the coordinates input by the user may be set as the reference point. The user can check the 3D image data through the display and rotate it so that the border of the affected area is visible the most. In addition, the user can select an arbitrary region within the boundary of the affected part, preferably by clicking the central region with a mouse.

In the next step S22, an average normal of a triangle formed by point clouds included in a preset radius around the reference point O may be calculated (refer to (a) of FIG. 8). 3D image data may be expressed as data about points on three-dimensional coordinates. Points on these three-dimensional coordinates will be referred to as a point cloud, and triangles made of three points will be referred to as '3D triangular surfaces' in this specification. The 3D triangular plane corresponds to a plane of the smallest unit constituting a three-dimensional shape. The preset radius is preferably set large enough to include both the affected area and the normal skin surface beyond the affected area boundary. Calculating the normal vector of each 3D triangular plane and calculating the average normal vector using the calculated normal vector are known to those skilled in the art, and thus detailed descriptions thereof will be omitted.

The calculated average normal may be an axis perpendicular to the entire affected area. That is, it can be the central axis that can best represent the entire affected area. Accordingly, in the next step S23, a plane including the average normal and a first axis perpendicular to the normal may be set. Preferably, the 3D image data may be rotated so that the average normal coincides with the Z-axis. In this case, the normal line may be the Z axis, the first axis may be the X axis, and the plane may be the XZ plane (refer to FIG. 8(b) ).

In the next step S24, an intersection point of the (XZ) plane and each 3D triangular plane may be calculated, and the slope of each 3D triangular plane may be calculated using the intersection point (refer to FIG. 8(c) ).

In the next step S25, the amount of inclination change of the adjacent 3D triangular plane in a direction away from the reference point may be calculated.

In the next step S26, it is possible to select a candidate expected to be the borderline of the affected area through two criteria. First, 3D triangular planes related to inclination changes that are maintained less than or equal to a preset reference inclination change value are selected from among the calculated inclination changes. That is, 3D triangular surfaces in which the change in inclination is not abrupt is selected. Then, it is determined whether a length of connecting 3D triangular faces connected to each other among the selected 3D triangular faces is maintained over a preset reference length value. That is, it is whether or not the predetermined length is maintained without a change in the slope. When the length is longer than a certain length, the starting point of each length is stored as a candidate list.

An uneven surface will be formed according to necrosis inside the affected area, and the slope will change rapidly at the boundary of the affected area. In addition, the area outside the boundary of the affected area, that is, normal skin, will have little change in slope. Therefore, if the state in which there is not much gradient change is the beginning of a place where a certain length is maintained, it can be a starting point leading to normal skin at the boundary of the affected area. Therefore, these starting points are stored as a candidate list.

In the next step S27, the (XZ) plane set in the step S23 may be rotated at a preset angle using the average normal (Z axis) as a rotation axis. In addition, the candidate list may be accumulated while repeating steps S24 to S27 for a new 3D triangular plane that meets the rotated (XZ) plane.

According to an example, the (XZ) plane may be rotated by 0.1 degree clockwise. Therefore, assuming that the starting point is 0 degrees, the candidate list can be accumulated 0 to 3599 times. When the initial starting point of 0 degrees is reached while repeating the above process ('YES' in step S28), calculations for all 3D triangular planes included in the 3D image data may be completed.

In the next step S29, among the start points included in the list after the accumulation, adjacent start points within a preset reference distance (eg, 0.5 mm) may be connected to each other to form a closed curve. The closed curve may directly correspond to the boundary of the affected area.

If a plurality of closed curves are formed in step S29, a closed curve including the reference point therein may be selected from among the plurality of closed curves. Also, when there are a plurality of closed curves including the reference point therein, the largest closed curve may be selected.

According to an example of the present specification, a closed curve may be corrected and supplemented by using a location where a surface color (RGB value) in 3D image data changes by more than a preset reference color value as additional information. It takes advantage of the fact that the color of the affected area and the normal skin are different from each other.

Thereafter, the closed curve may be smoothed with a moving average, and a 3D triangular information list within the closed curve may be managed as an array (WL). That is, only the 3D shape of the affected part can be extracted.

<Recognition of the boundary of the affected area using the temporary boundary line of the affected area>

On the other hand, in the method for recognizing the border of the affected part described with reference to FIGS. 7 and 8, calculation is made for all 3D triangular planes included in the 3D image data. This process may be a factor that lowers the processing speed due to the burden of computational load. If the approximate boundary of the affected area is known and only the exact boundary can be recognized at the approximate boundary of the affected area, the speed can be improved by reducing the amount of computation.

9 is a reference diagram of a method for recognizing an affected area boundary according to another embodiment of the present specification.

Referring to Fig. 9 (a), it can be seen that a temporary boundary line is set at the boundary of the affected area. The temporary boundary line may be set through the method of recognizing the affected area boundary through artificial intelligence learning described above with reference to FIG. 4 . In addition, the user may designate a temporary boundary line of the polygon by clicking the border of the affected part with a mouse or by touching the screen. In this case, it is possible to receive the temporary boundary line data including the 3D image data and the rim of the affected area taken in step S20 of Figure 7 together.

Referring to FIG. 9B , a first closed curve C1 extended by a preset length and a second closed curve C2 reduced by a preset length may be generated based on the temporary boundary line.

In addition, in step S21 of FIG. 7 , an intersection point at which two virtual lines equally dividing the inner region of the second closed curve into four may be set as a reference point. Thereafter, steps S22 and S23 are the same.

On the other hand, in step S24 of FIG. 7 , the intersection point with the plane may be calculated only for 3D triangular surfaces located between the first closed curve and the second closed curve. That is, not all 3D triangular planes included in the 3D image data, calculation is performed only on the area expected to be the boundary of the affected part. Through this, the amount of computation can be reduced and the computation speed can be improved. Thereafter, steps S25 to S29 are the same.

[Development of affected area]

10 is a reference view of the 3D modeled lesion after scanning the lesion.

Referring to Figure 10, it can be confirmed the appearance of the 3D modeled affected area. As in the example shown in the figure, since the affected area is formed widely from the lower surface of the foot to the side, it is curved in the shape of a curved surface. When 3D modeling data is generated for such an affected part, the generated 3D model will also take the shape of a curved surface as a whole. When a three-dimensional curved shape is directly input and output in a bioprinter, the structure may collapse during printing. Therefore, it is necessary to expand the curved shape in the horizontal direction so that the bioprinter can print it.

<Development of affected area using regression analysis>

11 is a 3D data development method of the affected part according to an embodiment of the present specification.

Referring to FIG. 11 , it is possible to store 3D data consisting of a point cloud for the affected part in step S30 first. The 3D data of the point cloud for the affected area may be the 3D data of the affected area extracted after the execution of the method for recognizing the boundary of the affected area described above with reference to FIGS. 7 to 9 .

According to an embodiment of the present specification, in step S30, the 3D data may be rotated based on any one of three axes serving as a reference of the three-dimensional coordinates for convenience of subsequent calculations. In this case, the size of the calculated value with respect to the point coordinate values of the rotated 3D data may be reduced. More specifically, the stored 3D data calculates the inclination values for the first and second axes, calculates the vector sum of the calculated inclination values, and calculates the stored 3D data as much as the inclination between the vector sum and the second axis to the second axis. It can be rotated axially. In this specification, the first axis will be described as 'Y-axis', the second axis as 'X-axis', and the third axis as 'Z-axis'. However, the above examples are for convenience of understanding, and the present invention is not limited thereto.

In the next step S31, the 3D data may be divided according to a preset interval based on the Y-axis. The divided 3D data will be referred to as 'slicing 3D data' hereinafter. The smaller the slicing interval, the more detailed the development is possible, but the amount of computation increases. The larger the slicing interval, the less the amount of computation, but the development is not fine. For example, the slicing interval may be 1/n interval (n is a natural number) of an average point interval of a point cloud constituting 3D data.

In the next step S32, any one of the slicing 3D data may be projected onto the XZ plane. An example in which the slicing 3D data is projected onto the XZ plane can be seen at the bottom of FIG. 11 .

In the next step S33, the curve equation of the boundary of the affected part can be calculated through regression analysis (polynomial regression) for the outer boundary points of the projected slicing 3D data. The curve of the border of the affected area may be in the form of a quadratic function.

12 is a reference diagram for various orders of a curve equation.

Referring to FIG. 12 , it can be seen that a first-order function, a second-order function, a third-order function, and a tenth-order function are shown for the point cloud. It can be seen that the first-order function is underfitting, and the higher-order function is overfitting. Considering the characteristics of the affected area in the skin tissue of the human body, the quadratic function can best express the shape of the affected area. The regression analysis can calculate the curve equation of the boundary of the affected area in the form of a quadratic function using the following equation.

[Equation]

Since the regression analysis is a formula and method known to those skilled in the art, a detailed description thereof will be omitted. However, the affected area boundary curve equation calculated through regression analysis becomes a quadratic function-shaped curve most closely expressed with respect to the affected area boundary of the slicing 3D data.

In the next step S34, it is possible to calculate the converted slicing 3D data for the second axis and the third axis by using the following equation including the curve equation of the affected part boundary.

[Equation]

z' _i = z _i - z_curve _i

z_curve _i : Same as y _i of the curve equation of the border of the affected area

That is, the difference between the transformed 3D slicing data {P(x, y, z)} in the transformed 3D slicing data {P'(x', y, z')} is in the X-axis direction by the distance between each point will unfold with At this time, the Z values of the point cloud constituting the converted 3D slicing data only have a difference value corresponding to the depth of the affected part, and the difference value according to the curved shape of the affected part is not reflected.

On the other hand, steps S32 to S34 are repeatedly executed until applied to all slicing 3D data. After all the slicing 3D data is transformed, if the transformed slicing 3D data is re-integrated, the shape of the affected part developed in the X-axis direction can be obtained.

13 is a reference diagram for the development of the affected area.

Referring to FIG. 13 , it is a result of developing point clouds in which the boundary of the affected part is projected on the XZ plane. It can be seen from the graph that the shape of the small and irregular affected part is maintained while the large curved surface is removed, so that the z-axis height is reduced and the x-axis length is increased. If the bioprinter prints the shape below using a flexible biomaterial and attaches it to the affected area, it can be modeled to cover the entire affected area while covering the minute curves of the affected area.

<Development of affected area using low-frequency filter>

14 is a 3D data development method of the affected part according to an embodiment of the present specification.

Referring to FIG. 14 , it is possible to store 3D data consisting of a point cloud for the affected part in step S40 first. The 3D data of the point cloud for the affected area may be the 3D data of the affected area extracted after the execution of the method for recognizing the boundary of the affected area described above with reference to FIGS. 7 to 9 .

According to an embodiment of the present specification, in step S40, the 3D data may be rotated based on any one of three axes serving as a reference of the three-dimensional coordinates for convenience of subsequent calculations. In this case, the size of the calculated value with respect to the point coordinate values of the rotated 3D data may be reduced. More specifically, the stored 3D data calculates the inclination values for the first and second axes, calculates the vector sum of the calculated inclination values, and calculates the stored 3D data as much as the inclination between the vector sum and the second axis to the second axis. It can be rotated axially. In this specification, the first axis will be described as 'Y-axis', the second axis as 'X-axis', and the third axis as 'Z-axis'. However, the above examples are for convenience of understanding, and the present invention is not limited thereto.

In the next step S41, the 3D data may be divided according to a preset sampling frequency fs based on the Y-axis. The divided 3D data will be referred to as 'slicing 3D data' hereinafter. The higher the sampling frequency fs, the more detailed development is possible, but the amount of computation increases, and the lower the sampling frequency fs, the less the amount of computation decreases, but the development is not detailed. For example, the sampling frequency fs may be 1/n interval (n is a natural number) of the minimum interval of the point cloud. An example of the 3D data sliced by the sampling frequency can be confirmed at the bottom of FIG. 14 .

In the next step S42, a frequency set higher by a preset ratio (a%) than a curved frequency of any one of the slicing 3D data among the slicing 3D data may be set as the cutoff frequency fc. The cutoff frequency fc may be expressed as a differential equation after inverse Laplace transform for the convenience of subsequent calculations as follows.

[Equation]

는 아래와 같이 표현될 수 있다.A curve point may be calculated by passing the slicing 3D data through a low-frequency filter having the cut-off frequency fc in the next step S43. In step S43, padding data may be further added back and forth at a point where the value for the Y-axis is 0 among the slicing 3D data, and the low-frequency filter may be passed through. Curve points calculated in this way

can be expressed as follows.

[Equation]

In the next step S44, the converted slicing 3D data for the X and Y axes is calculated using the formula below including the curve points.

It is possible to calculate the slicing 3D data transformed with respect to the second axis and the third axis by using the following equation including the curve equation of the affected part boundary.

[Equation]

z' _i = z _i - z_curve _i

z_curve _i : equal to y _i of the curve point

That is, the difference between the transformed 3D slicing data {P'(x', y, z')} from the transformed 3D slicing data {P(x, y, z_curve _i )} is that develops in the direction At this time, the Z values of the point cloud constituting the converted 3D slicing data only have a difference value corresponding to the depth of the affected part, and the difference value according to the curved shape of the affected part is not reflected.

On the other hand, steps S42 to S44 are repeatedly executed until applied to all slicing 3D data. After all the slicing 3D data is transformed, if the transformed slicing 3D data is re-integrated, the shape of the affected part developed in the X-axis direction can be obtained. Since the above content has been described with reference to FIG. 13, a repetitive description will be omitted.

[Affected area cover]

According to the 3D data development method of the affected part described with reference to FIG. 11 or FIG. 14, it is possible to secure 3D data that can be output with a bioprinter. However, the 3D data secured in this process is data on the affected part, and what is needed in the present invention is data on the cover that can cover the affected part.

<How to create 3D data for the affected area cover>

15 is a flowchart of a 3D data generating method for the affected part cover according to an embodiment of the present specification.

Referring to FIG. 15 , it is possible to store 3D data consisting of a point cloud for the affected part developed in step S50 first. The 3D data of the developed affected area can be obtained according to the 3D data development method of the affected area described with reference to FIGS. 11 or 14 .

According to an embodiment of the present specification, in step S50, the 3D data may be rotated based on any one of three axes serving as a reference of the three-dimensional coordinates for convenience of subsequent calculations. In this case, the size of the calculated value with respect to the point coordinate values of the rotated 3D data may be reduced. More specifically, the stored 3D data calculates the inclination values for the first and second axes, calculates the vector sum of the calculated inclination values, and calculates the stored 3D data as much as the inclination between the vector sum and the second axis to the second axis. It can be rotated axially. In this specification, the first axis will be described as 'Y-axis', the second axis as 'X-axis', and the third axis as 'Z-axis'. However, the above examples are for convenience of understanding, and the present invention is not limited thereto.

In the next step S51, it is possible to remove the point cloud located inside based on the boundary of the affected part among the point clouds constituting the 3D data. In this case, the boundary may mean an edge of the shape of the affected part, that is, not only the boundary with normal skin, but also the outermost point forming the affected part.

In the next step S52, the 3D data may be divided according to a preset interval based on the Y-axis. For example, the sampling frequency fs may be a 1/n interval (n is a natural number) of the minimum interval of the point cloud. An example of the 3D data sliced by the sampling frequency can be confirmed at the bottom of FIG. 15 .

In the next step S53, a curvature equation may be calculated through regression analysis on the slicing 3D data. The process of obtaining the curvature equation through the regression analysis has been described in the affected area development method described above with reference to FIG. 11, and since it is a process known to those skilled in the art, repeated descriptions will be omitted.

In the next step S54, the (X, Y) coordinate values of the point cloud included in the slicing 3D data may be substituted into the curve equation to calculate the (Z) coordinate values. The calculated coordinate values (X, Y, Z) correspond to the 'cover point cloud'.

In the next step S55, it is possible to remove a point outside the boundary of the affected part of the cover point cloud and combine it with the point cloud for the developed affected part.

16 is an exemplary view of the 3D data of the lesion cover.

Referring to Figure 16, it can be confirmed that the cover having a shape that can cover the affected area. It can be seen that the upper part of the cover has a flat shape similar to the skin surface, and the lower part has an uneven shape corresponding to the shape of the affected part.

<How to create 3D data on the affected area covering the affected area cover output>

On the other hand, the cover of the affected area shown in Fig. 16 is formed to reflect the presence of normal skin at the edge of the affected area. Therefore, as can be seen in FIG. 16, the outermost boundary surface of the affected part may have a vertically cut shape. If the vertically cut shape is printed on the bioprinter as it is, the outer area of the cover may collapse.

17 is a flowchart of a method for generating 3D data on an affected area cover according to another embodiment of the present specification.

Referring to FIG. 17 , the 3D data generation method for the affected part cover according to another embodiment of the present specification may be performed after the last step S55 of FIG. 15 above.

First, in step S56, it is possible to calculate the maximum value with respect to the Z-axis boundary points of the point cloud constituting the 3D data of the developed affected part.

In the next step S57, all values for the Z-axis of the boundary points may be substituted with the maximum value.

In the next step S58, the Z value of the point cloud may be filled with the maximum value up to a point extended by a preset offset in the X and Y directions of the boundary points. In addition, the Z value of the point cloud may be filled with 0 from a point after the offset. That is, in the process of manufacturing the cover for the affected area, it is a process of erecting the wall of the container that can contain the cover for the affected area. Referring to the bottom of Figure 17, it can be confirmed the 3D shape of the container for manufacturing the cover for the affected area generated through the above process.

The method may be implemented with a processor, application-specific integrated circuit (ASIC), other chipset, logic circuit, register, communication modem, data processing device, etc. known in the art for executing calculation and various control logic. have. In addition, when the above-described control logic is implemented in software, the control unit may be implemented as a set of program modules. In this case, the program module may be stored in the memory device and executed by the processor.

The program is, in order for the computer to read the program and execute the methods implemented as a program, C/C++, C#, JAVA, Python, which the processor (CPU) of the computer can read through the device interface of the computer; It may include code coded in a computer language such as machine language. Such code may include functional code related to a function defining functions necessary for executing the methods, etc., and includes an execution procedure related control code necessary for the processor of the computer to execute the functions according to a predetermined procedure. can do. In addition, the code may further include additional information necessary for the processor of the computer to execute the functions or code related to memory reference for which location (address address) in the internal or external memory of the computer to be referenced. have. In addition, when the processor of the computer needs to communicate with any other computer or server in a remote location in order to execute the functions, the code uses the communication module of the computer to determine how to communicate with any other computer or server remotely. It may further include a communication-related code for whether to communicate and what information or media to transmit and receive during communication.

The storage medium is not a medium that stores data for a short moment, such as a register, a cache, a memory, etc., but a medium that stores data semi-permanently and can be read by a device. Specifically, examples of the storage medium include, but are not limited to, ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device. That is, the program may be stored in various recording media on various servers accessible by the computer or in various recording media on the computer of the user. In addition, the medium may be distributed in a computer system connected to a network, and a computer-readable code may be stored in a distributed manner.

As mentioned above, although the embodiments of the present specification have been described with reference to the accompanying drawings, those of ordinary skill in the art to which this specification belongs can realize that the present invention may be embodied in other specific forms without changing the technical spirit or essential features thereof. you will be able to understand Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

Claims (6)

A method of generating a cover of an affected area using 3D data on the affected area, comprising:
(a) storing 3D data consisting of a point cloud for the developed affected part;
(b) removing the point cloud located inside based on the boundary of the affected part among the point clouds constituting the 3D data;
(c) dividing the 3D data according to a preset interval based on a first axis among three axes serving as a reference of the three-dimensional coordinates;
(d) calculating a curvature equation through regression analysis on the 3D data divided in step (c) (hereinafter, 'slicing 3D data');
(e) calculating the coordinate values for the third axis (hereinafter, 'cover point cloud') by substituting the coordinate values for the first and second axes of the point cloud included in the slicing 3D data into the curve equation; and
(f) removing the point out of the boundary of the affected part of the cover point cloud and combining with the point cloud for the developed affected part; 3D data generation method for cover of the affected part comprising a. The method according to claim 1,
The step (a) is,
(a-1) calculating inclination values for the first axis and the second axis from the stored 3D data;
(a-2) calculating a vector sum of the calculated gradient values; and
(a-3) rotating the 3D data stored as much as the inclination between the vector sum and the second axis in the second axis direction; further comprising, the affected area cover 3D data generation method. The method according to claim 1,
The step (c) is a step of dividing the 3D data according to the 1/n interval (n is a natural number) of the average point interval of the point cloud constituting the 3D data, the 3D data generating method for the affected area. The method according to claim 1,
The curve equation of the wound boundary is in the form of a quadratic function, the affected area 3D data development method. The method according to claim 1,
(g) calculating the maximum value with respect to the third axis of the boundary points among the point clouds constituting the 3D data of the developed affected part;
(h) substituting all the values for the third axis of the boundary points with the maximum value; and
(i) Fill the value of the third axis of the point cloud with the maximum value to a point extended by a preset offset in the first and second axis directions of the boundary points, and the third point of the point cloud from the point after the offset Filling in the value for the axis with 0; the affected area cover 3D data generation method further comprising. A computer program recorded in a computer-readable recording medium written to perform each step of the method for generating cover 3D data according to any one of claims 1 to 5 in a computer.

Images