CN114419137A

CN114419137A - Method, device, equipment and storage medium for straightening tubular object

Info

Publication number: CN114419137A
Application number: CN202210326060.7A
Authority: CN
Inventors: 周岩峰; 徐贤; 杨戈; 曹丰; 崔龙彪; 郭远昊; 孙沙沙; 蒋嘉诚
Original assignee: Institute of Automation of Chinese Academy of Science; University of Chinese Academy of Sciences; Second Medical Center of PLA General Hospital
Current assignee: Institute of Automation of Chinese Academy of Science; University of Chinese Academy of Sciences; Second Medical Center of PLA General Hospital
Priority date: 2022-03-30
Filing date: 2022-03-30
Publication date: 2022-04-29
Anticipated expiration: 2042-03-30
Also published as: CN114419137B

Abstract

The present invention relates to a method, apparatus, device and computer readable storage medium for straightening a tubular. The method comprises the following steps: determining the central line of a tubular object in the image to be detected; constructing a normal plane coordinate system at each central point on the central line based on the central line of the tubular object; determining the gray value of each coordinate point on the normal plane at each central point according to the coordinate transformation between the normal plane coordinate system and the original coordinate system in the image to be detected so as to obtain a section image at each central point; and stacking the sectional images at the central points based on the arrangement sequence of the central points on the central line to obtain a straightened image of the tubular object. According to the method provided by the embodiment of the invention, the three-dimensional straightening image information of the tubular object can be obtained, so that the morphological characteristics of the tubular object can be more comprehensively retained.

Description

Method, device, equipment and storage medium for straightening tubular object

Technical Field

The present invention relates generally to the field of image processing. More particularly, the present invention relates to a method, apparatus, device and computer readable storage medium for straightening a tubular in an image.

Background

In the field of medical image processing, visualization of tubular structure tissue (e.g. blood vessels) is a topic of intense research. Generally, the shape of a tubular structure tissue obtained from a medical image is complicated, and the form of the tubular structure tissue is often determined by the experience of a doctor or a researcher, and thus, there are disadvantages that the recognition speed is slow and the determination standard is not uniform. In order to facilitate the quantification and analysis of morphological characteristic parameters of the tubular structure tissue, the tubular structure tissue needs to be straightened before the quantification and analysis. Currently, the commonly used straightening processing method loses three-dimensional structure information in the straightening process, and only can obtain a straightened two-dimensional image, so that accurate measurement and analysis of tubular structure tissues are not facilitated.

Disclosure of Invention

In view of the above-mentioned technical problems, the technical solutions of the present invention provide, in various aspects, a method, an apparatus, a device, and a computer-readable storage medium for straightening a tubular in an image.

In a first aspect of the invention, there is provided a method for straightening a tubular in an image, comprising: determining the central line of a tubular object in the image to be detected; constructing a normal plane coordinate system at each central point on the central line based on the central line of the tubular object; determining the gray value of each coordinate point on the normal plane at each central point according to the coordinate transformation between the normal plane coordinate system and the original coordinate system in the image to be detected so as to obtain a section image at each central point; and stacking the sectional images at the central points based on the arrangement sequence of the central points on the central line to obtain a straightened image of the tubular object.

In one embodiment, determining the centerline of the tubular in the image under test comprises: and performing skeleton extraction on the tubular object in the image to be detected to determine the central line of the tubular object.

In another embodiment, skeletonizing the tubular to determine the centerline includes: preprocessing skeleton lines obtained by skeleton extraction to generate the preprocessed center lines, wherein the preprocessing comprises at least one of the following: removing burrs on the skeleton line; removing the annular structure on the skeleton line; and a skeleton line communicated with the inside of the same tubular object.

In yet another embodiment, constructing the normal plane coordinate system at each center point comprises: performing spline fitting on the center line of the tubular object for one time or multiple times to obtain a fitting center line; determining a tangent vector for each center point based on the fitted centerlines; and constructing a normal plane coordinate system at each central point according to the tangent vector of each central point.

In one embodiment, constructing the normal plane coordinate system at each center point based on the tangent vector of each center point comprises: selecting vectors in two directions which are orthogonal to tangent vectors of the starting point and are mutually orthogonal for the starting point of the fitting central line to construct a normal plane coordinate system at the starting point; determining a rotation angle and a rotation direction between normal planes at two adjacent central points according to tangent vectors of the two adjacent central points; and determining a normal plane coordinate system at the central point which is ranked later in the two adjacent central points based on the rotation angle and the rotation direction.

In another embodiment, determining the gray value according to the coordinate transformation between the normal plane coordinate system and the original coordinate system in the image to be measured includes: determining a rotation matrix and a translation matrix between the normal plane coordinate system at each central point and the original coordinate system according to the normal plane coordinate system at each central point on the central line; determining a transformation matrix between the normal plane coordinate system at each central point and the original coordinate system according to the rotation matrix and the translation matrix; and performing coordinate transformation on each coordinate point on the normal plane at each central point based on the transformation matrix to determine the gray value of each coordinate point.

In yet another embodiment, determining the grayscale value based on the transformation matrix includes: according to the radius of the tubular object at each central point, constructing a first coordinate matrix of each coordinate point on a normal plane at each central point; determining a second coordinate matrix of each coordinate point under the original coordinate system according to the first coordinate matrix and the transformation matrix; and calculating the gray value of each coordinate point on the normal plane at each central point by using an image interpolation algorithm based on the second coordinate matrix.

In one embodiment, further comprising: determining morphological features of the tubular based on the straightened image, wherein the morphological features include at least one of: a curvature; a length; an average diameter; an average radius; a cross-sectional radius; a cross-sectional diameter; the rate of stenosis.

In another embodiment, determining the morphological feature of the tubular comprises at least one of: determining a section radius or a section diameter of the tubular object at each central point based on the section images at each central point of the straightened tubular object in the straightened images; determining an average radius or average diameter of the tubular based on the cross-sectional radius or cross-sectional diameter at each center point; and determining a stenosis rate of the tubular based on the minimum section diameter of the section diameters and the average radius or based on the minimum section diameter of the section diameters and the average diameter.

In yet another embodiment, determining the section radius or section diameter at each center point comprises: setting a plurality of intersection lines on each sectional image with the central point in the sectional image as an intersection point; detecting the intersection point of each cross line and the edge of the tubular object in the cross-sectional image to determine sampling points in the cross-sectional image; and determining the section radius or the section diameter at the central point according to the average value of the distance between each sampling point and the intersection point.

In one embodiment, determining the sample points in the cross-sectional image comprises: dividing each cross line into two cross sub-lines at the cross point; and determining the intersection point which is closest to the intersection point on each cross sub-line as the sampling point.

In another embodiment, determining the morphological feature of the tubular comprises: performing weighted average operation on each section radius according to the distance between the central point of the minimum section radius in each section radius in the straightened image and each central point to determine the average radius; or performing weighted average operation on each section diameter according to the distance between the central point where the minimum section diameter in each section diameter in the straightened image is located and each central point to determine the average diameter.

In yet another embodiment, the tube comprises a cerebral arterial vessel.

In a second aspect of the invention, there is provided an apparatus for straightening a tubular in an image, comprising: a central line determining module configured to determine a central line of a tubular object in the image to be measured; a coordinate system construction module configured to construct a normal plane coordinate system at each center point on the centerline based on the centerline of the tubular; the gray value determining module is configured to determine the gray value of each coordinate point on the normal plane at each central point according to the coordinate transformation between the normal plane coordinate system and the original coordinate system in the image to be detected so as to obtain a cross-sectional image at each central point; and a stacking module configured to stack the sectional images at the respective center points based on an arrangement order of the respective center points on the center line to obtain a straightened image of the tubular object.

In a third aspect of the invention, there is provided an apparatus for straightening a tubular in an image, comprising, at least one processor; a memory storing program instructions that, when executed by the at least one processor, cause the apparatus to perform the method according to any one of the first aspects of the invention.

In a fourth aspect of the invention, a computer readable storage medium is provided, storing a program for straightening a tubular in an image, which program, when executed by a processor, performs the method according to any one of the first aspects of the invention.

Through the above description of the technical solution and the embodiments of the present invention, those skilled in the art can understand that the method for straightening a tubular object in an image according to the present invention can determine the gray value and the cross-sectional image of each coordinate point on each normal plane by constructing the normal plane coordinate system at each central point of the tubular object, and can obtain three-dimensional straightened image information of the tubular object by stacking the cross-sectional images, so that the morphological characteristics of the tubular object can be more comprehensively retained, and the subsequent analysis and measurement of the tubular object are facilitated, so as to obtain a more accurate analysis result.

Drawings

The above and other objects, features and advantages of exemplary embodiments of the present invention will become readily apparent from the following detailed description read in conjunction with the accompanying drawings. In the accompanying drawings, several embodiments of the present invention are illustrated by way of example and not by way of limitation, and like reference numerals designate like or corresponding parts throughout the several views, in which:

FIG. 1 is a flow chart illustrating a method for straightening a tubular in an image according to an embodiment of the present invention;

FIG. 2a is a schematic diagram illustrating the effect of straightening a tubular according to an embodiment of the present invention;

FIG. 2b is a schematic diagram illustrating a process of straightening a tubular in accordance with an embodiment of the present invention;

fig. 3 is a schematic view illustrating a process of removing burrs on a skeleton line according to an embodiment of the present invention, wherein (a) in fig. 3 is a schematic view illustrating a tubular object having burrs on the skeleton line according to an embodiment of the present invention, and (b) in fig. 3 is a schematic view illustrating a center line of the tubular object after removing the burrs according to an embodiment of the present invention;

fig. 4 is a schematic view illustrating a process of removing a loop structure on a skeleton line according to an embodiment of the present invention, wherein (a) in fig. 4 is a schematic view illustrating a tubular object having a loop structure on a skeleton line according to an embodiment of the present invention, and (b) in fig. 4 is a schematic view illustrating a center line of the tubular object after removing the loop structure according to an embodiment of the present invention;

fig. 5 is a schematic view illustrating a process of removing a loop structure on a skeleton line according to another embodiment of the present invention, in which fig. 5 (a) is a schematic view illustrating a tubular object having a loop structure on a skeleton line according to another embodiment of the present invention, and fig. 5 (b) is a schematic view illustrating a center line of the tubular object after removing the loop structure according to another embodiment of the present invention;

fig. 6 is a schematic view illustrating a process of communicating skeleton lines within the same tubular object according to an embodiment of the present invention, wherein (a) in fig. 6 is a schematic view illustrating that a plurality of nodes are included on the skeleton line of the same tubular object according to an embodiment of the present invention, and (b) in fig. 6 is a schematic view illustrating a centerline of the tubular object after communicating the skeleton lines of the same tubular object according to an embodiment of the present invention;

FIG. 7 is a flow chart illustrating a method of constructing a normal plane coordinate system according to an embodiment of the present invention;

FIG. 8 is a schematic diagram illustrating the construction of a normal plane coordinate system according to an embodiment of the invention;

FIG. 9 is a flow chart illustrating a method of determining a grayscale value for each coordinate point on a normal plane according to an embodiment of the invention;

FIG. 10 is a flow chart illustrating a method of determining a section radius or section diameter in accordance with an embodiment of the present invention;

fig. 11 is a schematic view showing arrangement of crossing lines according to an embodiment of the present invention, in which fig. 11 (a) is a schematic view showing arrangement of crossing lines for a circular blood vessel section according to an embodiment of the present invention, fig. 11 (b) is a schematic view showing arrangement of crossing lines for an oval blood vessel section according to an embodiment of the present invention, fig. 11 (c) is a schematic view showing arrangement of crossing lines for a blood vessel section in which a depression is present according to an embodiment of the present invention, fig. 11 (d) is a schematic view showing arrangement of crossing lines for a blood vessel section in which a blood vessel adhesion or a non-blood vessel region is present according to an embodiment of the present invention, and fig. 11 (e) is a schematic view showing arrangement of crossing lines for a blood vessel section in which a protrusion is present according to an embodiment of the present invention;

FIG. 12 is a comparison graph showing various cross-sectional radius measurement methods;

FIG. 13 is a schematic diagram illustrating the determination of the average radius according to an embodiment of the present invention;

FIG. 14 is a diagram illustrating weight versus distance according to an embodiment of the present invention;

FIG. 15 is a graph showing the error comparison between various stenosis rate determination methods and imaging physician standard results; and

FIG. 16 is a schematic diagram illustrating a system for straightening a tubular in an image according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

It should be understood that the terms "first", "second", "third" and "fourth", etc. in the claims, the description and the drawings of the present invention are used for distinguishing different objects and are not used for describing a particular order. The terms "comprises" and "comprising," when used in the specification and claims of this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

It is also to be understood that the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only, and is not intended to be limiting of the disclosure. As used in the specification and claims of this application, the singular form of "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should be further understood that the term "and/or" as used in the specification and claims of this specification refers to any and all possible combinations of one or more of the associated listed items and includes such combinations.

As used in this specification and claims, the term "if" may be interpreted contextually as "when", "upon" or "in response to a determination" or "in response to a detection". Similarly, the phrase "if it is determined" or "if a [ described condition or event ] is detected" may be interpreted contextually to mean "upon determining" or "in response to determining" or "upon detecting [ described condition or event ]" or "in response to detecting [ described condition or event ]".

Aiming at the defects of the prior art, the invention provides a brand-new realizable solution. Particularly, the method for straightening the tubular object in the image can obtain the sectional images at the central points and obtain the straightened images by overlapping the sectional images by constructing the normal plane coordinate system at the central points of the tubular object in the image to be detected, can obtain the complete three-dimensional information of the straightened tubular object by the process of obtaining the information of each sectional image, can embody the morphological characteristics of the central points of the tubular object more truly and accurately, and is favorable for obtaining more accurate quantitative parameters so as to facilitate scientific research and clinical diagnosis.

As will be appreciated by those skilled in the art from the following description, the present invention also provides embodiments for improving the straightening effect and accuracy in various embodiments. For example, in some embodiments, a smooth and complete center line may be obtained through a preprocessing manner, and based on the center line obtained after preprocessing, it is beneficial to accurately determine each center point on the center line, so that adverse effects that burrs, annular structures and the like may cause on constructing a normal plane coordinate system may be avoided, and accuracy of the straightening processing process may be ensured. The following detailed description of embodiments of the invention refers to the accompanying drawings.

FIG. 1 is a flow chart illustrating a method for straightening a tubular in an image according to an embodiment of the present invention. As shown in fig. 1, the method 100 may include: in step 110, the centerline of the tubular in the image under test may be determined. In some embodiments, the image under test may comprise a three-dimensional image. In other embodiments, the image under test may include a medical image, such as a Magnetic Resonance Angiography (MRA) image, a Computed Tomography (CT) image, a B-mode ultrasound image, or the like. In still other embodiments, one or more tubes may be included in the image to be measured. In some embodiments, the tube may comprise at least one of a blood vessel, small intestine, fallopian tube, etc. tubular tissue. In one embodiment of the invention, the tube may comprise at least one of a cardiovascular, cerebrovascular, capillary, arterial, venous vessel, and the like. In yet another embodiment, the tube may comprise a cerebral arterial vessel.

In some embodiments, the centerline of the tubular may be a refinement of the overall tubular, which may be used to reflect the overall skeletal structure of the tubular. In other embodiments, the centerline of the tubular may be the skeleton line of the tubular. In some application scenarios, for example when the tubular object has a cross section with an irregular shape, the center point of the cross section on the center line of the tubular object may not be at the center of the circle in the strict sense.

In one embodiment, step 110 may comprise: and performing skeleton extraction on the tubular object in the image to be detected to determine the central line of the tubular object. The skeleton extraction technology (or binary image thinning technology) can be used for thinning the tubular object in the image to be detected, and the skeleton line of the tubular object can be obtained. In some embodiments, the skeleton line may be taken as the center line. Each tube may correspond to a centerline. In some embodiments, the image to be measured includes a tubular, and the tubular may be skeleton-extracted to determine a centerline of the tubular. In other embodiments, the image to be measured may include a plurality of tubes, and all or a part of the tubes in the image to be measured may be skeleton-extracted to determine the center lines of all or a part of the tubes.

Next, in step 120, a normal plane coordinate system may be constructed at each center point on the centerline based on the centerline of the tubular. In some embodiments, each point on the centerline may be a center point of the tube cross-section where the centerline may be a collection of center points. In other embodiments, step 120 may include: determining a tangent vector at each central point; and constructing a normal plane coordinate system at each central point according to the tangent vector of each central point.

In still other embodiments, constructing the normal plane coordinate system at each center point may include: determining an x-axis vector (or called a horizontal coordinate axis vector) and a y-axis vector (or called a vertical coordinate axis vector) which are pairwise perpendicular to the tangent vector based on the tangent vector of each central point; and constructing a normal plane coordinate system (or space rectangular coordinate system) at each central point based on the tangent vector, the x-axis vector and the y-axis vector at each central point. In some embodiments, constructing the normal plane coordinate system at each center point based on the tangent vector, the x-axis vector, and the y-axis vector at each center point may include: and establishing a space rectangular coordinate system at each central point by taking each central point as an origin and taking the tangent vector, the x-axis vector and the y-axis vector at each central point as coordinate axes. In other embodiments, the determination of the x-axis vector and the y-axis vector perpendicular to the tangent vector in pairs may be implemented by using a rodlike rotation formula, which can ensure that there is no rotation error between the normal plane coordinate systems at the adjacent central points.

Then, the process may proceed to step 130, and the gray-scale values of the coordinate points on the normal plane at each central point may be determined according to the coordinate transformation between the normal plane coordinate system and the original coordinate system in the image to be measured, so as to obtain the cross-sectional image at each central point. In some embodiments, the coordinate transformation may include a translation transformation and/or a rotation transformation, among others. According to the coordinate transformation between the normal plane coordinate system at each central point and the original coordinate system in the image to be detected, the coordinate position of each coordinate point on the normal plane at each central point corresponding to the original coordinate system can be determined, and therefore the gray value of the corresponding coordinate point on the normal plane can be determined according to the gray value of each coordinate position in the image to be detected. An image of the entire normal plane, i.e., a cross-sectional image at each center point, can be formed from the grayscale values of the respective coordinate points on the normal plane at each center point.

Further, in step 140, the sectional images at the respective center points may be superimposed based on the arrangement order of the respective center points on the center line to obtain a straightened image of the tubular object. The overlapping is carried out based on the arrangement sequence of the central points, so that the consistency of the morphological characteristics of the straightened tubular object and the tubular object before straightening can be ensured. The cross-sectional images at the respective center points may be stacked in one direction to form a three-dimensional straightened image.

In some embodiments, the stacking of the cross-sectional images at the respective center points to obtain the straightened image may be obtained based on the following formula one:

(formula one)

Where, Fi denotes a straightened image,

the second representing a tubular object i

In oneThe gray value of the cross-sectional image at the center point,

indicating the length of the centerline of the tubular i. In some embodiments, the length of each centerline may be determined based on its start and stop points.

Fig. 2a is a schematic view illustrating the effect of straightening a tubular object according to an embodiment of the present invention. As shown in fig. 2a, a tubular 201 may be straightened using, for example, the method 100 shown in fig. 1 to obtain a straightened image, wherein the straightened image may include a straightened tubular 202 after the tubular 201 has been straightened.

Fig. 2b is a schematic diagram illustrating a process of straightening a tubular according to an embodiment of the present invention. As shown in fig. 2b, a normal plane coordinate system at each center point on the center line of the tubular 203 may be constructed according to step 120 described in fig. 1, and then the sectional images at each center point of the tubular 203 are superimposed on the same straight line by performing step 130 and step 140, thereby obtaining a three-dimensional straightened image of the straightened tubular 204.

While the method for straightening the tubular objects in the image according to the embodiment of the present invention is exemplarily described above with reference to fig. 1-2 b, it can be understood that, in some application scenarios, when a plurality of tubular objects are included in the image to be measured, the center line of each tubular object may be determined, and steps 120-140 may be separately performed to obtain a straightened image of each tubular object. It is also understood that the method according to the embodiment of the present invention may not be limited to the above steps, for example, in one embodiment, step 110 may further include: preprocessing the skeleton line obtained by skeleton extraction to generate a preprocessed center line, wherein the preprocessing may include at least one of: removing burrs on the skeleton line; removing the annular structure on the skeleton line; and a skeleton line communicated with the inside of the same tubular object. In some embodiments, the skeleton line may include a set of multiple nodes (or center points). For ease of understanding, the following exemplary description is provided in connection with fig. 3-6.

Fig. 3 is a schematic view showing a process of removing burrs on a skeleton line according to an embodiment of the present invention. As shown in fig. 3 (a), skeleton extraction of a tubular object 301 may obtain a skeleton line 302, and a burr 303 is connected at a node 304 of the skeleton line 302. In some application scenarios, for a complete tube, the starting point and the ending point are usually included in the node set on the skeleton line, and all nodes including the starting point and the ending point should theoretically appear only once. However, for the skeleton line 302 in which the burr is present, the node 304 appears once in the node set as both the start point or the end point of the burr 303 and the non-start point of the skeleton line 302, that is, the node 304 appears twice in the node set of the skeleton line 302. In other embodiments, the length between the start and stop points of the burr 303 is shorter.

Based on the above-mentioned features of the burr 303, by traversing the skeleton line, the position of the node 304 where the burr 303 appears may be determined based on the number of times the central point (or node) on the skeleton line appears in the node set and/or the length of the skeleton line, so that the burr 303 may be removed to obtain the central line 305 after the burr is removed, as shown in (b) of fig. 3.

Fig. 4 is a schematic view illustrating a process of removing a loop structure on a skeleton line according to an embodiment of the present invention. As shown in (a) of fig. 4, skeleton extraction of a tubular object 401 may obtain a skeleton line 402, where a ring-shaped structure composed of a first skeleton line 405 and a second skeleton line 406 exists between a first node 403 and a second node 404 on the skeleton line 402. In some embodiments, the first node 403 and the second node 404 each occur at least twice in the set of nodes in which the skeleton line 402 of the ring structure exists. Based on such a feature of the ring structure, the positions of the first node 403 and the second node 404 can be found by traversing the skeleton lines, and the first skeleton line 405 or the second skeleton line 406 in the ring structure can be selectively deleted according to the length relationship between the first skeleton line 405 and the second skeleton line 406, so as to obtain the center line 407 after removing the ring structure, as shown in (b) of fig. 4. In other embodiments, one of the first and

second skeleton lines

405 and 406 may be randomly removed in response to a length ratio of the first and

second skeleton lines

405 and 406 being greater than a second threshold. In still other embodiments, the second threshold may comprise 0.8-1.2.

Fig. 5 is a schematic view illustrating a process of removing a loop structure on a skeleton line according to another embodiment of the present invention. As shown in (a) of fig. 5, skeleton extraction of a tubular object 501 may obtain a skeleton line 502, where a ring-shaped structure composed of a first skeleton line 505 and a second skeleton line 506 exists between a first node 503 and a second node 504 on the skeleton line 502. In some embodiments, in response to the length ratio of the first skeleton line 505 and the second skeleton line 506 being less than the third threshold, a shorter center line (e.g., the first skeleton line 505 in the illustration) in the ring-shaped structure may be removed to obtain a center line 507 after removing the ring-shaped structure, as shown in (b) in fig. 5. In other embodiments, the third threshold may be 1/3. According to the arrangement, the influence of adhesion of the tubular objects on the construction of the normal plane coordinate system can be eliminated.

In some application scenarios, the two nodes forming the ring structure may not be limited to the first skeleton line and the second skeleton line shown in fig. 4 and 5, and a greater number of skeleton lines of the ring structure may also be present, and the method for removing the ring structure in such a scenario may be similar to that described above in conjunction with fig. 4 and 5, for example, every two skeleton lines may be compared and removed until only one skeleton line is finally left.

Fig. 6 is a schematic view illustrating a process of communicating skeleton lines within the same tubular object according to an embodiment of the present invention. As shown in (a) in fig. 6, skeleton extraction is performed on a tubular 601 to obtain a skeleton line 602. In some embodiments, the skeleton line 602 of the same tubular object may include a plurality of nodes (e.g., 603, 604, 605, 606, etc. in the figure) that occur multiple times, so that the skeleton line 602 is divided into a plurality of tube segments, and in response to the absence of burrs and/or ring structures between the plurality of nodes, a complete skeleton line (e.g., the preprocessed center line 607 shown in fig. 6 (b)) may be formed by connecting the skeleton lines between the plurality of tube segments. In other embodiments, the communication of the skeleton lines within the same tubular may be performed after deburring and/or removing the annular structure.

As described above with reference to fig. 3 to 6, by way of example, the skeleton line preprocessing according to the embodiment of the present invention is described, it can be understood that, through the above preprocessing operation, the skeleton line extracted from the skeleton of the tubular object can be corrected to eliminate the influence of situations, such as segmentation errors or adhesion of the tubular object, on the straightening processing, which is beneficial to reducing the data processing amount of subsequent operations, such as constructing a planar coordinate system and determining gray scale values, and can significantly improve the accuracy of the obtained straightened image. After obtaining the center line of the tubular object in the image to be measured, the normal plane coordinate system may be constructed based on the center line, and in order to make it easier to understand the implementation of the construction of the normal plane coordinate system according to the embodiment of the present invention, a specific embodiment of the construction of the normal plane coordinate system will be described below.

Fig. 7 is a flowchart illustrating a method of constructing a normal plane coordinate system according to an embodiment of the present invention. As will be appreciated from the following description, the method 700 shown in fig. 7 may be an embodied expression of the step 120 described above in connection with fig. 1, and thus the description of the step 120 in the foregoing may also be applied to the description of the method 700 below.

As shown in fig. 7, method 700 may include: in step 710, one or more spline fits may be performed to the centerline of the tubular to obtain a fitted centerline. Spline fitting may be used to smooth the centerline of the tubular for subsequent operations to determine tangent vectors. For ease of understanding, the following description will be given by taking the example of cubic spline fitting on the centerline of a tubular.

In some embodiments, the set of center points on the centerline of a tube i may be P_i=

Wherein

The length of the centerline, the coordinates of the jth center point in the set is

. Cubic spline fitting of the centerline of the tubular i may result in a fitted centerline, such that the fitted centerline is smooth and conductive. The coordinates of the center point of the fitted centerline may be

The set of center points on the fitted centerline may be P_i=

Wherein

，

，

And t may be used to represent a curve parameter that fits the centerline.

Next, in step 720, a tangent vector for each center point may be determined based on the fitted centerlines. The tangent vector may be obtained by, for example, a transposition operation of the reciprocal of the coordinates of the center point. In particular, the center point p_ijThe tangent vector of (b) can be calculated by the following formula two:

(formula two)

Wherein the content of the first and second substances,

representing the central point p_ijThe vector of the tangent at the point (c),

，

，

。

the process may then proceed to step 730, where a normal plane coordinate system at each center point may be constructed based on the tangent vectors at each center point. In some embodiments, step 730 may include: determining a normal plane coordinate axis at each central point according to the tangent vector of each central point; and constructing a normal plane coordinate system at each central point based on the normal plane coordinate axes at each central point.

As further shown in fig. 7, in one embodiment, step 730 may comprise: in step 731 (shown in dashed box), for the starting point of the fitted centerline, vectors in two directions orthogonal to the tangent vector of the starting point and to each other may be chosen to construct a normal plane coordinate system at the starting point. Specifically, in one embodiment, it is assumed that the normal plane coordinate axis for each center point includes

If j =0 (i.e. the beginning of the fitted centerline), then the and is taken arbitrarily

Orthogonal and mutually orthogonal vectors as

If it is taken

Then, then

。

Next, in step 732 (shown by a dashed box), a rotation angle and a rotation direction between normal planes at two adjacent central points can be determined according to tangent vectors of the two adjacent central points. For ease of understanding, the following description is made in conjunction with fig. 8.

Fig. 8 is a schematic diagram illustrating the construction of a normal plane coordinate system according to an embodiment of the present invention. As shown in the schematic view of figure 8,

a unit direction vector of the rotation axis, i.e., the intersection line of the jth normal plane and the j-1 th normal plane, which can be used to reflect the rotation direction between the normal planes at two adjacent center points.

The calculation formula of (c) may be:

(formula three).

Further, the air conditioner is provided with a fan,

and the rotation angle between the normal planes at two adjacent central points, namely the included angle between the jth normal plane and the jth-1 normal plane is shown.

The calculation formula of (c) may be:

(equation four).

Then, each center point on the fitted center line may be used as an origin to

For the coordinate axes, a space rectangular coordinate system (namely a normal plane coordinate system) at each central point is established

。

Continuing with the description returning to FIG. 7, at step 733 (shown by a dashed line), a normal plane coordinate system at a central point ranked further back in two adjacent central points can be determined based on the rotation angle and the rotation direction. For example, the jth normal plane is the normal plane at the center point ordered later than the jth-1 normal plane. Specifically, in one embodiment, if j>0 (i.e., the non-starting point of the fitted centerline), can be obtained using, for example, the rodreg rotation equation

And no rotation error can be ensured between the normal plane coordinate systems of the jth central point and the jth-1 central point. Calculated by using the Rodrigue rotation algorithm

The formula of (1) is as follows:

(formula five);

(equation six).

Wherein RRF represents the Rodrigue rotation algorithm,

showing the axis of rotation of the shaft,

which represents the rotation angle between normal planes at two adjacent center points (the j-1 st center point and the j-th center point).

Represents the x-axis coordinate vector at the j-1 st center point,

represents the y-axis coordinate vector at the j-1 st center point,

representing the x-axis coordinate vector at the jth center point,

representing the y-axis coordinate vector at the jth center point.

While the specific implementation of constructing the normal plane coordinate system according to the embodiment of the present invention is exemplarily described above with reference to fig. 7 and 8, after the normal plane coordinate system is determined, the gray-scale values of the coordinate points on the normal plane may be determined based on the normal plane coordinate system so as to obtain the cross-sectional image. A specific implementation of determining the gray-scale value according to an embodiment of the present invention will be exemplarily described below with reference to fig. 9.

FIG. 9 is a flow chart illustrating a method for determining grayscale values for coordinate points on a normal plane according to an embodiment of the invention. As will be appreciated from the following description, the method 900 shown in fig. 9 may be an embodied expression of the step 130 described above in connection with fig. 1, and thus the description of the step 130 in the foregoing may also be applied to the following description of the method 900.

As shown in fig. 9, method 900 may include: in step 910, a rotation matrix and a translation matrix between the normal plane coordinate system at each center point and the original coordinate system may be determined according to the normal plane coordinate system at each center point on the center line. In some embodiments, the translation matrix may be obtained by a transpose operation of the coordinates of the center point. For example, the translation matrix M_ijCan be calculated by the following formula seven:

(formula seven)

Wherein M is_ijTranslation matrix, p, representing the jth centre point of a tubular i_ijThe coordinates of the jth center point of the tubular i are indicated.

In othersIn the embodiment, the rotation matrix R_ijCan be calculated by the following formula eight.

(formula eight)

Wherein the content of the first and second substances,

representing the axial vector of the original coordinate system. In some embodiments of the present invention, the,

，

representing a tubular object

To (1) a

The normal plane coordinate system corresponding to each central point.

Next, in step 920, a transformation matrix between the normal plane coordinate system and the original coordinate system at each center point may be determined according to the rotation matrix and the translation matrix. In one embodiment, the transformation matrix T_ijCan be calculated by the following formula nine:

(formula nine)

Wherein M is_ijIs a translation matrix (i.e. the transpose of the coordinates of the jth centre point of the tubular i), R_ijIs the rotation matrix (e.g., the calculation of equation eight) for the jth center point of the tubular i.

Then, the flow may proceed to step 930, and coordinate transformation may be performed on the respective coordinate points on the normal plane at each center point based on the transformation matrix to determine the grayscale values of the respective coordinate points. In one embodiment, step 930 may comprise: in step 931 (shown in dashed box), a first coordinate matrix for each coordinate point on the normal plane at each center point may be constructed based on the radius of the tubular at each center point. In some embodiments, the first coordinate matrix may be constructed according to the following equation:

(formula ten)

Wherein the content of the first and second substances,

representing a first coordinate matrix, r_ijRepresents the section radius at the jth center point of the tubular i, and k in equation ten represents the coordinate coefficient. In one embodiment, k may be 3-5 to ensure that the coordinate points at all pixel points of the cross-sectional image can be calculated.

Next, in step 932 (shown by the dashed box), a second coordinate matrix of each coordinate point in the original coordinate system may be determined according to the first coordinate matrix and the transformation matrix. In other embodiments, the second coordinate matrix may be calculated according to the following formula eleven:

(formula eleven)

Wherein the content of the first and second substances,

a second matrix of coordinates is represented, which,

representing a first coordinate matrix, T_ijRepresenting a transformation matrix.

Further, in step 933 (shown by a dashed box), a gray scale value of each coordinate point on the normal plane at each center point may be calculated by using an image interpolation algorithm based on the second coordinate matrix. In some embodiments, calculating the gray value using an image interpolation algorithm may be implemented by the following equation twelve:

(formula twelve)

Wherein f is_ijI (-) represents the gray value of each coordinate point on the normal plane at the jth central point of the tubular object I, I (-) represents an image interpolation algorithm, for example, at least one of nearest neighbor interpolation and bilinear interpolation can be adopted, and k represents a coordinate coefficient.𝑔(𝑥,𝑦) Represents a coordinate point (𝑥,𝑦0) gray value. After determining the gray values of the coordinate points, the entire cross-sectional image can be obtained, and then by overlaying the cross-sectional images at the center points, a straightened image of the tubular can be obtained.

In one embodiment, the method according to an embodiment of the present invention may further include: based on the straightened image, morphological features of the tubular are determined, wherein the morphological features may include at least one of: a curvature; a length; an average diameter; an average radius; a cross-sectional radius; a cross-sectional diameter; stenosis rate, etc. In some embodiments, the straightened tube in the straightened image can be detected to determine a morphological feature of the straightened tube and thus the corresponding tubular before straightening.

In one embodiment, determining the morphological feature of the tubular comprises at least one of: the curvature at each central point of the tube may be determined based on the tangent vector at each central point obtained in the straightening process of the tube; the length of the tubular can be determined according to the length of the central line; determining a section radius or a section diameter of the tubular object at each central point based on the section images at each central point of the straightened tubular object in the straightened image; determining an average radius or average diameter of the tubular based on the cross-sectional radius or cross-sectional diameter at each center point; and determining the stenosis rate of the tubular based on the minimum section radius and the average radius of the section diameters or the minimum section diameter and the average diameter of the section diameters.

In some embodiments, the curvature of the tube may be determined from tangent vectors at each center point, such as obtained in connection with step 720 of FIG. 7, supra. Specifically, it can be calculated based on the following formula thirteen:

(formula thirteen)

Wherein Cur represents the curvature of the tubular object,

center point p representing fitted center line_ijThe vector of the tangent at the point (c),

representing the central point p_ijThe vector of the second derivative of (a),

，

，

。

in one embodiment, the stenosis rate of the tubular may be determined based on a ratio of a minimum cross-sectional radius to an average radius of the cross-sectional radii, or based on a ratio of a minimum cross-sectional diameter to an average diameter of the cross-sectional diameters. The minimum cross-sectional radius may be the smallest of the respective cross-sectional radii of the tubular. The minimum cross-sectional diameter may be the smallest of the respective cross-sectional diameters of the tubes. In one embodiment, the stenosis rate of a tubular may be calculated based on the following equation:

(A)Fourteen type)

Wherein s is_iWhich is indicative of the rate of stenosis,

represents the smallest cross-sectional radius among the cross-sectional radii,

the average radius is indicated.

In another embodiment, the radius of the cross-section may be detected from the cross-sectional image by a method such as minimum distance method, area measurement method, or the like. In another embodiment, the cross-sectional diameter may be determined according to twice the cross-sectional radius. In one embodiment, the average radius may be determined from the average of the cross-sectional radii at each center point of the straightened tube. In another embodiment, the average radius may be determined from a weighted average of the cross-sectional radii at each center point of the straightened tube. In one embodiment, the average diameter may be determined from the mean of the cross-sectional diameters at each central point of the straightened tube. In another embodiment, the average diameter may be determined from a weighted average of the cross-sectional diameters at each center point of the straightened tube. The following exemplary description is provided in connection with fig. 10-14.

FIG. 10 is a flow chart illustrating a method of determining a section radius or section diameter in accordance with an embodiment of the present invention. As shown in fig. 10, the method 1000 may include: in step 1010, a plurality of intersecting lines may be disposed on the cross-sectional images with the center point in each cross-sectional image as the intersection point. In some embodiments, the plurality of crossing lines may be uniformly arranged, for example, the angle of separation between each adjacent two crossing lines in the plurality of crossing lines may be the same. In other embodiments, the plurality of crossing lines may be non-uniformly arranged, for example, the angle of separation between each adjacent two of the plurality of crossing lines may be different.

In still other embodiments, the intersection line may be a straight line passing through the intersection point. The larger the number of intersecting lines, the more accurate the detection result of the section radius or the section diameter. In other embodiments, the number of crossing lines may be set to 8 to 16. According to the arrangement, the accuracy of the section radius or the section diameter can be ensured, and the data processing amount can be reduced to improve the data processing speed. Further, the provision of more than 8 cross-lines can be better suited for use in the measurement of the cross-sectional radius or diameter of the cross-section of an irregular tubular object than if less than 8 (e.g., two) cross-lines were provided.

Next, in step 1020, the intersection of each intersection line with the edge of the tubular in the cross-sectional image may be detected to determine the sample points in the cross-sectional image. In some application scenarios, the cross-sectional image may include a tubular cross-section and a background image, and the cross-line may pass through the intersection point and extend into the background image such that the cross-line intersects the tubular edge to produce an intersection point. In other application scenarios, the intersection line may extend only to the tubular edge to intersect the tubular edge. In some embodiments, it may be determined that all of the intersection points are sample points.

In one embodiment, as shown in FIG. 10, step 1020 may comprise: in step 1021 (shown in dashed box), each crossing line may be split into two crossing sub-lines at the crossing point. Next, in step 1022 (shown by the dashed box), the intersection point on each cross sub-line closest to the intersection point may be determined as the sampling point. According to the arrangement, the influence of the irregular tubular object area possibly existing in the sectional image on the determination result of the sectional radius or the sectional diameter of the tubular object can be effectively eliminated, and the accuracy of the detection result is further improved.

Further, in step 1030, a cross-sectional radius or diameter at the center point may be determined based on an average of the distances between each sampling point and the intersection point. In one embodiment, the section radius may be calculated based on the following formula fifteen:

(formula fifteen)

Wherein r is_ijDenotes the section radius at the jth centre point of the tubular i, n denotes the sampling pointThe number of the first and second groups is,

denotes the distance between the kth sampling point and the intersection point, where f_ijRepresents the j-th sectional image of the tubular i.

In another embodiment, the cross-sectional diameter may be calculated sixteen based on the following equation:

(formula sixteen).

In yet another embodiment, step 1030 may include: the cross-sectional diameter at the center point is determined based on the average of the distances between two sampling points on each intersection line.

To facilitate understanding of the manner in which cross-hatching is provided to determine the cross-sectional radius or diameter of an embodiment of the present invention, an exemplary description will be given below in conjunction with fig. 11.

Fig. 11 is a schematic diagram illustrating the arrangement of cross-hatching according to an embodiment of the present invention. The drawings (a), (b), (c), (d), and (e) of fig. 11 show cross-sectional images of different blood vessel cross-sections, respectively, using a tubular object as an example of a blood vessel. As can be seen from fig. 11 (a), (b), (c), (d), and (e), a plurality of intersecting lines passing through the center point may be provided, and the sampling points (such as the light dots in the figure) may be determined according to the intersection points of the intersecting lines and the edge of the blood vessel, and the cross-sectional radius of the blood vessel may be determined according to the distance between the sampling point and the center point, or the cross-sectional diameter of the blood vessel may be determined according to the distance between two sampling points on the same intersecting line. By determining the intersection point on each intersection sub-line, which is closest to the intersection point, as a sampling point, the influence of the intersection point (e.g., a dark dot in the drawing) of the intersection line with the non-blood vessel region or the blood vessel adhesion region, as shown in (d) of fig. 11, on the determination result of the cross-sectional radius of the blood vessel can be effectively excluded.

It will be appreciated that the above embodiments with respect to determining a cross-sectional radius or diameter are exemplary and not limiting, and in another embodiment of the present invention, determining a cross-sectional radius based on a cross-sectional image may include: setting a plurality of radial lines towards the edge of the tubular object by taking the central point in each sectional image as a starting point; detecting the intersection point of each radioactive ray and the edge of the tubular object in the sectional image to determine sampling points in the sectional image; and determining the section radius of the tubular at the center point according to the average value of the distances between the sampling points and the center point. In some embodiments, the angle of separation between the plurality of radial lines may be the same or different. In other embodiments, the number of the radial lines may be set to 32 to 48. The technical effects of setting 32 to 48 radial lines are similar to those of setting 8 to 16 crossing lines with respect to the number setting, and will not be described herein.

Compared with a scheme of setting a cross line, the operation of setting the radioactive rays is simpler and more flexible, for example, odd number of sampling points can be obtained by setting odd number of radioactive rays, and the sampling points do not have symmetry, so that the sampling points can be more representative in some application scenes. Compared with the scheme of setting the radiation, the scheme of setting the cross line can directly determine the section diameter at the central point without determining the section radius and then determining the section diameter, and the section diameter directly determined based on the cross line is more accurate.

In yet another embodiment of the present invention, determining the cross-sectional radius at the center point may further comprise: determining the distance between the sampling point and the central point as a sampling radius; calculating the relative difference between each sampling radius and the section radius of the section image; in response to the relative difference being greater than a preset threshold, determining a sampling radius for which the relative difference is greater than the preset threshold as an abnormal sampling radius; and updating the section radius of the tubular according to the average value of other sampling radii except the abnormal sampling radius. According to the arrangement, the influence of local abnormal features (such as adhesion, abnormal bulges, burrs and the like) of the tubular object on the detection result of the section radius of the tubular object can be effectively eliminated, and the accuracy of the detection result of the section radius is further improved.

Compared with other methods for determining the section radius, the technical scheme for setting the cross line or the radioactive ray according to the embodiment of the invention is not only beneficial to improving the accuracy of the measurement result, but also can be more widely applied to the measurement of the section radius or the section diameter of the section of the tubular object with irregular shape caused by the tubular object or image processing errors and the like. This will be explained below with reference to fig. 12.

FIG. 12 is a comparison graph showing various cross-sectional radius measurement methods. As shown in fig. 12, the first row shows a cross-sectional image such as that shown in fig. 11. The minimum distance method is a method in which the distance between the point on the edge of the tubular closest to the center point and the center point is determined as the radius of the cross section.

The method is a method of calculating the radius of the cross section of a tubular object based on the area of the cross section of the tubular object.

The iteration is to

And (4) performing iterative computation on the result of the computation. In addition, the method of setting a cross line and the method of setting a radiation line in the embodiment of the present invention may be collectively referred to as a multipassage method. As is apparent from fig. 12, the multipath measurement method according to the embodiment of the present invention can accurately measure the cross-sectional radii of blood vessel cross-sections of various shapes, and other methods cannot achieve the applicable range of the multipath measurement method according to the embodiment of the present invention. In some application scenarios, two intersecting lines may be more suitable for detecting the vessel section radius of a circular vessel section, and in comparison, more than 8 intersecting lines can significantly improve the reliability and accuracy of the measurement result of a non-circular vessel section.

In another embodiment, determining the morphological feature of the tubular may comprise: performing weighted average operation on each section radius according to the distance between the central point of the minimum section radius in each section radius in the straightened image and each central point to determine an average radius; or performing weighted average operation on each section diameter according to the distance between the central point where the minimum section diameter is located and each central point in each section diameter in the straightened image so as to determine the average diameter. In one embodiment, the average radius may be calculated based on the following formula seventeen:

(formula seventeen)

Wherein the content of the first and second substances,

the average radius of the tube i is indicated,

represents the weight of the cross-sectional radius of the tubular in the cross-sectional image at the jth center point of the tubular i,

represents the cross-sectional radius of the tubular in the cross-sectional image at the j-th center point of the tubular i.

Eighteen can be calculated based on the following formula:

(eighteen formula)

Wherein the content of the first and second substances,

represents D_ijFunction of D_ijThe distance between the center point of the smallest section radius in the straightened image of the tubular i and the jth center point can be expressed as the following formula nineteen:

(formula nineteen)

Wherein the content of the first and second substances,x_ijthe abscissa representing the jth center point of the tube i,

the abscissa indicates the center point of the tube i at which the smallest cross-sectional radius is located. For example, as shown in FIG. 13, where x_i1The abscissa representing the 1 st center point of the tube i,

the second representing a tubular object i

Abscissa of the center point, D_ijRepresenting the distance between the section where the smallest section radius is located and the section where the jth centre point is located. In other embodiments, the calculation method of the average diameter may be similar to the calculation method of the average radius, and is not described herein again.

Further, in one embodiment of the present invention, the function

Can be constructed according to preset rules and then the respective section radius or the weight of the respective section diameter of the tubular can be determined based on the constructed function

And the weighted average operation can be performed on each section radius or each section diameter according to the determined weight to obtain an average radius or an average diameter. In other embodiments, the preset rules may include: distance D_ijThe greater the center point, the less the corresponding weight, such as shown in FIG. 14, where

A weight representing a section radius of a section of the tubular i of the section image at the 1 st center point of the tubular i,

weight representing minimum section radius，

The second representing a tubular object i

The weight of the section radius of the tubular section of the sectional image at the respective center point. By the preset rule, the influence of the distance from the minimum section radius or the minimum section diameter can be considered when the average radius or the average diameter is determined, so that the accuracy of the operation result is improved, and the real state of the tubular object can be reflected more objectively.

In still other embodiments, the function may include, for example, at least one of a first order function, a second order function, an inverse proportional function, a Gaussian function, and the like. In some embodiments of the present invention, the,

a weight representing the radius of the section at the jth center point of the tubular i, a linear function of the distance can be obtained by, for example, the following equation twenty:

(formula twenty)

Wherein the content of the first and second substances,

1 st to 1 st of the tubular object i

The maximum value among the distances of the sectional radii of the sectional images at the respective center points.

In another embodiment, the quadratic function of the distance may be obtained by, for example, the following equation twenty one:

(equation twenty one).

In yet another embodiment, an inverse proportional function of distance may be obtained by, for example, the following equation twenty-two:

(equation twenty-two).

In one embodiment, the gaussian function of the distance may be obtained by, for example, the following equation twenty three:

(formula twenty-three)

Wherein k may take a value of 3.

While the method for determining the morphological characteristics of the pipe, such as the section radius, the section diameter, the average radius, the average diameter, etc., is described above with reference to fig. 10-14, it will be understood by those skilled in the art that the above description is exemplary and not limiting, for example, the above-mentioned linear function, quadratic function, inverse proportion function, and gaussian function are exemplary, and in some embodiments, other functional forms satisfying the preset rule may be provided according to the needs.

In an experimental example, the average radius or the average diameter of the straightened images of 7 kinds of tubular objects is determined according to the start-stop point method, the average method and the weighted average method, and then the average error of the determined stenosis rate result is counted, so that the experimental result shown in fig. 15 can be obtained. The start-stop method may be a method of determining an average radius or an average diameter according to the radius or the diameter of the section at the start point and the end point of straightening the tubular object. The averaging method may be a method of directly determining an average radius or an average diameter from the average of the section radius or the section diameter at each center point of the straightened tube. The weighted average method is a method of determining an average radius or an average diameter by a weighted average operation according to an embodiment of the present invention.

As shown in fig. 15, the weighted average method according to an embodiment of the present invention determines the average error of the stenosis rate to be only 0.0210, compared to the average error 0.0837 of the stenosis rate determined by the start-stop method and the average error 0.0534 of the stenosis rate determined by the average method. Therefore, the weighted average method according to the present invention can significantly reduce the average error of the stenosis rate detection of the tubular.

Through the above description of the technical solution and the embodiments of the present invention for straightening a tubular object in an image, it can be understood by those skilled in the art that the method for straightening a tubular object in an image according to the embodiments of the present invention can determine the gray scale value and the sectional image of each coordinate point on each normal plane by constructing the normal plane coordinate system at each central point of the tubular object based on the central line, and obtain three-dimensional straightened image information of the tubular object by stacking the sectional images, so that the morphological features of the tubular object can be more comprehensively retained, and a better straightening effect can be obtained.

In some embodiments, the normal plane coordinate system is constructed by determining the rotation angle and the rotation direction between the normal planes at two adjacent central points, so that the rotation error between the normal plane coordinate systems at the adjacent central points can be effectively avoided, and the phenomenon that the straightened tubular object is radially twisted due to the rotation error can be avoided. In other embodiments, the weighted average operation of the section radii according to the distance between the central point of the minimum section radius of the section radii in the straightened image and the central points can obtain the average radius of the tubular object which is more accurate and more in line with the objective rule.

In a second aspect of the invention, there is provided an apparatus for straightening a tubular in an image, which may comprise: a centerline determination module that may be configured to determine a centerline of a tubular in an image under test; a coordinate system construction module configurable to construct a normal plane coordinate system at each center point on the centerline based on the centerline of the tubular; the gray value determining module can be configured to determine the gray value of each coordinate point on the normal plane at each central point according to the coordinate transformation between the normal plane coordinate system and the original coordinate system in the image to be detected so as to obtain a cross-sectional image at each central point; and a stacking module configured to stack the sectional images at the respective center points based on an arrangement order of the respective center points on the center line to obtain a straightened image of the tubular object.

The apparatus of the embodiment of the present invention has been described and explained in detail in the above with reference to any one of fig. 1 to 14, and will not be described again here.

In a third aspect of the invention, there is provided an apparatus for straightening a tubular in an image, comprising, at least one processor; a memory storing program instructions that, when executed by the at least one processor, cause the apparatus to perform the method according to any one of the first aspects of the invention. This will be described in an exemplary manner with reference to fig. 16.

FIG. 16 is a schematic diagram illustrating a system for straightening a tubular in an image according to an embodiment of the present invention. The system 1600 may include a device 1610 and its peripheral devices and external networks according to embodiments of the invention, where the device 1610 performs operations for straightening a tubular in an image to implement the solution of an embodiment of the invention described above in connection with any of fig. 1-14.

As shown in fig. 16, device 1610 may include a CPU1611, which may be a general purpose CPU, a special purpose CPU, or other execution unit where information processing and programs run. Further, device 1610 may also include a mass storage 1612 and a read only memory ROM 1613, where mass storage 1612 may be configured to store various types of data including centerline data, transformation matrices, and the like, as well as various programs needed to straighten tubes in an image, and ROM 1613 may be configured to store data needed to initialize the various functional modules in the system of device 1610, drivers for basic input/output of the system, and to boot the operating system.

Further, the device 1610 may also include other hardware or components, such as a graphics processor ("GPU") 1615 and a field programmable gate array ("FPGA") 1616, as shown. It is to be understood that although various hardware or components are shown in the device 1610, this is merely exemplary and not limiting, and those skilled in the art can add or remove corresponding hardware as needed.

The device 1610 of embodiments of the invention may also include a communication interface 1618 such that it may be connected via the communication interface 1618 to a local area network/wireless local area network (LAN/WLAN) 1650, which in turn may be connected via the LAN/WLAN to a local server 1660 or to the Internet ("Internet") 1670. Alternatively or additionally, the device 1610 of embodiments of the invention may also be directly connected to the internet or a cellular network over a wireless communication technology, such as a third generation ("3G"), fourth generation ("4G"), or 5 generation ("5G") based wireless communication technology, via the communication interface 1618. In some application scenarios, the device 1610 of the present embodiment may also access the server 1680 and possibly the database 1690 of the external network as needed to obtain various known attribute reference values, data and modules, etc., for example, of the target tubular, and may remotely store the various data detected.

Peripheral devices to device 1610 may include a display device 1620, an input device 1630, and a data transfer interface 1640. In one embodiment, the display device 1620 may comprise, for example, one or more speakers and/or one or more visual displays configured to provide voice prompts and/or image visual displays for the detection process or the final result of the apparatus of the present invention. Input device 1630 may include, for example, a keyboard, mouse, microphone, gesture capture camera, or other input buttons or controls configured to receive input of detection information or user instructions. The data transfer interface 1640 may include, for example, a serial interface, a parallel interface, or a universal serial bus interface ("USB"), a small computer system interface ("SCSI"), serial ATA, FireWire ("FireWire"), PCI Express, and a high-definition multimedia interface ("HDMI"), which are configured for data transfer and interaction with other devices or systems. According to the scheme of the present invention, the data transmission interface 1640 can receive an image to be tested or the like and transmit various types of data and results to the device 1610.

The CPU1611, mass storage 1612, read only memory ("ROM") 1613, GPU 1615, FPGA 1616, and communications interface 1618 of the device 1610 of embodiments of the invention described above may be interconnected by a bus 1619, and enable data interaction with peripheral devices through the bus. Through the bus 1619, the CPU1611 may control other hardware components and their peripherals within the device 1610, in one embodiment.

In operation, the processor CPU1611 of the device 1610 of embodiments of the present invention may receive data via the input device 1630 or the data transfer interface 1640, and retrieve computer program instructions or code (e.g., code related to straightening a tubular in an image) stored in the memory 1612 to detect a received image under test and its detection request to obtain a straightened image, etc. Next, the processor CPU1611 starts to perform operations of determining the center line of the tubular object, constructing a normal plane coordinate system, determining a gradation value, stacking sectional images, and the like, based on the obtained image to be measured. After the CPU1611 obtains the straightened image by executing the program for straightening the tubular object, the straightened image may be displayed on the display device 1620 or output by means of voice prompt. In addition, the device 1610 may also upload the straightened results to a network, such as a remote database 1690, via the communication interface 1618.

It should also be appreciated that any module, unit, component, server, computer, terminal, or device executing instructions of the examples of the invention may include or otherwise access a computer-readable medium, such as a storage medium, computer storage medium, or data storage device (removable) and/or non-removable) such as a magnetic disk, optical disk, or magnetic tape. Computer storage media may include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules or other data.

The computer readable storage medium may be any suitable magnetic or magneto-optical storage medium, such as resistive Random Access Memory (rram), Dynamic Random Access Memory (dram), Static Random Access Memory (SRAM), enhanced Dynamic Random Access Memory (edram), High-Bandwidth Memory (HBM), hybrid Memory cubic (hmc) Memory cube, and the like, or any other medium that can be used to store the desired information and that can be accessed by an application, module, or both. Any such computer storage media may be part of, or accessible or connectable to, a device. Any applications or modules described herein may be implemented using computer-readable/executable instructions that may be stored or otherwise maintained by such computer-readable media.

Although the embodiments of the present invention have been described above, the description is only for the convenience of understanding the present invention, and is not intended to limit the scope and application of the present invention. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

Claims

1. A method for straightening a tubular in an image, comprising:

determining the central line of a tubular object in the image to be detected;

constructing a normal plane coordinate system at each central point on the central line based on the central line of the tubular object;

determining the gray value of each coordinate point on the normal plane at each central point according to the coordinate transformation between the normal plane coordinate system and the original coordinate system in the image to be detected so as to obtain a section image at each central point; and

and stacking the sectional images at the central points based on the arrangement sequence of the central points on the central line to obtain a straightened image of the tubular object.

2. The method of claim 1, wherein determining the centerline of the tubular in the image under test comprises:

and performing skeleton extraction on the tubular object in the image to be detected to determine the central line of the tubular object.

3. The method of claim 2, wherein skeletonizing the tubular to determine the centerline comprises:

preprocessing skeleton lines obtained by skeleton extraction to generate the preprocessed center lines, wherein the preprocessing comprises at least one of the following:

removing burrs on the skeleton line;

removing the annular structure on the skeleton line; and

and the skeleton lines in the same tubular object are communicated.

4. The method of claim 1, wherein constructing a normal plane coordinate system at each center point comprises:

performing spline fitting on the center line of the tubular object for one time or multiple times to obtain a fitting center line;

determining a tangent vector for each center point based on the fitted centerlines; and

and constructing a normal plane coordinate system at each central point according to the tangent vector of each central point.

5. The method of claim 4, wherein constructing a normal plane coordinate system at each center point from the tangent vectors of each center point comprises:

selecting vectors in two directions which are orthogonal to tangent vectors of the starting point and are mutually orthogonal for the starting point of the fitting central line to construct a normal plane coordinate system at the starting point;

determining a rotation angle and a rotation direction between normal planes at two adjacent central points according to tangent vectors of the two adjacent central points; and

and determining a normal plane coordinate system at the central point which is sequenced at the back of the two adjacent central points based on the rotation angle and the rotation direction.

6. The method of claim 1, wherein determining a gray value according to the coordinate transformation between the normal plane coordinate system and the original coordinate system in the image to be measured comprises:

determining a rotation matrix and a translation matrix between the normal plane coordinate system at each central point and the original coordinate system according to the normal plane coordinate system at each central point on the central line;

determining a transformation matrix between the normal plane coordinate system at each central point and the original coordinate system according to the rotation matrix and the translation matrix; and

and performing coordinate transformation on each coordinate point on the normal plane at each central point based on the transformation matrix to determine the gray value of each coordinate point.

7. The method of claim 6, wherein determining a grayscale value based on a transformation matrix comprises:

according to the radius of the tubular object at each central point, constructing a first coordinate matrix of each coordinate point on a normal plane at each central point;

determining a second coordinate matrix of each coordinate point under the original coordinate system according to the first coordinate matrix and the transformation matrix; and

and calculating the gray value of each coordinate point on the normal plane at each central point by using an image interpolation algorithm based on the second coordinate matrix.

8. The method of claim 1, further comprising:

determining morphological features of the tubular based on the straightened image, wherein the morphological features include at least one of:

a curvature;

a length;

an average diameter;

an average radius;

a cross-sectional radius;

a cross-sectional diameter;

the rate of stenosis.

9. The method of claim 8, wherein determining the morphological feature of the tubular comprises at least one of:

determining a section radius or a section diameter of the tubular object at each central point based on the section images at each central point of the straightened tubular object in the straightened images;

determining an average radius or average diameter of the tubular based on the cross-sectional radius or cross-sectional diameter at each center point; and

determining a stenosis rate of the tubular based on a minimum section radius of the section diameters and the average radius, or based on a minimum section diameter of the section diameters and the average diameter.

10. The method of claim 9, wherein determining a cross-sectional radius or diameter at each center point comprises:

setting a plurality of intersection lines on each sectional image with the central point in the sectional image as an intersection point;

detecting the intersection point of each cross line and the edge of the tubular object in the cross-sectional image to determine sampling points in the cross-sectional image;

and determining the section radius or the section diameter at the central point according to the average value of the distance between each sampling point and the intersection point.

11. The method of claim 10, wherein determining sample points in the cross-sectional image comprises:

dividing each cross line into two cross sub-lines at the cross point; and

and determining the intersection point which is closest to the intersection point on each cross sub-line as the sampling point.

12. The method of claim 8 or 9, wherein determining the morphological feature of the tubular comprises:

performing weighted average operation on each section radius according to the distance between the central point of the minimum section radius in each section radius in the straightened image and each central point to determine the average radius; or

And performing weighted average operation on each section diameter according to the distance between the central point of the minimum section diameter in each section diameter in the straightened image and each central point to determine the average diameter.

13. The method of any one of claims 1-11, wherein the tubular object comprises a cerebral arterial vessel.

14. An apparatus for straightening a tubular in an image, comprising:

a central line determining module configured to determine a central line of a tubular object in the image to be measured;

a coordinate system construction module configured to construct a normal plane coordinate system at each center point on the centerline based on the centerline of the tubular;

the gray value determining module is configured to determine the gray value of each coordinate point on the normal plane at each central point according to the coordinate transformation between the normal plane coordinate system and the original coordinate system in the image to be detected so as to obtain a cross-sectional image at each central point; and

a stacking module configured to stack the sectional images at the respective center points based on an arrangement order of the respective center points on the center line to obtain a straightened image of the tubular object.

15. An apparatus for straightening a tubular in an image, comprising,

at least one processor;

a memory storing program instructions that, when executed by the at least one processor, cause the apparatus to perform the method of any of claims 1-13.

16. A computer-readable storage medium, characterized in that it stores a program for straightening a tubular in an image, which program, when executed by a processor, performs the method according to any one of claims 1-13.