CN116802686A - ROI (region of interest) region association and labeling method - Google Patents
ROI (region of interest) region association and labeling method Download PDFInfo
- Publication number
- CN116802686A CN116802686A CN202280007558.5A CN202280007558A CN116802686A CN 116802686 A CN116802686 A CN 116802686A CN 202280007558 A CN202280007558 A CN 202280007558A CN 116802686 A CN116802686 A CN 116802686A
- Authority
- CN
- China
- Prior art keywords
- point set
- surface contour
- contour point
- reference surface
- roi
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000002372 labelling Methods 0.000 title claims abstract description 14
- 230000009466 transformation Effects 0.000 claims abstract description 183
- 239000011159 matrix material Substances 0.000 claims abstract description 107
- 238000004364 calculation method Methods 0.000 claims abstract description 69
- 238000001514 detection method Methods 0.000 claims abstract description 35
- 230000001131 transforming effect Effects 0.000 claims abstract description 25
- 230000005012 migration Effects 0.000 claims abstract description 12
- 238000013508 migration Methods 0.000 claims abstract description 12
- 238000000034 method Methods 0.000 claims description 53
- 230000008569 process Effects 0.000 claims description 26
- 239000013598 vector Substances 0.000 claims description 8
- 238000001914 filtration Methods 0.000 claims description 3
- 238000001959 radiotherapy Methods 0.000 description 9
- 206010028980 Neoplasm Diseases 0.000 description 5
- 238000006073 displacement reaction Methods 0.000 description 5
- 238000012544 monitoring process Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000013507 mapping Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 239000012535 impurity Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000029058 respiratory gaseous exchange Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The application discloses a ROI (region of interest) region association and labeling method, which comprises the following steps: acquiring a real-time surface contour point set of a currently scanned detection object; acquiring a reference surface profile point set of a detection object, wherein the reference surface profile point set is a surface profile point set obtained by scanning the detection object with a positioning standard; performing pose association calculation on the reference surface contour point set and the real-time surface contour point set to obtain an association transformation matrix; transforming the real-time surface contour point set according to the association transformation matrix to generate a transformed contour point set; determining a first ROI (region of interest) of a reference surface contour point set, and performing migration transformation on the first ROI to a transformation contour point set to generate a second ROI; and carrying out inverse transformation on the transformation contour point set containing the second ROI area according to the association transformation matrix to generate a target contour point set. According to the application, the automatic labeling of the ROI region on the real-time surface contour point set can be realized according to the reference surface contour point set and the first ROI region.
Description
Technical Field
The application relates to the technical field of images, in particular to a ROI (region of interest) region association and labeling method.
Background
In recent years, various radiotherapy devices are widely applied to diagnosis and treatment of various complex and changeable diseases, and in the radiotherapy process, accurate monitoring of the conditions of a diseased target area of a patient has a great influence on accurate treatment of tumors.
For example, in the current tumor radiotherapy means, a doctor needs to manually delineate and label a tumor area or other critical areas by using a surface contour point set obtained by scanning during each treatment, and then radiotherapy equipment performs radiotherapy on the delineated and labeled tumor area or other critical areas. On the one hand, the existing method can only achieve drawing of a frame of scanned picture, and in fact, the real-time surface contour point set is updated continuously along with continuous scanning of a camera, so that a human body generates some tiny displacement due to respiration and other conditions in the radiotherapy process, the existing method does not consider the situation, and meanwhile, the existing method cannot achieve real-time tracking and identification of a tumor area or other key areas, so that the situation that a diseased area cannot be monitored in real time is caused.
Therefore, how to quickly and automatically determine the affected area of the body surface of the person in real time is a technical problem to be solved.
Disclosure of Invention
In view of the above, the application provides a ROI region association and labeling method, which can realize association identification and automatic labeling of the ROI region on a real-time surface contour point set.
In order to achieve the above object, the following solutions have been proposed:
a ROI area association and labeling method, comprising:
acquiring a real-time surface contour point set of a currently scanned detection object;
acquiring a reference surface profile point set of the detection object, wherein the reference surface profile point set is a surface profile point set obtained by scanning the detection object with a positioning standard;
performing pose association calculation on the reference surface contour point set and the real-time surface contour point set to obtain an association transformation matrix;
transforming the real-time surface contour point set according to the association transformation matrix to generate a transformed contour point set;
determining a first ROI (region of interest) of the reference surface contour point set, and performing migration transformation on the first ROI to the transformation contour point set to generate a second ROI, wherein the first ROI is an ROI marked on the reference surface contour point set;
and carrying out inverse transformation on the transformation contour point set containing the second ROI according to the association transformation matrix to generate a target contour point set, wherein the target contour point set contains a third ROI obtained by carrying out inverse transformation on the second ROI.
Preferably, determining a first ROI area of the reference surface contour point set, and migration transforming the first ROI area onto the transformed contour point set, generating a second ROI area, includes:
acquiring coordinate positions of feature points in a first ROI (region of interest) of the reference surface contour point set, wherein the first ROI is an ROI marked on the reference surface contour point set;
determining feature points with the same coordinate positions as the feature points in the first ROI area in the transformation contour point set;
and determining a second ROI area according to the feature points which are the same as the coordinate positions of the feature points in the first ROI area.
Preferably, determining the first ROI area of the set of reference surface contour points comprises:
responding to the manual operation, and acquiring a sketched graph area and a point set parameter of the reference surface profile point set;
determining a view volume space transformation matrix corresponding to the point set parameters;
and performing view volume space transformation between the graph area and the reference surface contour point set according to the view volume space transformation matrix, and generating a first ROI area corresponding to the graph area on the reference surface contour point set according to a transformation result.
Preferably, performing pose association calculation on the reference surface contour point set and the real-time surface contour point set to obtain an association transformation matrix, including:
performing first pose association calculation on the reference surface contour point set and the real-time surface contour point set to generate an association transformation matrix;
on the basis of the last pose association calculation, carrying out second pose association calculation on the reference surface contour point set and the real-time surface contour point set, and updating the association transformation matrix;
and if the current pose correlation calculation result is not in the allowable difference range, returning to the process of performing second pose correlation calculation on the reference surface contour point set and the real-time surface contour point set on the basis of the last pose correlation calculation until the current pose correlation calculation result is in the allowable difference range.
Preferably, performing a first pose correlation calculation on the reference surface contour point set and the real-time surface contour point set, and generating a correlation transformation matrix, including:
respectively filtering and denoising the reference surface profile point set and the real-time surface profile point set;
determining normal vectors of each characteristic point constituting the reference surface profile point set and each characteristic point constituting the real-time surface profile point set;
calculating and generating a feature point geometric description set of the reference surface contour point set and the real-time surface contour point set according to the normal vector;
and determining an association transformation matrix for transforming the real-time surface contour point set to be calculated in association with the pose of the reference surface contour point set based on the feature point geometric description set.
Preferably, performing a view volume spatial transformation between the graphic region and the reference surface contour point set according to the view volume spatial transformation matrix, and generating a first ROI region corresponding to the graphic region on the reference surface contour point set according to a transformation result, including:
determining 2D coordinates of each feature point in the graphic region;
according to the view volume space transformation matrix, determining 2D coordinates corresponding to each characteristic point of the reference surface contour point set;
determining feature points with the 2D coordinates corresponding to the reference surface contour point set being the same as the 2D coordinates of the feature points in the graph area as target feature points;
and determining the region where the target feature point is located on the reference surface contour point set as a first ROI region.
Preferably, performing a view volume spatial transformation between the graphic region and the reference surface contour point set according to the view volume spatial transformation matrix, and generating a first ROI region corresponding to the graphic region on the reference surface contour point set according to a transformation result, including:
determining 2D coordinates of each feature point in the graphic region;
determining 3D coordinates corresponding to the 2D coordinates of each feature point in the graphic area according to the inverse matrix of the view volume space transformation matrix;
determining feature points with the same coordinates as the 3D coordinates corresponding to each feature point in the graph area on the reference surface contour point set as target feature points;
and determining the region where the target feature point is located on the reference surface contour point set as a first ROI region.
Preferably, before transforming the real-time surface contour point set according to the association transformation matrix to generate a transformed contour point set, the method further includes:
detecting whether the pose correlation calculation result is within an allowable difference range;
if yes, executing a process of transforming the real-time surface contour point set according to the association transformation matrix to generate a transformed contour point set;
if not, displaying prompt information to prompt the detection object to adjust the positioning.
Preferably, after transforming the real-time surface contour point set according to the association transformation matrix to generate a transformed contour point set, the method further includes:
and determining a unique corresponding relation between each characteristic point forming the transformation contour point set and each characteristic point forming the reference surface contour point set, and setting the same index number for each group of corresponding characteristic points.
According to the technical scheme, the ROI region association and labeling method provided by the application is characterized in that the real-time surface contour point set of the currently scanned detection object and the reference surface contour point set of the detection object are obtained, and the reference surface contour point set is the surface contour point set obtained by scanning the detection object with the positioning standard. And performing pose association calculation on the reference surface contour point set and the real-time surface contour point set to obtain an association transformation matrix, and transforming the real-time surface contour point set according to the association transformation matrix to generate a transformation contour point set. Determining a first ROI (region of interest) of the reference surface contour point set, and performing migration transformation on the first ROI to the transformation contour point set to generate a second ROI, wherein the first ROI is an ROI marked on the reference surface contour point set. And finally, carrying out inverse transformation on the transformation contour point set containing the second ROI according to the association transformation matrix to generate a target contour point set, wherein the target contour point set contains a third ROI obtained by carrying out inverse transformation on the second ROI.
The method comprises the steps of performing migration transformation on a first ROI region marked on a reference surface contour point set to a transformation contour point set obtained after pose correlation calculation transformation through a pose correlation calculation detection object reference surface contour point set and a real-time surface contour point set to obtain a transformation contour point set containing a second ROI region, and performing inverse transformation on the transformation contour point set to obtain a target contour point set containing a third ROI region. Since the transformation process transforms through the associated transformation matrix and the inverse transformation process transforms through the inverse matrix of the associated transformation matrix, the set of target contour points is identical to the set of real-time surface contour points except for the third ROI region containing the labels. According to the application, the ROI area can be automatically identified and marked on the same area on the scanned real-time surface contour point set according to the reference surface contour point set containing the first ROI area, so that the tracking monitoring and treatment of the ROI area on the real-time surface contour point set can be realized in the radiotherapy process.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only embodiments of the present application, and that other drawings can be obtained according to the provided drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a method for associating and labeling ROI areas disclosed in the present application;
FIG. 2 is a schematic diagram of a reference surface contour point set and a target contour point set according to an embodiment of the present application;
fig. 3 is a schematic diagram illustrating an ROI area according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
Referring to fig. 1, fig. 1 is a flowchart of an ROI area associating and labeling method according to an embodiment of the present application.
As shown in fig. 1, the method may include:
and S1, acquiring a real-time surface contour point set of the currently scanned detection object.
Specifically, the real-time surface contour point set is composed of a plurality of feature points. After the detection object enters the scanning instrument, the detection object can be scanned by various depth scanning instruments, and a real-time surface profile point set is constructed by modeling software such as 3dsMAX, maya and the like. The application does not limit the specific scanning equipment, and the equipment capable of scanning the detection object and generating the corresponding real-time surface contour point set belongs to the scanning equipment which can be used by the application.
And S2, acquiring a reference surface contour point set of the detection object.
Specifically, the reference surface contour point set is a surface contour point set obtained by scanning a detection object with a positioning standard, wherein the reference surface contour point set is composed of a plurality of characteristic points. At this time, the posture, position and the like of the detection object meet the detection and scanning requirements. For example, the reference surface contour point set may be a surface contour point set obtained by scanning a human body of the positioning standard.
In practical application, after the instrument scans the detection object with the positioning standard, the reference surface contour point set of the detection object can be obtained and stored, and in the subsequent determination process of the ROI area, the corresponding reference surface contour point set of the detection object can be directly called out according to the identity information, the ID information, the code and the like of the detection object, and the subsequent determination of the ROI area can be performed.
And S3, performing pose association calculation on the reference surface contour point set and the real-time surface contour point set to obtain an association transformation matrix.
Specifically, the reference surface contour point set and the real-time surface contour point set perform global pose correlation calculation, the pose correlation calculation mode of the reference surface contour point set and the real-time surface contour point set can be realized through a first pose correlation calculation mode, a second pose correlation calculation mode and the like, and a correlation transformation matrix is generated after the pose correlation calculation is completed. The associative transformation matrix may be used to transform the set of real-time surface contour points.
And S4, transforming the real-time surface contour point set according to the association transformation matrix to generate a transformed contour point set.
Specifically, the real-time surface contour point set carries out matrix operation on the real-time surface contour point set according to an associated transformation matrix obtained by pose associated calculation, the transformation contour point set is obtained after transformation such as rotation, displacement and stretching, the transformation contour point set and the reference surface contour point set basically keep consistent in all aspects such as position and positioning, and the transformation contour point set obtained by transformation can be considered to be basically close to and coincident with the reference surface contour point set.
And S5, determining a first ROI (region of interest) of the reference surface contour point set, and performing migration transformation on the first ROI to the transformation contour point set to generate a second ROI.
Specifically, the first ROI area is an ROI area marked on the reference surface contour point set. The first ROI area is an ROI area marked on the reference surface contour point set in advance, and can also be an ROI area marked by instant sketching at the current moment. After determining the first ROI area on the reference surface contour point set, the first ROI area may be shifted onto the transformed contour point set to generate the second ROI area, since the transformed contour point set substantially coincides closely with the reference surface contour point set. The method for generating the second ROI through the first ROI area migration transformation can include mapping the first ROI area onto a transformation contour point set to determine the second ROI area of the transformation contour point set, wherein the position, the size and the like of the second ROI area on the transformation contour point set are the same as the position, the size and the like of the first ROI area on a reference surface contour point set.
And S6, carrying out inverse transformation on the transformation contour point set containing the second ROI area according to the association transformation matrix to generate a target contour point set.
Specifically, the target contour point set includes a third ROI area obtained by inverse transforming the second ROI area. And (3) calculating an inverse matrix of the association transformation matrix, and inversely transforming the transformation contour point set comprising the second ROI area through the inverse matrix of the association transformation matrix, namely restoring the real-time surface contour point set subjected to rotation, displacement and stretching in the step S4 to a form before rotation, displacement and stretching, so as to generate a target contour point set comprising a third ROI area obtained by inversely transforming the second ROI area.
In practical application, after the reference surface contour point set marked with the first ROI area and the real-time surface contour point set are obtained, the target contour point set containing the corresponding third ROI area can be obtained according to the method. As shown in fig. 2, the position, the size, etc. of the second ROI area on the transformation contour point set are the same as the position, the size, etc. of the first ROI area on the reference surface contour point set, and the third ROI area is obtained by transforming the second ROI area, and the area represented on the target surface contour point set is the area represented on the reference surface contour point set of the first ROI area, and is the affected area requiring radiotherapy.
According to the technical scheme, the ROI region association and labeling method provided by the application is characterized in that the real-time surface contour point set of the currently scanned detection object and the reference surface contour point set of the detection object are obtained, and the reference surface contour point set is the surface contour point set obtained by scanning the detection object with the positioning standard. And performing pose association calculation on the reference surface contour point set and the real-time surface contour point set to obtain an association transformation matrix, and transforming the real-time surface contour point set according to the association transformation matrix to generate a transformation contour point set. Determining a first ROI (region of interest) of the reference surface contour point set, and performing migration transformation on the first ROI to the transformation contour point set to generate a second ROI, wherein the first ROI is an ROI marked on the reference surface contour point set. And finally, carrying out inverse transformation on the transformation contour point set containing the second ROI according to the association transformation matrix to generate a target contour point set, wherein the target contour point set contains a third ROI obtained by carrying out inverse transformation on the second ROI.
The method comprises the steps of performing migration transformation on a first ROI region marked on a reference surface contour point set to a transformation contour point set obtained after pose correlation calculation transformation through a pose correlation calculation detection object reference surface contour point set and a real-time surface contour point set to obtain a transformation contour point set containing a second ROI region, and performing inverse transformation on the transformation contour point set to obtain a target contour point set containing a third ROI region. Since the transformation process transforms through the associated transformation matrix and the inverse transformation process transforms through the inverse matrix of the associated transformation matrix, the set of target contour points is identical to the set of real-time surface contour points except for the third ROI region containing the labels. According to the application, the ROI area can be automatically identified and marked on the same area on the scanned real-time surface contour point set according to the reference surface contour point set containing the first ROI area, so that the tracking monitoring and treatment of the ROI area on the real-time surface contour point set can be realized in the radiotherapy process.
Optionally, in step S4, transforming the real-time surface contour point set according to the association transformation matrix, after generating a transformed contour point set, the method may further include:
step S7, determining the unique corresponding relation between each characteristic point forming the transformation contour point set and each characteristic point forming the reference surface contour point set, and setting the same index number for each group of corresponding characteristic points.
Specifically, a unique correspondence between each feature point constituting the transformation contour point set and each feature point constituting the reference surface contour point set is determined, and for each feature point constituting the transformation contour point set, there is a feature point constituting the reference surface contour point set that corresponds uniquely to each feature point, so that in order to avoid performing the task of re-pose correlation calculation, each group of corresponding feature points may be set to the same index number, and the index numbers may be kept unrepeated. According to the index sequence number, the set of mutually corresponding feature points can be directly searched.
In practical application, if the ROI area is required to be increased, each characteristic point on the transformation contour point set with the same index sequence number can be found only by determining each characteristic point in the newly increased ROI area on the reference surface contour point set, so that the corresponding second ROI area is rapidly determined, and the previous pose association calculation and transformation process is not required to be repeated.
Optionally, considering that in practical application, the human body may be displaced, to avoid the damage caused by ineffective irradiation due to the displacement of the human body, whether the object to be detected needs to be adjusted for positioning may be further determined according to the association transformation matrix.
In step S4, before transforming the real-time surface contour point set according to the association transformation matrix to generate a transformed contour point set, the method may further include:
s8, detecting whether the pose correlation calculation result is within an allowable difference range;
if yes, executing a step S4, and transforming the real-time surface contour point set according to the association transformation matrix to generate a transformation contour point set;
if not, displaying prompt information to prompt the detection object to adjust the positioning.
Specifically, whether the line treatment is emitted is judged according to the detected condition, if the detected pose correlation calculation result is within the allowable difference range, the subsequent ROI region determination and marking work is continued, if the pose correlation calculation result is not within the allowable difference range if the amplitude of the pose correlation change of a certain detection object in the middle is larger in real-time monitoring, prompt information is displayed to prompt the detection object to adjust the positioning, rays are not emitted, the next monitored detection object positioning is restored to the set threshold range, namely, the pose correlation calculation result is within the allowable difference range, the ROI region determination and marking work is restored through linkage control, and the ray treatment on the ROI region is performed.
In practical applications, the first ROI area on the reference surface contour point set may be a ROI area that is labeled in advance, or may be a ROI area labeled at the current time. In view of the two different ways described above, two different embodiments are provided.
In some embodiments of the present application, the process of determining the first ROI area of the reference surface contour point set in step S5 and transferring the first ROI area migration to the transferred contour point set to generate the second ROI area may specifically include:
the second, reference cell model has been labeled with the first ROI area.
Step S51, acquiring coordinate positions of feature points in a first ROI (region of interest) of the reference surface contour point set, wherein the first ROI is an ROI marked on the reference surface contour point set.
Specifically, the reference surface contour point set contains a first ROI area which is already marked, and at this time, coordinate positions of all feature points in the first ROI area on the reference surface contour point set are obtained.
Step S52, determining the feature points with the same coordinate positions as the feature points in the first ROI area in the transformed contour point set.
Specifically, each feature point in the transformed contour point set, which is close to and coincident with the reference surface contour point set, is determined with the same coordinate position as each feature point in the transformed contour point set. It is also possible to determine, by means of mapping, that each feature point in the first ROI area is mapped on the set of transformed contour points, feature points having the same coordinate positions as the feature points in the first ROI area.
Step S53, determining a second ROI area according to the feature points with the same coordinate positions as the feature points in the first ROI area.
Specifically, after the feature points with the same coordinate positions as the feature points in the first ROI area are determined, the area surrounded by the feature points is the second ROI area.
The second, real-time, labeling of the first ROI area on the reference cell model.
The procedure of determining the first ROI area of the reference surface contour point set in step S5 may specifically include:
and step S54, responding to the manual operation, and acquiring the delineated graph area and the point set parameters of the reference surface contour point set.
Specifically, as shown in fig. 3, the reference surface contour point set may be labeled with a first ROI area at the current time. For example, the user may determine the ROI area to be generated by rotating the reference surface contour point set, by mouse pointing, or the like. And acquiring a graph area of a closed shape formed by manual sketching, and determining the graph area and the point set parameters of the reference surface contour point set.
Step S55, determining a view volume space transformation matrix corresponding to the point set parameters.
In particular, the point set parameters may be used to obtain a view volume space transformation matrix that matches the model. The view volume space transformation matrix is used for view volume space transformation and realizes the switching between the three-dimensional space and the two-dimensional plane. The view volume spatial transformation process sequentially comprises model matrix transformation, view matrix transformation and projection matrix transformation, and comprises the following steps:
1) Model matrix transformation: calculating the positions of the points of the model in world space so as to place the model in the world space;
2) View matrix transformation: calculating the relative position of the object to the camera;
3) Projection matrix transformation: is a perspective projective transformation that converts a three-dimensional object into a two-dimensional image that can be displayed on a screen.
And step S56, performing visual space transformation between the graph area and the reference surface contour point set according to the visual space transformation matrix, and generating a first ROI area corresponding to the graph area on the reference surface contour point set according to a transformation result.
Specifically, through the view volume space transformation matrix, view volume space transformation between the graph area and the reference surface contour point set can be realized, namely, characteristic points corresponding to the graph area on the reference surface contour point set can be determined through characteristic points of the graph area, so that a first ROI area corresponding to the graph area on the reference surface contour point set can be determined, or the characteristic points in the reference surface contour point set can be transformed, the characteristic points which fall into the graph area after transformation can be determined, and the area formed by the reference surface contour point set is determined as the first ROI area corresponding to the graph area.
In some embodiments of the present application, two alternative ways of determining the first ROI area according to the view volume space transformation matrix are provided, that is, step S56, performing view volume space transformation between the graphic area and the reference surface contour point set according to the view volume space transformation matrix, and generating two different implementations of the first ROI area procedure corresponding to the graphic area on the reference surface contour point set according to the transformation result. The following two ways are respectively described, which specifically may include:
a first kind of,
(1) And determining the 2D coordinates of each characteristic point in the graph area.
(2) And determining 2D coordinates corresponding to each characteristic point of the reference surface contour point set according to the view volume space transformation matrix.
(3) And determining the feature points with the 2D coordinates corresponding to the reference surface contour point set and the same as the 2D coordinates of the feature points in the graph area as target feature points.
(4) And determining the region where the target feature point is located on the reference surface contour point set as a first ROI region.
Specifically, according to the point set parameters, a view volume space transformation matrix matched with the point set parameters can be determined. After determining the 3D coordinates of each feature point of the reference surface contour point set, determining the 2D coordinates corresponding to each feature point of the reference surface contour point set according to the view volume space transformation matrix. And determining 2D coordinates of each feature point in the graph area, comparing the 2D coordinates corresponding to each feature point of the reference surface contour point set with the 2D coordinates of each feature point in the graph area, determining the feature point with the 2D coordinates corresponding to the reference surface contour point set being the same as the 2D coordinates of the feature point in the graph area as a target feature point, namely determining the feature point of the reference surface contour point set falling into the graph area after conversion as the target feature point. And finally determining the region or the enclosed region where the target feature points are located on the reference surface contour point set as a first ROI region.
A second kind of,
(1) And determining the 2D coordinates of each characteristic point in the graph area.
(2) And determining 3D coordinates corresponding to the 2D coordinates of each characteristic point in the graph area according to the inverse matrix of the view volume space transformation matrix.
(3) And determining the feature points with the same coordinates as the 3D coordinates corresponding to the feature points in the graph area on the reference surface contour point set as target feature points.
(4) And determining the region where the target feature point is located on the reference surface contour point set as a first ROI region.
Specifically, according to the point set parameters, a view volume space transformation matrix matched with the point set parameters can be determined, and the view volume space transformation matrix can be converted to obtain an inverse matrix of the view volume space transformation matrix. After determining the 2D coordinates of each feature point in the graphics area, determining the 3D coordinates corresponding to the 2D coordinates of each feature point in the graphics area according to the inverse matrix of the view volume space transformation matrix. And determining 3D coordinates corresponding to each feature point in the reference surface contour point set, comparing the 3D coordinates corresponding to each feature point in the graph area with the 3D coordinates corresponding to each feature point in the reference surface contour point set, determining the feature point with the same coordinates on the reference surface contour point set as the 3D coordinates corresponding to each feature point in the graph area as a target feature point, namely mapping each feature point in the graph area to the corresponding feature point on the reference surface contour point set, and determining the feature point as the target feature point. And finally determining the region or the enclosed region where the target feature points are located on the reference surface contour point set as a first ROI region.
In some embodiments of the present application, in order to accelerate the pose correlation calculation speed as much as possible, improve the pose correlation calculation efficiency, and not affect the pose correlation calculation precision, the present embodiment provides a pose correlation calculation mode combining the first pose correlation calculation and the second pose correlation calculation.
Introducing a process of performing pose correlation calculation on the reference surface contour point set and the real-time surface contour point set to obtain a correlation transformation matrix in step S3, wherein the process specifically comprises the following steps:
and S31, performing first pose association calculation on the reference surface contour point set and the real-time surface contour point set to generate an association transformation matrix.
And step S32, performing second pose association calculation on the reference surface contour point set and the real-time surface contour point set on the basis of the last pose association calculation, and updating the association transformation matrix.
And step S33, if the current pose correlation calculation result is detected to be not in the allowable difference range, returning to the process of performing second pose correlation calculation on the reference surface contour point set and the real-time surface contour point set on the basis of the last pose correlation calculation until the current pose correlation calculation result is in the allowable difference range.
Specifically, first pose association calculation is performed on the reference surface contour point set and the real-time surface contour point set, an association transformation matrix is generated at the same time, second pose association calculation is performed on the reference surface contour point set and the real-time surface contour point set on the basis of the completion of the first pose association calculation, and meanwhile, the association transformation matrix is updated. And detecting whether a current pose correlation calculation result obtained after the second pose correlation calculation is completed is within an allowable difference range, if so, determining that the pose correlation calculation is completed, and if not, carrying out second pose correlation calculation on the reference surface contour point set and the real-time surface contour point set again on the basis of the second pose correlation calculation, and updating the correlation transformation matrix again until the current pose correlation calculation result is within the allowable difference range.
The following provides an optional implementation process of the first pose correlation calculation, and introduces a process of generating a correlation transformation matrix by performing the first pose correlation calculation on the reference surface contour point set and the real-time surface contour point set in step S31, which specifically may include:
(1) and respectively filtering and denoising the real-time surface profile point set and the reference surface profile point set.
Specifically, the real-time surface contour point set and the reference surface contour point set are respectively filtered and denoised, so that the influence of impurity factors on the pose correlation calculation process is avoided.
(2) And determining normal vectors of each first characteristic point in the reference surface profile point set and each second characteristic point in the real-time surface profile point set.
(3) And calculating and generating a characteristic point geometric description set of the real-time surface contour point set and the reference surface contour point set according to the normal vector.
Specifically, the feature point geometric description set describes geometric attributes in k adjacent domains by parameterizing the space difference between the query point and the adjacent domains and forming a multi-dimensional histogram. The step of generating the feature point geometry description set comprises:
1) Presetting a three-dimensional point set P { P } 1 ,p 2 ,...p k Point P as point set P n ,n∈[1,k]The algorithm vector is calculated.
2) At p i For the center point, determining a k neighborhood of a field radius r, and calculating p i And feature triplets (alpha, phi, theta) between each point in its k neighborhood, and then statistically obtaining a feature descriptor SP (p) i ). Wherein:
α＝v·n j
θ＝arctan(w·n j ,u·n j )
p i ,p j two three-dimensional coordinate points, and the normal vectors of the two three-dimensional coordinate points are respectively n i ,n j
3) And respectively determining a k neighborhood for each point in the k neighborhood, and respectively calculating to obtain the feature descriptors of each point according to the mode of the last step.
4) Carrying out weighted statistics on each descriptor in the neighborhood, wherein the formula is as follows;
wherein w is j For p as a point pair i ,p j Weight is through p i ,p j The distance in space.
(4) And determining an association transformation matrix when the real-time surface contour point set is transformed to be associated with the reference surface contour point set in a pose based on the feature point geometric description set.
In addition, the second pose correlation calculation method adopted in step S32 of the present application may include the following steps:
1) And determining a first characteristic point closest to each second characteristic point in the real-time surface profile point set in the reference surface profile point set, wherein the closest point is used as a matching point of the second characteristic points.
2) And calculating the matching error among the minimized matching points according to the following formula to obtain the pose transformation matrix.
t * ＝p-Rp'
Wherein p is a reference surface profile point set, p' is a real-time surface profile point set, R is a rotation matrix, t is a translation matrix, and R and t form an updated association transformation matrix.
3) And transforming the real-time surface profile point set according to the association transformation matrix obtained in the previous step to obtain a transformed real-time surface profile point set.
4) Repeating the process of calculating the pose transformation matrix and transforming the real-time surface contour point set according to the pose transformation matrix until the matching error among minimized matching points is smaller than a set threshold value, and determining the update of the association transformation matrix based on the finally generated pose transformation matrix.
Finally, it is further noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
In the present specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, and identical and similar parts between the embodiments are all enough to refer to each other.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (9)
1. A method for associating and labeling a region of interest (ROI), comprising:
acquiring a real-time surface contour point set of a currently scanned detection object;
acquiring a reference surface profile point set of the detection object, wherein the reference surface profile point set is a surface profile point set obtained by scanning the detection object with a positioning standard;
performing pose association calculation on the reference surface contour point set and the real-time surface contour point set to obtain an association transformation matrix;
transforming the real-time surface contour point set according to the association transformation matrix to generate a transformed contour point set;
determining a first ROI (region of interest) of the reference surface contour point set, and performing migration transformation on the first ROI to the transformation contour point set to generate a second ROI, wherein the first ROI is an ROI marked on the reference surface contour point set;
and carrying out inverse transformation on the transformation contour point set containing the second ROI according to the association transformation matrix to generate a target contour point set, wherein the target contour point set contains a third ROI obtained by carrying out inverse transformation on the second ROI.
2. The method of claim 1, wherein determining a first ROI region of the set of reference surface contour points and transforming the first ROI region migration onto the set of transformed contour points, generating a second ROI region, comprises:
acquiring coordinate positions of feature points in a first ROI (region of interest) of the reference surface contour point set, wherein the first ROI is an ROI marked on the reference surface contour point set;
determining feature points with the same coordinate positions as the feature points in the first ROI area in the transformation contour point set;
and determining a second ROI area according to the feature points which are the same as the coordinate positions of the feature points in the first ROI area.
3. The method of claim 1, wherein determining a first ROI region of the set of reference surface contour points comprises:
responding to the manual operation, and acquiring a sketched graph area and a point set parameter of the reference surface profile point set;
determining a view volume space transformation matrix corresponding to the point set parameters;
and performing view volume space transformation between the graph area and the reference surface contour point set according to the view volume space transformation matrix, and generating a first ROI area corresponding to the graph area on the reference surface contour point set according to a transformation result.
4. The method of claim 1, wherein performing pose correlation calculations on the set of reference surface contour points and the set of real-time surface contour points to obtain a correlation transformation matrix comprises:
performing first pose association calculation on the reference surface contour point set and the real-time surface contour point set to generate an association transformation matrix;
on the basis of the last pose association calculation, carrying out second pose association calculation on the reference surface contour point set and the real-time surface contour point set, and updating the association transformation matrix;
and if the current pose correlation calculation result is not in the allowable difference range, returning to the process of performing second pose correlation calculation on the reference surface contour point set and the real-time surface contour point set on the basis of the last pose correlation calculation until the current pose correlation calculation result is in the allowable difference range.
5. The method of claim 4, wherein performing a first pose correlation calculation on the set of reference surface contour points and the set of real-time surface contour points to generate a correlation transformation matrix comprises:
respectively filtering and denoising the reference surface profile point set and the real-time surface profile point set;
determining normal vectors of each characteristic point constituting the reference surface profile point set and each characteristic point constituting the real-time surface profile point set;
calculating and generating a feature point geometric description set of the reference surface contour point set and the real-time surface contour point set according to the normal vector;
and determining an association transformation matrix for transforming the real-time surface contour point set to be calculated in association with the pose of the reference surface contour point set based on the feature point geometric description set.
6. A method according to claim 3, wherein performing a view volume spatial transformation between the graphical region and the set of reference surface contour points according to the view volume spatial transformation matrix and generating a first ROI area corresponding to the graphical region on the set of reference surface contour points according to the transformation result comprises:
determining 2D coordinates of each feature point in the graphic region;
according to the view volume space transformation matrix, determining 2D coordinates corresponding to each characteristic point of the reference surface contour point set;
determining feature points with the 2D coordinates corresponding to the reference surface contour point set being the same as the 2D coordinates of the feature points in the graph area as target feature points;
and determining the region where the target feature point is located on the reference surface contour point set as a first ROI region.
7. A method according to claim 3, wherein performing a view volume spatial transformation between the graphical region and the set of reference surface contour points according to the view volume spatial transformation matrix and generating a first ROI area corresponding to the graphical region on the set of reference surface contour points according to the transformation result comprises:
determining 2D coordinates of each feature point in the graphic region;
determining 3D coordinates corresponding to the 2D coordinates of each feature point in the graphic area according to the inverse matrix of the view volume space transformation matrix;
determining feature points with the same coordinates as the 3D coordinates corresponding to each feature point in the graph area on the reference surface contour point set as target feature points;
and determining the region where the target feature point is located on the reference surface contour point set as a first ROI region.
8. The method of claim 1, further comprising, prior to transforming the set of real-time surface contour points according to the associated transformation matrix to generate a set of transformed contour points:
detecting whether the pose correlation calculation result is within an allowable difference range;
if yes, executing a process of transforming the real-time surface contour point set according to the association transformation matrix to generate a transformed contour point set;
if not, displaying prompt information to prompt the detection object to adjust the positioning.
9. The method according to any of claims 1-8, further comprising, after transforming the set of real-time surface contour points according to the association transformation matrix, generating a set of transformed contour points:
and determining a unique corresponding relation between each characteristic point forming the transformation contour point set and each characteristic point forming the reference surface contour point set, and setting the same index number for each group of corresponding characteristic points.
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/CN2022/088879 WO2023205984A1 (en) | 2022-04-25 | 2022-04-25 | Roi association and labeling method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116802686A true CN116802686A (en) | 2023-09-22 |
Family
ID=88034888
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202280007558.5A Pending CN116802686A (en) | 2022-04-25 | 2022-04-25 | ROI (region of interest) region association and labeling method |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN116802686A (en) |
| WO (1) | WO2023205984A1 (en) |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101887525A (en) * | 2010-07-09 | 2010-11-17 | 北京师范大学 | Grading-based positive and inverse reversing three-dimensional dense point set rapid registering method |
| WO2012069965A1 (en) * | 2010-11-23 | 2012-05-31 | Koninklijke Philips Electronics N.V. | Interactive deformation map corrections |
| CN110276790A (en) * | 2019-06-28 | 2019-09-24 | 易思维（杭州）科技有限公司 | Point cloud registration method based on shape constraining |
-
2022
- 2022-04-25 CN CN202280007558.5A patent/CN116802686A/en active Pending
- 2022-04-25 WO PCT/CN2022/088879 patent/WO2023205984A1/en unknown
Also Published As
| Publication number | Publication date |
|---|---|
| WO2023205984A1 (en) | 2023-11-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP4825253B2 (en) | System and method for deformable object recognition | |
| CN106611411B (en) | Method for segmenting ribs in medical image and medical image processing device | |
| CN106709947B (en) | Three-dimensional human body rapid modeling system based on RGBD camera | |
| JP4785880B2 (en) | System and method for 3D object recognition | |
| CN104424629B (en) | A kind of x-ray chest radiograph lung segmentation method and apparatus | |
| CN109215032A (en) | The method and system of image segmentation | |
| JP2015522200A (en) | Human face feature point positioning method, apparatus, and storage medium | |
| KR20010042659A (en) | Face recognition from video images | |
| CN110782428B (en) | Method and system for constructing clinical brain CT image ROI template | |
| CN112132876B (en) | Initial pose estimation method in 2D-3D image registration | |
| CN106611416A (en) | Method and apparatus for lung segmentation in medical image | |
| CN111241331A (en) | Image searching method, device, equipment and medium based on artificial intelligence | |
| JP2003061936A (en) | Moving three-dimensional model formation apparatus and method | |
| CN114022554A (en) | Massage robot acupuncture point detection and positioning method based on YOLO | |
| Fan et al. | Convex hull aided registration method (CHARM) | |
| Chen et al. | 3D reconstruction of unstructured objects using information from multiple sensors | |
| Delibasis et al. | Automatic point correspondence using an artificial immune system optimization technique for medical image registration | |
| CN109064473B (en) | 2.5D ultrasonic panoramic image segmentation method | |
| CN112381952B (en) | Face contour point cloud model reconstruction method and device based on multiple cameras | |
| CN108804861B (en) | Spinal minimally invasive surgery training system and method with true force feedback | |
| CN114699661B (en) | Pose association determination and display method | |
| CN116802686A (en) | ROI (region of interest) region association and labeling method | |
| CN115331153B (en) | Posture monitoring method for assisting vestibule rehabilitation training | |
| CN108694348B (en) | Tracking registration method and device based on natural features | |
| CN110415292A (en) | Movement attitude vision measurement method of ring identification and application thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |