CN112950684A - Target feature extraction method, device, equipment and medium based on surface registration - Google Patents

Target feature extraction method, device, equipment and medium based on surface registration Download PDF

Info

Publication number
CN112950684A
CN112950684A CN202110229166.0A CN202110229166A CN112950684A CN 112950684 A CN112950684 A CN 112950684A CN 202110229166 A CN202110229166 A CN 202110229166A CN 112950684 A CN112950684 A CN 112950684A
Authority
CN
China
Prior art keywords
data
dimensional
template
processed
grid data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110229166.0A
Other languages
Chinese (zh)
Other versions
CN112950684B (en
Inventor
张旭
方伟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wuhan United Imaging Zhirong Medical Technology Co Ltd
Original Assignee
Wuhan United Imaging Zhirong Medical Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuhan United Imaging Zhirong Medical Technology Co Ltd filed Critical Wuhan United Imaging Zhirong Medical Technology Co Ltd
Priority to CN202310975174.9A priority Critical patent/CN117058409A/en
Priority to CN202110229166.0A priority patent/CN112950684B/en
Publication of CN112950684A publication Critical patent/CN112950684A/en
Application granted granted Critical
Publication of CN112950684B publication Critical patent/CN112950684B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10081—Computed x-ray tomography [CT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20112—Image segmentation details
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30008—Bone
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Multimedia (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

The application relates to a target feature extraction method, a device, equipment and a medium based on surface registration. The method comprises the following steps: acquiring to-be-processed three-dimensional surface grid data corresponding to a to-be-processed target; acquiring pre-generated three-dimensional template data corresponding to the target to be processed; performing surface registration on the surface template grid data in the three-dimensional template data and the to-be-processed three-dimensional surface grid data to establish a corresponding relation between the surface template grid data and the to-be-processed three-dimensional surface grid data; reading three-dimensional template feature labeling information from the three-dimensional template data; and mapping the three-dimensional template feature marking information to the three-dimensional surface mesh data to be processed according to the corresponding relation so as to extract the target feature in the three-dimensional surface mesh data to be processed. The method can improve the treatment efficiency.

Description

Target feature extraction method, device, equipment and medium based on surface registration
Technical Field
The present application relates to the field of image processing technologies, and in particular, to a method, an apparatus, a device, and a medium for extracting target features based on surface registration.
Background
With the development of image processing technology, the automatic feature extraction of three-dimensional target organs in CT or MR images can be widely applied to the application scenes of auxiliary diagnosis and auxiliary treatment based on medical images. Through the extracted characteristic points, characteristic linear structures or characteristic areas, doctors are helped to carry out disease diagnosis, or carry out operation planning, or input for subsequent intelligent processing of a computer, and the input is used as an unavailable key step in a full-automatic algorithm.
In the conventional technology, a feature detection method through parameter fitting, a feature detection method based on local feature analysis and a feature point detection method based on machine learning are included, but in the methods, a target to be detected is required to have local specific features, otherwise, the target cannot be located, or different algorithms need to be designed for detecting the feature points, the feature lines or the feature areas. There is no way to design an algorithm to simultaneously implement the detection of various types of feature structures.
Therefore, in order to improve the above method, a feature point, line, or region extraction method based on image registration is introduced, which can overcome the above problems.
However, in the feature point, line or region extraction method based on image registration, which uses a three-dimensional matrix as a target storage carrier, the efficiency of the whole operation process is very low, and the whole registration process is concerned, so that the processing amount is very large and the efficiency is further reduced.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a method, an apparatus, a device and a medium for extracting target features based on surface registration, which can improve processing efficiency.
A method of target feature extraction based on surface registration, the method comprising:
acquiring to-be-processed three-dimensional surface grid data corresponding to a to-be-processed target;
acquiring pre-generated three-dimensional template data corresponding to the target to be processed;
performing surface registration on the surface template grid data in the three-dimensional template data and the to-be-processed three-dimensional surface grid data to establish a corresponding relation between the surface template grid data and the to-be-processed three-dimensional surface grid data;
reading three-dimensional template feature labeling information from the three-dimensional template data;
and mapping the three-dimensional template feature marking information to the three-dimensional surface mesh data to be processed according to the corresponding relation so as to extract the target feature in the three-dimensional surface mesh data to be processed.
In one embodiment, the generating manner of the three-dimensional template data includes:
acquiring three-dimensional surface grid data of a sample;
carrying out surface registration on the sample three-dimensional surface grid data to obtain surface template grid data;
and marking the features on the surface template grid data to obtain three-dimensional template data.
In one embodiment, the labeling the features on the surface template mesh data to obtain three-dimensional template data includes:
recording the serial number or the point set serial number group of the points corresponding to the target feature structure on the surface grid data of the three-dimensional template;
and obtaining three-dimensional template data according to the serial numbers of the points or the serial number groups of the point sets and the surface template grid data.
In one embodiment, the surface registration of the sample three-dimensional surface mesh data to obtain surface template mesh data includes:
acquiring corresponding points in the three-dimensional surface grid data of each sample;
and processing the corresponding points according to a preset rule to obtain the surface template grid data.
In one embodiment, before the obtaining the corresponding point in each of the sample three-dimensional surface mesh data, the method further includes:
all sample three-dimensional grid data are aligned to the same coordinate space.
In one embodiment, the aligning all of the sample three-dimensional mesh data to the same coordinate space includes:
all sample three-dimensional mesh data are aligned to the same coordinate space by an affine transformation technique.
In one embodiment, the mapping, according to the correspondence, the three-dimensional template feature labeling information to the to-be-processed three-dimensional surface mesh data to extract a target feature in the to-be-processed three-dimensional surface mesh data includes:
acquiring the closest point corresponding to the template feature marking information in the three-dimensional surface grid data to be processed, and taking the closest point as a feature point corresponding to a target to be processed; or
And acquiring a set of closest points corresponding to the template feature marking information in the three-dimensional surface grid data to be processed, and taking the set of closest points as a feature line or a feature area corresponding to a target to be processed.
In one embodiment, the performing surface registration on the surface template mesh data in the three-dimensional template data and the three-dimensional surface mesh data to be processed includes:
taking surface template grid data in the three-dimensional template data as floating grid data, taking the three-dimensional surface grid data to be processed as target grid data, and performing surface registration on the floating grid data to the target grid data; the surface registration includes a correspondence of the floating grid data to points in the target grid data and a processing of the corresponding points.
A target feature extraction apparatus based on surface registration, the apparatus comprising:
the system comprises a to-be-processed data acquisition module, a processing module and a processing module, wherein the to-be-processed data acquisition module is used for acquiring to-be-processed three-dimensional surface grid data corresponding to a to-be-processed target;
the template data acquisition module is used for acquiring pre-generated three-dimensional template data corresponding to the target to be processed;
the first surface registration module is used for carrying out surface registration on the surface template grid data in the three-dimensional template data and the three-dimensional surface grid data to be processed so as to establish the corresponding relation between the surface template grid data and the three-dimensional surface grid data to be processed;
the mapping module is used for reading three-dimensional template feature marking information from the three-dimensional template data; and mapping the three-dimensional template feature marking information to the three-dimensional surface mesh data to be processed according to the corresponding relation so as to extract the target feature in the three-dimensional surface mesh data to be processed.
In one embodiment, the apparatus further comprises:
the sample data acquisition module is used for acquiring the three-dimensional surface grid data of the sample;
the second surface registration module is used for carrying out surface registration on the sample three-dimensional surface grid data to obtain surface template grid data;
and the template generation module is used for marking the features on the surface template grid data to obtain three-dimensional template data.
A computer device comprising a memory storing a computer program and a processor implementing the steps of the method in any of the above embodiments when executing the computer program.
A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method of any of the above embodiments.
According to the target feature extraction method, device, equipment and medium based on surface registration, the processed correspondence is three-dimensional surface grid data instead of three-dimensional matrix data, so that the processing efficiency can be improved, secondly, as the three-dimensional template feature labeling information is stored in the three-dimensional template data, after the surface registration, various types of target features can be directly extracted from the three-dimensional surface grid data to be processed through mapping at one time, the processing efficiency is further improved, and finally, as the target features concern is a surface region with rich features, namely, the surface registration is carried out instead of the image registration, the processing efficiency is further improved.
Drawings
FIG. 1 is a diagram of an application environment of a target feature extraction method based on surface registration in an embodiment;
FIG. 2 is a schematic flow chart of a method for extracting target features based on surface registration in one embodiment;
FIG. 3 is a schematic illustration of the effect of hip segmentation in one embodiment;
FIG. 4 is a schematic illustration of the effect of a three-dimensional reconstruction of a hip joint in one embodiment;
FIG. 5 is a mapping diagram of three-dimensional template data and three-dimensional surface mesh data to be processed, under an embodiment;
FIG. 6 is a flow diagram that illustrates a manner in which three-dimensional template data may be generated, according to one embodiment;
FIG. 7 is a schematic illustration of initial states of three-dimensional template data and three-dimensional surface mesh data to be processed in one embodiment;
FIG. 8 is a diagram illustrating coarse alignment of grids in one embodiment;
FIG. 9 is a schematic diagram of grid rigid registration in one embodiment;
FIG. 10 is a schematic diagram of mesh elastic registration in one embodiment;
FIG. 11 is a flowchart illustrating a method for extracting target features based on surface registration according to another embodiment;
FIG. 12 is a block diagram of an embodiment of a target feature extraction apparatus based on surface registration;
FIG. 13 is a diagram illustrating an internal structure of a computer device according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The target feature extraction method based on surface registration provided by the application can be applied to the application environment shown in fig. 1. Wherein the terminal 102 communicates with the medical imaging device 104 over a network. The terminal 102 may receive a three-dimensional image scanned by the medical imaging device 104 and using a three-dimensional matrix as a storage mode, or the terminal 102 acquires a three-dimensional image scanned by the medical imaging device 104 and using a three-dimensional matrix as a storage mode from a database or the like, and performs three-dimensional reconstruction on the three-dimensional image to obtain three-dimensional surface mesh data, so as to acquire pre-generated three-dimensional template data corresponding to a target to be processed; performing surface registration on the surface template grid data in the three-dimensional template data and the three-dimensional surface grid data to be processed to establish a corresponding relation between the surface template grid data and the three-dimensional surface grid data to be processed, and finally reading three-dimensional template feature marking information from the three-dimensional template data; and mapping the three-dimensional template feature labeling information to the to-be-processed three-dimensional surface mesh data according to the corresponding relation so as to extract the target feature in the to-be-processed three-dimensional surface mesh data. The processed data is three-dimensional surface grid data instead of three-dimensional matrix data, so that the processing efficiency can be improved, secondly, three-dimensional template feature labeling information is stored in the three-dimensional template data, so that after surface registration, various types of target features can be directly extracted from the three-dimensional surface grid data to be processed at one time through mapping, the processing efficiency is further improved, and finally, the processing efficiency is further improved because the processing method focuses on a surface area with rich features, namely surface registration is carried out instead of image registration.
The terminal 102 may be, but is not limited to, a functional module and a dedicated circuit of various personal computers, notebook computers, smart phones, tablet computers, portable wearable devices, and medical imaging devices themselves. In this embodiment, the terminal 102 may include a mobile terminal device of a patient and/or a mobile terminal device of a medical operator. The medical imaging apparatus 104 includes, but is not limited to, various imaging apparatuses such as a CT imaging apparatus (Computed Tomography, which uses a precisely collimated X-ray beam to perform a cross-sectional scan around a certain portion of a human body together with a detector having a very high sensitivity one by one and can reconstruct a precise three-dimensional position image of a tumor or the like through the CT scan), a magnetic resonance apparatus (which is a kind of Tomography, which uses a magnetic resonance phenomenon to obtain an electromagnetic signal from a human body and reconstruct an information image of the human body), a Positron Emission Computed Tomography (Positron Emission Computed Tomography) apparatus, a Positron Emission magnetic resonance imaging system (PET/MR), and the like.
In one embodiment, as shown in fig. 2, a method for extracting target features based on surface registration is provided, which is described by taking the method as an example applied to the terminal in fig. 1, and includes the following steps:
s202: and acquiring the to-be-processed three-dimensional surface grid data corresponding to the to-be-processed target.
Specifically, the three-dimensional surface mesh data to be processed refers to three-dimensional surface mesh data, which may be obtained by three-dimensionally reconstructing a three-dimensional image acquired by the medical imaging device.
In the field of medical imaging, medical image data of three-dimensional scanning, such as CT or MR, is generally three-dimensional image, that is, medical image data stored in a three-dimensional matrix form, where the three-dimensional image includes an object to be processed, such as a target organ or tissue to be subjected to feature extraction. In other embodiments, the to-be-processed three-dimensional surface mesh data may also be obtained by performing three-dimensional reconstruction on a human body surface, so as to perform facial feature extraction and feature region segmentation, and be used for realizing functions of facial expression recognition, facial analysis, five sense organs analysis, and the like. Specifically, when a human face or the like is identified, a 3D camera may be used to acquire an image for feature extraction, where the image acquired by the 3D camera is, for example, facial point cloud data acquired by scanning a human face. The terminal can perform facial feature extraction on such data. The difference between this case and the above example is that the acquired image is no longer 3D matrix data, such as CT or MR, but point cloud data, and therefore there is only a difference in the surface reconstruction algorithm. For point cloud data, the point cloud data can be reconstructed into three-dimensional grid data through algorithms such as Poisson surface reconstruction and the like. And then, the same method can be adopted to perform three-dimensional grid data registration by using the three-dimensional grid data to complete feature extraction. For example, extraction of facial features, extraction of facial symmetry lines, extraction of different facial regions (forehead, cheek, etc.), and application to different scenes such as face recognition, facial feature analysis, etc. are completed.
Specifically, the three-dimensional reconstruction may specifically include: firstly, the terminal carries out image segmentation on a target to be processed in a three-dimensional matrix through an image segmentation technology to obtain mask data stored in the form of the three-dimensional matrix, and then carries out three-dimensional reconstruction on the mask data. The image segmentation technology includes, but is not limited to, an image segmentation technology based on a deep learning full convolution network, or based on a traditional machine learning (such as a random forest, etc.), or based on a segmentation technology such as clustering, region growing, active contour, level set, thresholding, etc., and specifically, fig. 3 is a schematic diagram of the hip segmentation effect in one embodiment. The method for three-dimensionally reconstructing mask data includes, but is not limited to, a Marching Cube algorithm, interpolation reconstruction using the Marching Cube algorithm based on a surface threshold near a contour, a poisson surface reconstruction algorithm, and the like, and specifically, fig. 4 is a schematic diagram of a hip joint three-dimensional reconstruction effect in one embodiment.
The hip surface feature extraction application in the hip replacement surgery is taken as an example to explain, and the terminal can segment the hip bone in the CT data by using an image segmentation technology based on a deep learning full convolution network to obtain segmented mask data of the hip bone on one side. The segmented mask data of the hip bone is then used to perform a three-dimensional reconstruction of the bone surface. Specifically, the following methods are explained:
the Marching Cube algorithm comprises the following steps: and performing isosurface reconstruction on the contour of the hip bone segmentation mask data based on a Marching Cube algorithm, wherein the set isosurface reconstruction threshold can select any value between a value representing a background pixel and a value representing a target structure pixel in the segmentation mask data, such as a value between values of pixels of the hip bone and the background. In the present example, the background pixel of the divided mask data is represented by 0, and the target structure pixel is represented by 1, so that any value between 0 and 1 can be selected, and the threshold value selected in the present example is 0.5.
The interpolation reconstruction near the contour according to the surface threshold value by using the Marching Cube algorithm comprises the following steps: specifically, the terminal performs interpolation reconstruction on original CT data by combining the contour of the segmentation mask data and using a Marching Cube algorithm to perform interpolation reconstruction on the CT data near the contour according to a bone surface threshold. The threshold is selected to be any gray value between the edge of the target structure, which in this example is the hip, and the threshold is selected to be 150HU, and the target structure is the hip. The reconstruction range of the Marching Cube is limited within the range of 1-5 pixels from the contour line of the distance division mask data, and the Marching Cube is reconstructed within the range of 3 pixels from the contour line in the embodiment.
The Poisson surface reconstruction algorithm comprises the following steps: and performing three-dimensional reconstruction on the contour edge points of the segmentation mask data based on a Poisson surface reconstruction algorithm.
S204: and acquiring pre-generated three-dimensional template data corresponding to the target to be processed.
Specifically, the three-dimensional template data includes three-dimensional template surface mesh data and three-dimensional template feature labeling information, which is a pre-generated template, that is, a template corresponding to the three-dimensional surface mesh data to be processed, and the template is generated according to the sample three-dimensional surface mesh data, for example, the sample three-dimensional surface mesh data is registered to obtain a surface mesh template, and then the target feature of the target to be processed in the surface mesh template is manually or semi-automatically extracted. The semi-automatic extraction mainly aims at points which are inconvenient to select manually, and target features need to be extracted by combining some algorithms, for example, the center point of an acetabulum fossa, some points on the acetabulum fossa need to be selected manually, then a ball is fitted to the selected points, and then the center of the ball is calculated.
Optionally, the three-dimensional template data may be stored in an organ manner when being stored, so that after the three-dimensional surface mesh data to be processed is obtained, the stored corresponding three-dimensional template data may be selected according to the organ corresponding to the three-dimensional surface mesh data to be processed.
S206: and carrying out surface registration on the surface template grid data in the three-dimensional template data and the processed three-dimensional surface grid data to establish the corresponding relation between the surface template grid data and the three-dimensional surface grid data to be processed.
Specifically, the surface registration refers to unifying the surface template mesh data in the three-dimensional template data and the three-dimensional surface mesh data to be processed into the same coordinate system. The positions on the surface template grid data and the positions on the three-dimensional surface grid data to be processed can be in one-to-one correspondence through registration, so that a foundation is laid for subsequent feature mapping.
Wherein the surface registration may include linear registration and elastic registration, and in some special cases, the linear registration includes Mesh global registration, affine registration or rigid body registration. The elastic registration relies on Mesh to complete the linear registration. The method of surface registration in this embodiment includes, but is not limited to, transformation cpd (coherent Point drift) algorithm which can implement linear and elastic registration, ICP algorithm in rigid registration algorithm which can implement affine registration with scale transformation parameters, NDT (Normal-distribution Transform), phase correlation algorithm, and the like. Global registration includes, but is not limited to, a principal axis alignment algorithm based on PCA method, a feature matching family method based on RANSAC framework, brute force search method (by traversal search in angle and orientation), 4PCS, Super4PCS, and the like.
S208: reading three-dimensional template feature marking information from the three-dimensional template data; and mapping the three-dimensional template feature labeling information to the to-be-processed three-dimensional surface mesh data according to the corresponding relation so as to extract the target feature in the to-be-processed three-dimensional surface mesh data.
Specifically, the mapping refers to mapping positions of the surface template mesh data and the to-be-processed three-dimensional surface mesh data, so that a corresponding relationship between the positions can be formed, and thus, the three-dimensional template feature labeling information can be read, and the target feature corresponding to the three-dimensional template feature labeling information is determined in the to-be-processed three-dimensional surface mesh data.
Specifically, referring to fig. 5, fig. 5 is a schematic mapping diagram of three-dimensional template data and three-dimensional surface mesh data to be processed in an embodiment. The three-dimensional template feature labeling information in the three-dimensional template data comprises labeling information of feature points, feature lines and feature areas, and is mapped to the three-dimensional surface mesh data to be processed, so that target features in the three-dimensional surface mesh data to be processed can be extracted.
According to the target feature extraction method based on surface registration, the processed data corresponds to three-dimensional surface grid data instead of three-dimensional matrix data, so that the processing efficiency can be improved, secondly, three-dimensional template feature labeling information is stored in the three-dimensional template data, so that after surface registration, various types of target features can be directly extracted from the three-dimensional surface grid data to be processed through mapping once, the processing efficiency is further improved, and finally, the target feature extraction method focuses on a surface area with rich features, namely, surface registration is carried out instead of image registration, so that the processing efficiency is further improved.
In one embodiment, referring to fig. 6, fig. 6 is a flowchart of a generation manner of three-dimensional template data in an embodiment, where the generation manner of the three-dimensional template data may include:
s602: and acquiring the three-dimensional surface grid data of the sample.
Specifically, the sample three-dimensional surface mesh data may be obtained by performing three-dimensional reconstruction according to different three-dimensional images, and the specific three-dimensional reconstruction method may be as described above. Still taking the hip as an example for explanation, the hip medical image data of a large number of different patients is collected by the terminal as a training set, and then the medical image data in the training set is segmented and reconstructed according to the three-dimensional reconstruction method to obtain the sample three-dimensional surface mesh data. If the medical image data in the training set is three-dimensional surface grid data, three-dimensional reconstruction is not needed.
It should be noted that, when only one set of sample three-dimensional surface mesh data exists, the sample three-dimensional surface mesh data is directly used as the surface template mesh data of the three-dimensional template data, and if at least two sets of sample three-dimensional surface mesh data exist, the sample three-dimensional surface mesh data is subjected to surface registration to obtain the surface template mesh data, and the three-dimensional template data is generated after feature labeling, which can be specifically referred to below.
S604: and carrying out surface registration on the three-dimensional surface grid data of the sample to obtain surface template grid data.
Specifically, the surface registration here is the same as the method of surface registration mentioned above, and only the object becomes the sample three-dimensional surface mesh data, and thus the description is omitted.
Specifically, when the positions of the corresponding points after the registration are substantially consistent, one position of each corresponding point is directly acquired to serve as the surface mesh data. In other embodiments, all the sample mesh data may be aligned to a space, and then the positions corresponding to all the mesh data are averaged to obtain average surface mesh data, specifically, the averaging of the positions here refers to that after the sample three-dimensional surface mesh data in the training set is registered, the terminal may obtain the position of the corresponding point, then average the positions to obtain the average position of the point, and finally obtain the average positions of all the corresponding points to obtain the average three-dimensional surface mesh data.
S606: and marking the features on the surface template grid data to obtain the three-dimensional template data.
Specifically, features to be extracted are labeled on the average three-dimensional surface mesh data to obtain three-dimensional template data. The labeling here may be manual or semi-automatic labeling. The semi-automatic labeling mainly aims at points which are inconvenient to select manually, and target features need to be extracted by combining algorithms, for example, the center point of an acetabulum fossa, some points on the acetabulum fossa need to be selected manually, then a ball is fitted to the selected points, and then the center of the ball is calculated.
In the above embodiment, by registering the sample three-dimensional surface mesh data to obtain the surface mesh data, and then labeling the features in the surface grid data to generate three-dimensional template data, so that the features can be labeled according to the needs, and the method is suitable for extracting various types of features, including various different types of feature structures on the surface and inside of the target, such as various features of feature points, feature lines, feature areas and the like, and is applicable to both the characteristic target with local specificity and the characteristic without local specificity, in addition, a plurality of characteristics can be marked when marking, therefore, the extraction efficiency is high, the method is suitable for rapid extraction of a large number of features, all required features can be extracted at one time in parallel, the number of the features to be detected has no upper limit, and the algorithm efficiency cannot be reduced due to the increase of the number of the features to be detected. Finally, the method does not limit the characteristics of points, lines, surfaces and the like with volumes, performs template matching and characteristic extraction by taking the surface of the target structure as a carrier, and has natural advantages for characteristic extraction on the surface structure, particularly the characteristics of points, lines, surfaces and the like without volumes.
In one embodiment, labeling the features on the surface template mesh data to obtain three-dimensional template data includes: recording the serial number or the point set serial number group of points corresponding to the target feature structure on the surface grid data of the three-dimensional template; and obtaining three-dimensional template data according to the serial numbers of the points or the serial number groups of the point sets and the surface template grid data.
During calibration, the terminal may record a list of serial numbers of corresponding vertices of each feature structure on the average three-dimensional surface mesh data. If the structure is a point structure, recording the serial number of the corresponding vertex of the corresponding structure point on the template; if the structure is a line structure or a surface structure, the sequence number of all points belonging to the line structure or the surface structure on the template can be recorded.
In the above embodiment, by applying the vertices in the mesh data, the positions of the target features can be accurately recorded.
In one embodiment, surface registering the sample three-dimensional surface mesh data to obtain surface template mesh data includes: acquiring corresponding points in the three-dimensional surface grid data of each sample; and processing the corresponding points according to a preset rule to obtain the surface template grid data.
In one embodiment, before obtaining the corresponding point in the mesh data of the three-dimensional surface of each sample, the method further includes: all sample three-dimensional grid data are aligned to the same coordinate space.
In one embodiment, aligning all sample three-dimensional mesh data to the same coordinate space comprises: all sample three-dimensional mesh data are aligned to the same coordinate space by an affine transformation technique.
Specifically, if all the sample three-dimensional surface mesh data are captured at the same position view angle, that is, all the sample three-dimensional surface mesh data are in the same coordinate space, the terminal first determines a corresponding point in each sample three-dimensional surface mesh data, where the corresponding point may be determined by a position, for example, a closest point in different sample three-dimensional surface mesh data is determined as the corresponding point. The surface template grid data is then obtained by processing the corresponding points, e.g. by elastic or rigid registration. If all the sample three-dimensional surface mesh data are not shot at the same position and view angle, that is, all the sample three-dimensional surface mesh data are not shot in the same coordinate space, all the sample three-dimensional mesh data are preferentially aligned to the same coordinate space, for example, all the sample three-dimensional mesh data are aligned to the same coordinate space by an affine transformation technology, including that all the sample three-dimensional surface mesh data are linearly aligned as a whole by translation, rotation and scaling.
Specifically, in order to ensure the orderliness of surface registration, in this embodiment, one sample three-dimensional surface mesh data is randomly selected as the reference mesh data, and then the remaining sample three-dimensional surface mesh data in the training set is subjected to surface registration to the reference mesh data, that is, the remaining sample three-dimensional surface mesh data is registered to a space corresponding to the reference mesh data by using a surface registration technique, for example, an affine transformation technique, specifically, all sample three-dimensional mesh data are aligned to the same coordinate space by using a rough alignment method, and then, the corresponding average three-dimensional surface mesh data is obtained after averaging.
The surface registration comprises at least one of grid coarse alignment, grid rigid registration and grid elastic registration, the execution of the grid coarse alignment, the grid rigid registration and the grid elastic registration has a sequence, the grid coarse alignment, the grid rigid registration and the grid elastic registration are generally performed in sequence, when the initial states of two grid data are better aligned, or a rigid registration algorithm of the grid data selects a global registration algorithm, the grid coarse alignment is not depended on, and the grid coarse alignment can be omitted.
Wherein specific definitions for mesh coarse alignment, mesh rigid registration and mesh elastic registration can be seen below.
In one embodiment, mapping the three-dimensional template feature labeling information to the to-be-processed three-dimensional surface mesh data according to the corresponding relationship to extract a target feature in the to-be-processed three-dimensional surface mesh data, includes: acquiring the closest point corresponding to the template feature marking information in the three-dimensional surface grid data to be processed, and taking the closest point as the feature point corresponding to the target to be processed; or acquiring a set of closest points corresponding to the template feature marking information in the three-dimensional surface grid data to be processed, and taking the set of closest points as a feature line or a feature area corresponding to the target to be processed.
Specifically, the mapping manner is specifically defined in this embodiment, where the mapping is implemented by means of nearest neighbor point search, and specifically, the mapping manner may be divided into mapping of feature points and mapping of feature lines and feature areas. The mapping of the feature points is to obtain the closest points corresponding to the template feature marking information in the three-dimensional surface mesh data to be processed, and the closest points are used as the feature points corresponding to the target to be processed, namely the target features. The mapping of the characteristic lines and the characteristic areas can be split into a plurality of characteristic point sets, so that the mapping is realized by adopting a characteristic point mapping mode for a plurality of times.
In the above embodiment, the surface of the target structure is used as a carrier to perform template matching and feature extraction, and feature extraction on the surface structure, especially features such as points, lines, and planes without volume, has natural advantages. The method has high feature extraction efficiency, is suitable for rapid extraction of a large number of features, can extract all required features in parallel at one time, has no upper limit on the number of the features to be detected, and does not reduce the algorithm efficiency due to the increase of the number of the features to be detected.
In one embodiment, performing surface registration on the surface template mesh data in the three-dimensional template data and the processed three-dimensional surface mesh data to establish a corresponding relationship between the surface template mesh data and the three-dimensional surface mesh data to be processed includes: taking the surface template grid data in the three-dimensional template data as floating grid data, taking the three-dimensional surface grid data to be processed as target grid data, and performing surface registration on the floating grid data to the target grid data; wherein surface registration comprises correspondence of the floating grid data to points in the target grid data and processing of the corresponding points.
Specifically, the surface registration technique is realized by multiple steps, and grid coarse alignment, grid rigid registration and grid elastic registration of grid data are sequentially performed. Wherein the coarse alignment of the mesh may be omitted in some cases, for example, the initial state of the two mesh data itself is better aligned, or the rigid registration algorithm of the mesh data selects a global registration algorithm, which does not rely on the coarse alignment of the mesh data.
Specifically, referring to fig. 7, fig. 7 is a schematic diagram of initial states of surface template mesh data and three-dimensional surface mesh data to be processed in three-dimensional template data in an embodiment, where one is the surface template mesh data in the three-dimensional template data and the other is the three-dimensional surface mesh data to be processed, in which in the registration, the surface template mesh data in the three-dimensional template data is used as floating mesh data, the three-dimensional surface mesh data to be processed is used as target mesh data, and the floating mesh data is surface-registered to the target mesh data.
Specifically, referring to fig. 8, fig. 8 is a schematic diagram of coarse alignment of grids in an embodiment, in this embodiment, the purpose of the coarse alignment algorithm of grids is to perform approximate alignment in space for two structurally similar objects, and perform matching in scale at the same time, in this embodiment, an alignment method based on PCA principal axis detection is adopted. The method only uses the point cloud data formed by the vertexes in the Mesh. Assuming that the target mesh data is Pt and the floating mesh data is Pf, the following steps are performed:
firstly, carrying out translational alignment: and calculating the centers of the two pieces of grid data, and aligning the centers of the two point clouds by translating the grid data Pf to the grid data Pt.
Secondly, establishing a main shaft coordinate system: using a PCA algorithm to perform principal component analysis on the grid data Pt and the grid data Pf respectively to obtain 3 × 3 matrices Rt and Rf formed by three principal component vectors, where the matrices represent rotation matrices of the point cloud from the current coordinate system to a coordinate system established by the three principal component vectors thereof (hereinafter referred to as principal axis coordinate systems).
Third, alignment of the main axes: because the grid data Pt and the grid data Pf are grid data of the same three-dimensional structure of different patients, the grid data Pt and the grid data Pf are similar in form and have similar respective principal axes, and after the grid data Pt and the grid data Pf are respectively transformed to a principal axis coordinate system by using Rt and Rf, alignment in the directions of the two grid data can be realized.
Fourth, dimension alignment: the grid data Pt and Pf may have a difference in scale, and the grid data Pf may be subjected to scale correction on the three main axes, respectively, so that the difference between the grid data Pf on the three main axes and the maximum and minimum values of the grid data Pt is equal. The transformation scale on the three principal axes of the grid data Pf is calculated by the following formula:
Figure BDA0002958255630000131
in the formula maxPTxRepresents the maximum value of the x coordinate, Scale, of the Pt point cloud under a principal axis coordinate systemxRepresenting the x direction in a principal axis coordinate system
Fifthly, correcting the point cloud of the grid data Pf.
Sixthly, azimuth correction: after the main shafts are aligned, the condition that the main shaft direction is not matched with the positive direction and the negative direction may still exist, so that the grid data Pf is traversed to the positive direction and the negative direction of three shafts, the directions are respectively corrected in 8 directions, 8 grid data are obtained, the grid data Pf which are closest to the grid data Pt are respectively calculated, and the best transformation of the grid data Pf is selected as the minimum distance. This completes the best matching of the mesh data Pf to the mesh data Pt.
Specifically, referring to fig. 9, fig. 9 is a schematic diagram of a rigid grid registration in an embodiment, in this example, the rigid grid registration is implemented by an Iterative Closest Point (ICP) algorithm, and an ICP algorithm core is implemented by minimizing an objective function:
Figure BDA0002958255630000141
wherein P istIs a target point cloud, PfFor floating point cloud, R and T are rotation matrix and translation vector to be optimized, and the R and T are adjusted through optimization to enable f (R, T) to be minimum.
The algorithm is optimized through iteration, and specifically comprises the following steps:
firstly, searching a nearest corresponding point on the other grid data for each point of the two grid data, wherein the nearest point solves a transformation matrix, namely R and T, in an SVD decomposition mode, then executing rigid transformation once, and repeating the steps until an error is smaller than a set threshold value or the iteration times are reached.
Besides, a point cloud rigid registration algorithm based on deep learning, or a rigid registration method based on corresponding feature points can also be used.
Specifically, referring to fig. 10, fig. 10 is a schematic diagram of the mesh elastic registration in an embodiment, and the mesh elastic registration in this example is performed in an iterative manner:
firstly, searching each point in two grid data for the nearest corresponding point on the other grid data; a sticky transformation is then performed between the two pairs of points. The viscosity transformation is defined as directly displacing each point on the floating grid data to the direction of the corresponding point on the target grid data; and performing elastic transformation between the measuring point pairs. The elastic transformation is defined as that for each point p on the target grid data, the original coordinate position is replaced according to the weighted average of the coordinate positions of the nearest N adjacent points on the target grid data and the grid data, which is equivalent to that the position of each point p on the target grid data is subjected to one smooth operation; the weighting of each neighboring point depends on the distance from the neighboring point to the p point, and the closer the distance, the greater the weighting. Defined in this example as the gaussian radial basis function of the distance to the p-point. And repeating the steps until the iteration times are met.
Optionally, the terminal can also be elastically registered in combination with a multi-scale idea to improve the operation efficiency. The specific idea is as follows: down-sampling the grid data from large to small in different degrees to obtain a plurality of grid data pairs (which can be 2-6 grid data pairs with different resolutions) from low resolution to high resolution; elastically registering the grid data with the lowest resolution; applying the deformation field to the floating grid data of the next level of resolution to perform elastic registration of the next level of resolution; and repeating the steps to perform elastic registration from coarse resolution to fine resolution until the grid data registration of the highest resolution (original resolution) is completed.
In addition, other elastic registration schemes, such as an elastic registration method based on elastic deformation field estimation of deep learning, etc., may also be employed.
In one embodiment, see fig. 11, where fig. 11 is a flowchart of a target feature extraction method based on surface registration in another embodiment, in this embodiment, two parts are mainly included, one part is the making of a template, and the other part is feature extraction.
The template can be manufactured as shown in fig. 6, in the template manufacturing process, a large amount of existing training set data is utilized to perform three-dimensional reconstruction on a target to be processed in the data to form sample three-dimensional surface grid data, the sample three-dimensional surface grid data is registered to the same space through a three-dimensional grid surface registration technology to form average three-dimensional surface grid data, and then manual advance marking of features to be extracted is performed on the average data to form marked three-dimensional template data.
In the process of feature extraction, namely an algorithm execution stage, three-dimensional reconstruction is carried out on an input three-dimensional image or point cloud data to obtain three-dimensional grid data to be processed, then three-dimensional grid data to be processed and surface template grid data in the three-dimensional template data are registered by utilizing three-dimensional grid surface registration calculation, namely the three-dimensional grid data and the surface template grid data are spatially registered, and then three-dimensional template feature marking information in the three-dimensional template data is mapped into the three-dimensional grid data to be processed, so that the extraction of target features is completed.
In the above embodiment, the features may be labeled as needed, so that the method is suitable for extracting various types of features, including various types of feature structures on the surface and inside of the target, such as various features including feature points, feature lines, feature areas, and the like, and is suitable for both a feature target with local specificity and a feature without local specificity. Finally, the method does not limit the characteristics of points, lines, surfaces and the like with volumes, performs template matching and characteristic extraction by taking the surface of the target structure as a carrier, and has natural advantages for characteristic extraction on the surface structure, particularly the characteristics of points, lines, surfaces and the like without volumes.
It should be understood that although the steps in the flowcharts of fig. 2, 6 and 11 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 2, 6 and 11 may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of performing the steps or stages is not necessarily sequential, but may be performed alternately or alternately with other steps or at least some of the other steps.
In one embodiment, as shown in fig. 12, there is provided a target feature extraction apparatus based on surface registration, including: a pending data acquisition module 100, a template data acquisition module 200, a first surface registration module 300, and a mapping module 400, wherein:
the to-be-processed data acquiring module 100 is configured to acquire to-be-processed three-dimensional surface mesh data corresponding to a to-be-processed target.
The template data obtaining module 200 is configured to obtain three-dimensional template data that is generated in advance and corresponds to a target to be processed.
The first surface registration module 300 is configured to perform surface registration on the surface template mesh data in the three-dimensional template data and the processed three-dimensional surface mesh data to establish a corresponding relationship between the surface template mesh data and the three-dimensional surface mesh data to be processed.
The mapping module 400 is used for reading three-dimensional template feature labeling information from three-dimensional template data; and mapping the three-dimensional template feature labeling information to the to-be-processed three-dimensional surface mesh data according to the corresponding relation so as to extract the target feature in the to-be-processed three-dimensional surface mesh data.
In one embodiment, the above target feature extraction apparatus based on surface registration may further include:
and the sample data acquisition module is used for acquiring the three-dimensional surface grid data of the sample.
And the second surface registration module is used for carrying out surface registration on the three-dimensional surface grid data of the sample to obtain surface template grid data.
And the template generation module is used for marking the features on the surface template grid data to obtain three-dimensional template data.
In one embodiment, the template generating module may include:
the recording unit is used for recording the serial numbers or the point set serial number groups of the points corresponding to the target characteristic structures on the grid data on the surface of the three-dimensional template;
and the generating unit is used for obtaining the three-dimensional template data according to the serial numbers of the points or the serial number groups of the point sets and the surface template grid data.
In one embodiment, the second surface registration module includes:
a corresponding point obtaining unit, configured to obtain a corresponding point in the three-dimensional surface mesh data of each sample;
and the data processing unit is used for processing the corresponding points according to a preset rule to obtain the surface template grid data.
In one embodiment, the second surface registration module further includes:
and the aligning unit is used for aligning all the sample three-dimensional grid data to the same coordinate space.
In one embodiment, the above-mentioned aligning unit is configured to align all the sample three-dimensional mesh data to the same coordinate space by an affine transformation technique.
In one embodiment, the mapping module 400 is configured to obtain a closest point corresponding to the template feature labeling information in the to-be-processed three-dimensional surface mesh data, and use the closest point as a feature point corresponding to the to-be-processed target; or acquiring a set of closest points corresponding to the template feature marking information in the three-dimensional surface grid data to be processed, and taking the set of closest points as a feature line or a feature area corresponding to the target to be processed.
In one embodiment, the first surface registration module 300 is configured to use surface template mesh data in the three-dimensional template data as floating mesh data, use three-dimensional surface mesh data to be processed as target mesh data, and perform surface registration on the floating mesh data to the target mesh data; wherein the surface registration comprises a correspondence of the floating grid data to points in the target grid data and a processing of the corresponding points.
For specific definition of the target feature extraction device based on surface registration, reference may be made to the above definition of the target feature extraction method based on surface registration, and details are not repeated here. The respective modules in the above target feature extraction apparatus based on surface registration may be wholly or partially implemented by software, hardware, and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as shown in fig. 13. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless communication can be realized through WIFI, an operator network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement a method of target feature extraction based on surface registration. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.
Those skilled in the art will appreciate that the architecture shown in fig. 13 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is provided, comprising a memory and a processor, the memory having a computer program stored therein, the processor implementing the following steps when executing the computer program: acquiring to-be-processed three-dimensional surface grid data corresponding to a to-be-processed target; acquiring pre-generated three-dimensional template data corresponding to a target to be processed; carrying out surface registration on the surface template grid data in the three-dimensional template data and the processed three-dimensional surface grid data to establish a corresponding relation between the surface template grid data and the three-dimensional surface grid data to be processed; reading three-dimensional template feature marking information from the three-dimensional template data; and mapping the three-dimensional template feature labeling information to the to-be-processed three-dimensional surface mesh data according to the corresponding relation so as to extract the target feature in the to-be-processed three-dimensional surface mesh data.
In one embodiment, the three-dimensional template data generated by the processor when executing the computer program is generated by: acquiring three-dimensional surface grid data of a sample; carrying out surface registration on the three-dimensional surface grid data of the sample to obtain surface template grid data; and marking the features on the surface template grid data to obtain the three-dimensional template data.
In one embodiment, the labeling of features on the surface template mesh data to obtain three-dimensional template data, which is implemented when the processor executes the computer program, includes: recording the serial number or the point set serial number group of points corresponding to the target feature structure on the surface grid data of the three-dimensional template; and obtaining three-dimensional template data according to the serial numbers of the points or the serial number groups of the point sets and the surface template grid data.
In one embodiment, surface registration of sample three-dimensional surface mesh data to surface template mesh data, as implemented by a processor executing a computer program, comprises: acquiring corresponding points in the three-dimensional surface grid data of each sample; and processing the corresponding points according to a preset rule to obtain the surface template grid data.
In one embodiment, before the obtaining of the corresponding point in each of the sample three-dimensional surface mesh data, which is implemented when the processor executes the computer program, the method further includes: all sample three-dimensional grid data are aligned to the same coordinate space.
In one embodiment, the aligning of all sample three-dimensional mesh data to the same coordinate space, as implemented by the processor when executing the computer program, comprises: all sample three-dimensional mesh data are aligned to the same coordinate space by an affine transformation technique.
In one embodiment, the mapping of the three-dimensional template feature labeling information to the to-be-processed three-dimensional surface mesh data according to the corresponding relationship, which is realized when the processor executes the computer program, so as to extract the target feature in the to-be-processed three-dimensional surface mesh data, includes: acquiring the closest point corresponding to the template feature marking information in the three-dimensional surface grid data to be processed, and taking the closest point as the feature point corresponding to the target to be processed; or acquiring a set of closest points corresponding to the template feature marking information in the three-dimensional surface grid data to be processed, and taking the set of closest points as a feature line or a feature area corresponding to the target to be processed.
In one embodiment, the surface registration of the surface template mesh data in the three-dimensional template data and the processed three-dimensional surface mesh data to establish a correspondence between the surface template mesh data and the three-dimensional surface mesh data to be processed, as implemented by a processor executing a computer program, comprises: taking the surface template grid data in the three-dimensional template data as floating grid data, taking the three-dimensional surface grid data to be processed as target grid data, and performing surface registration on the floating grid data to the target grid data; wherein the surface registration comprises a correspondence of the floating grid data to points in the target grid data and a processing of the corresponding points.
In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, performs the steps of: acquiring to-be-processed three-dimensional surface grid data corresponding to a to-be-processed target; acquiring pre-generated three-dimensional template data corresponding to a target to be processed; carrying out surface registration on the surface template grid data in the three-dimensional template data and the processed three-dimensional surface grid data to establish a corresponding relation between the surface template grid data and the three-dimensional surface grid data to be processed; reading three-dimensional template feature marking information from the three-dimensional template data; and mapping the three-dimensional template feature labeling information to the to-be-processed three-dimensional surface mesh data according to the corresponding relation so as to extract the target feature in the to-be-processed three-dimensional surface mesh data.
In one embodiment, the manner in which the three-dimensional template data is generated when the computer program is executed by the processor comprises: acquiring three-dimensional surface grid data of a sample; carrying out surface registration on the three-dimensional surface grid data of the sample to obtain surface template grid data; and marking the features on the surface template grid data to obtain the three-dimensional template data.
In one embodiment, the labeling of features on the surface template mesh data to obtain three-dimensional template data, which is implemented when the computer program is executed by the processor, includes: recording the serial number or the point set serial number group of points corresponding to the target feature structure on the surface grid data of the three-dimensional template; and obtaining three-dimensional template data according to the serial numbers of the points or the serial number groups of the point sets and the surface template grid data.
In one embodiment, surface registration of sample three-dimensional surface mesh data to surface template mesh data, as implemented by a computer program when executed by a processor, comprises: acquiring corresponding points in the three-dimensional surface grid data of each sample; and processing the corresponding points according to a preset rule to obtain the surface template grid data.
In one embodiment, the computer program, when executed by the processor, further comprises prior to obtaining corresponding points in each of the sample three-dimensional surface mesh data: all sample three-dimensional grid data are aligned to the same coordinate space.
In one embodiment, the computer program, when executed by a processor, implements aligning all sample three-dimensional mesh data to the same coordinate space, comprising: all sample three-dimensional mesh data are aligned to the same coordinate space by an affine transformation technique.
In one embodiment, mapping in three-dimensional surface mesh data to be processed based on location information to extract target features, implemented when a computer program is executed by a processor, comprises: acquiring the closest point corresponding to the template feature marking information in the three-dimensional surface grid data to be processed, and taking the closest point as the feature point corresponding to the target to be processed; or acquiring a set of closest points corresponding to the template feature marking information in the three-dimensional surface grid data to be processed, and taking the set of closest points as a feature line or a feature area corresponding to the target to be processed.
In one embodiment, the computer program, when executed by a processor, for performing surface registration of surface template mesh data in three-dimensional template data with processed three-dimensional surface mesh data to establish correspondence of the surface template mesh data and the three-dimensional surface mesh data to be processed, comprises: taking the surface template grid data in the three-dimensional template data as floating grid data, taking the three-dimensional surface grid data to be processed as target grid data, and performing surface registration on the floating grid data to the target grid data; wherein the surface registration comprises a correspondence of the floating grid data to points in the target grid data and a processing of the corresponding points.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include at least one of non-volatile and volatile memory. Non-volatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical storage, or the like. Volatile Memory can include Random Access Memory (RAM) or external cache Memory. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims (11)

1. A method for extracting target features based on surface registration, the method comprising:
acquiring to-be-processed three-dimensional surface grid data corresponding to a to-be-processed target;
acquiring pre-generated three-dimensional template data corresponding to the target to be processed;
performing surface registration on the surface template grid data in the three-dimensional template data and the to-be-processed three-dimensional surface grid data to establish a corresponding relation between the surface template grid data and the to-be-processed three-dimensional surface grid data;
reading three-dimensional template feature labeling information from the three-dimensional template data; and mapping the three-dimensional template feature marking information to the three-dimensional surface mesh data to be processed according to the corresponding relation so as to extract the target feature in the three-dimensional surface mesh data to be processed.
2. The method of claim 1, wherein the three-dimensional template data is generated in a manner comprising:
acquiring three-dimensional surface grid data of a sample;
carrying out surface registration on the sample three-dimensional surface grid data to obtain surface template grid data;
and marking the features on the surface template grid data to obtain three-dimensional template data.
3. The method of claim 2, wherein labeling the features on the surface template grid data to obtain three-dimensional template data comprises:
recording the serial number or the point set serial number group of the points corresponding to the target feature structure on the surface grid data of the three-dimensional template;
and obtaining three-dimensional template data according to the serial numbers of the points or the serial number groups of the point sets and the surface template grid data.
4. The method of claim 2, wherein surface registering the sample three-dimensional surface mesh data to obtain surface template mesh data comprises:
acquiring corresponding points in the three-dimensional surface grid data of each sample;
and processing the corresponding points according to a preset rule to obtain the surface template grid data.
5. The method of claim 4, wherein prior to obtaining the corresponding point in each of the sample three-dimensional surface mesh data, further comprising:
all sample three-dimensional grid data are aligned to the same coordinate space.
6. The method of claim 5, wherein the aligning all sample three-dimensional mesh data to the same coordinate space comprises:
all sample three-dimensional mesh data are aligned to the same coordinate space by an affine transformation technique.
7. The method according to any one of claims 1 to 5, wherein the mapping the three-dimensional template feature labeling information to the to-be-processed three-dimensional surface mesh data according to the corresponding relationship to extract a target feature in the to-be-processed three-dimensional surface mesh data comprises:
acquiring the closest point corresponding to the template feature marking information in the three-dimensional surface grid data to be processed, and taking the closest point as a feature point corresponding to a target to be processed; or
And acquiring a set of closest points corresponding to the template feature marking information in the three-dimensional surface grid data to be processed, and taking the set of closest points as a feature line or a feature area corresponding to a target to be processed.
8. The method according to any one of claims 1 to 6, wherein the surface registering the surface template mesh data in the three-dimensional template data and the three-dimensional surface mesh data to be processed comprises:
taking surface template grid data in the three-dimensional template data as floating grid data, taking the three-dimensional surface grid data to be processed as target grid data, and performing surface registration on the floating grid data to the target grid data; the surface registration includes a correspondence of the floating grid data to points in the target grid data and a processing of the corresponding points.
9. An apparatus for extracting a target feature based on surface registration, the apparatus comprising:
the system comprises a to-be-processed data acquisition module, a processing module and a processing module, wherein the to-be-processed data acquisition module is used for acquiring to-be-processed three-dimensional surface grid data corresponding to a to-be-processed target;
the template data acquisition module is used for acquiring pre-generated three-dimensional template data corresponding to the target to be processed;
the first surface registration module is used for carrying out surface registration on the surface template grid data in the three-dimensional template data and the three-dimensional surface grid data to be processed so as to establish the corresponding relation between the surface template grid data and the three-dimensional surface grid data to be processed;
the mapping module is used for reading three-dimensional template feature marking information from the three-dimensional template data; and mapping the three-dimensional template feature marking information to the three-dimensional surface mesh data to be processed according to the corresponding relation so as to extract the target feature in the three-dimensional surface mesh data to be processed.
10. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any of claims 1 to 8.
11. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 8.
CN202110229166.0A 2021-03-02 2021-03-02 Target feature extraction method, device, equipment and medium based on surface registration Active CN112950684B (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202310975174.9A CN117058409A (en) 2021-03-02 2021-03-02 Target feature extraction method, device, equipment and medium based on surface registration
CN202110229166.0A CN112950684B (en) 2021-03-02 2021-03-02 Target feature extraction method, device, equipment and medium based on surface registration

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110229166.0A CN112950684B (en) 2021-03-02 2021-03-02 Target feature extraction method, device, equipment and medium based on surface registration

Related Child Applications (1)

Application Number Title Priority Date Filing Date
CN202310975174.9A Division CN117058409A (en) 2021-03-02 2021-03-02 Target feature extraction method, device, equipment and medium based on surface registration

Publications (2)

Publication Number Publication Date
CN112950684A true CN112950684A (en) 2021-06-11
CN112950684B CN112950684B (en) 2023-07-25

Family

ID=76247165

Family Applications (2)

Application Number Title Priority Date Filing Date
CN202310975174.9A Pending CN117058409A (en) 2021-03-02 2021-03-02 Target feature extraction method, device, equipment and medium based on surface registration
CN202110229166.0A Active CN112950684B (en) 2021-03-02 2021-03-02 Target feature extraction method, device, equipment and medium based on surface registration

Family Applications Before (1)

Application Number Title Priority Date Filing Date
CN202310975174.9A Pending CN117058409A (en) 2021-03-02 2021-03-02 Target feature extraction method, device, equipment and medium based on surface registration

Country Status (1)

Country Link
CN (2) CN117058409A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115100258A (en) * 2022-08-29 2022-09-23 杭州三坛医疗科技有限公司 Hip joint image registration method, device, equipment and storage medium
WO2023078309A1 (en) * 2021-11-05 2023-05-11 苏州微创畅行机器人有限公司 Method and apparatus for extracting target feature point, and computer device and storage medium

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080123927A1 (en) * 2006-11-16 2008-05-29 Vanderbilt University Apparatus and methods of compensating for organ deformation, registration of internal structures to images, and applications of same
CN102982552A (en) * 2012-12-14 2013-03-20 中国科学院自动化研究所 Surface registration method based on ricci flow
CN103258349A (en) * 2013-05-30 2013-08-21 西北大学 Model base for craniofacial reconstruction and craniofacial reconstruction method
KR20160049639A (en) * 2014-10-28 2016-05-10 한국기계연구원 Stereoscopic image registration method based on a partial linear method
CN106846330A (en) * 2016-12-22 2017-06-13 浙江大学宁波理工学院 Human liver's feature modeling and vascular pattern space normalizing method
US20180218507A1 (en) * 2015-08-14 2018-08-02 Thomson Licensing 3d reconstruction of a human ear from a point cloud

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080123927A1 (en) * 2006-11-16 2008-05-29 Vanderbilt University Apparatus and methods of compensating for organ deformation, registration of internal structures to images, and applications of same
CN102982552A (en) * 2012-12-14 2013-03-20 中国科学院自动化研究所 Surface registration method based on ricci flow
CN103258349A (en) * 2013-05-30 2013-08-21 西北大学 Model base for craniofacial reconstruction and craniofacial reconstruction method
KR20160049639A (en) * 2014-10-28 2016-05-10 한국기계연구원 Stereoscopic image registration method based on a partial linear method
US20180218507A1 (en) * 2015-08-14 2018-08-02 Thomson Licensing 3d reconstruction of a human ear from a point cloud
CN106846330A (en) * 2016-12-22 2017-06-13 浙江大学宁波理工学院 Human liver's feature modeling and vascular pattern space normalizing method

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023078309A1 (en) * 2021-11-05 2023-05-11 苏州微创畅行机器人有限公司 Method and apparatus for extracting target feature point, and computer device and storage medium
CN115100258A (en) * 2022-08-29 2022-09-23 杭州三坛医疗科技有限公司 Hip joint image registration method, device, equipment and storage medium

Also Published As

Publication number Publication date
CN117058409A (en) 2023-11-14
CN112950684B (en) 2023-07-25

Similar Documents

Publication Publication Date Title
CN112967236B (en) Image registration method, device, computer equipment and storage medium
EP3828825A1 (en) Image segmentation method and apparatus, diagnosis system, storage medium, and computer device
Trinh et al. Novel example-based method for super-resolution and denoising of medical images
US8457373B2 (en) System and method for robust 2D-3D image registration
CN111862249A (en) System and method for generating canonical imaging data for medical image processing using deep learning
CN111080573B (en) Rib image detection method, computer device and storage medium
US20110058720A1 (en) Systems and Methods for Automatic Vertebra Edge Detection, Segmentation and Identification in 3D Imaging
JP2009520558A (en) Point-based adaptive elasticity image registration
JP2004509723A (en) Image registration system and method with cross entropy optimization
JP2007054636A (en) Method for positioning a pair of images and program storing apparatus for executing above method by realizing program comprised of command executed by computer
CN112950684B (en) Target feature extraction method, device, equipment and medium based on surface registration
CN111488872B (en) Image detection method, image detection device, computer equipment and storage medium
CN111583184A (en) Image analysis method, network, computer device, and storage medium
CN113129418B (en) Target surface reconstruction method, device, equipment and medium based on three-dimensional image
CN116071404A (en) Image registration method, device, computer equipment and storage medium
Xiao et al. An improved 3D shape context based non-rigid registration method and its application to small animal skeletons registration
Tang et al. Retinal image registration based on robust non-rigid point matching method
CN111192268A (en) Medical image segmentation model construction method and CBCT image bone segmentation method
CN117911432A (en) Image segmentation method, device and storage medium
CN113393498B (en) Image registration method, device, computer equipment and storage medium
CN114913287B (en) Three-dimensional human body model reconstruction method and system
Heinrich et al. Textural mutual information based on cluster trees for multimodal deformable registration
CN112862975B (en) Bone data processing method, system, readable storage medium and device
CN113962957A (en) Medical image processing method, bone image processing method, device and equipment
Feigin et al. Statistical consensus matching framework for image registration

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant