CN115705640A - Automatic registration method, device and application for local rigid part of image - Google Patents
Automatic registration method, device and application for local rigid part of image Download PDFInfo
- Publication number
- CN115705640A CN115705640A CN202110928176.3A CN202110928176A CN115705640A CN 115705640 A CN115705640 A CN 115705640A CN 202110928176 A CN202110928176 A CN 202110928176A CN 115705640 A CN115705640 A CN 115705640A
- Authority
- CN
- China
- Prior art keywords
- image
- registered
- rigid part
- local rigid
- local
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Image Analysis (AREA)
Abstract
The application provides an automatic registration method, an automatic registration device and application for local rigid parts of an image, wherein the method comprises the following steps: acquiring a group of same-position reference images and images to be registered, wherein the reference images and the images to be registered correspondingly comprise one or more local rigid positions; respectively inputting the reference image and the image to be registered into a key point detection model, and acquiring the key points with the same name of the corresponding target local rigid part in the reference image and the image to be registered, wherein the key point detection model is obtained based on the training of the target local rigid part; establishing a matching point pair set based on the key points with the same name, and establishing a transformation matrix according to the matching point pair set; and registering the target local rigid part according to the transformation matrix. According to the method, the specific part in the image to be registered and the reference image are registered by a local rigid registration method according to the transformation matrix, and the method has the advantages of accurate registration and small model pressure.
Description
Technical Field
The present application relates to the field of medical image processing, and in particular, to an automatic registration method, apparatus and application for local rigid parts of an image.
Background
When medical image analysis is performed, several images of the same patient are often required to be put together for analysis, so that comprehensive information of the patient in various aspects is obtained, and the medical diagnosis and treatment level is improved. Due to the shooting angle of a part of medical images, shooting equipment or other interference factors, the obtained medical images of the same part may have differences, that is, the medical images of the same part are not aligned, so that quantitative analysis is performed on several medical images of the same part, and firstly, the problem of strict alignment of the several medical images is solved.
The medical image registration is a basic subject of medical image analysis, and has very important significance for determining the position of a focus, analyzing the condition of the focus and making an optimal operation scheme, so that the medical image registration has important theoretical research and clinical application values. Specifically, the medical image registration refers to finding a spatial transformation for one medical image to make it spatially consistent with a corresponding point on another medical image, and commonly used spatial transformation methods include rigid transformation, affine transformation, projective transformation and nonlinear transformation, where the rigid transformation is that the distance between any two points in the object is kept constant. Conventional medical image registration methods can be divided into two categories: the method is characterized by comprising the following steps of firstly, determining rigid transformation parameters by detecting correlation and gray value difference, and carrying out registration by using a simple image to assist diagnosis; the other is a feature-based registration method, which is to select feature points on a two-dimensional image manually or automatically by a computer and achieve the purpose of image registration by adopting rigid transformation.
The registration method based on gray scale has high automation degree, but does not directly consider image shape information, and the calculation speed is relatively slow. The method based on the characteristics depends on a characteristic extraction method, is high in calculation speed and suitable for rigid registration and some deformation registration, however, the traditional rigid registration generally adopts the traditional CV algorithms such as an SIFT operator and an ORB operator to extract key points, and searches for characteristic point pairs through matching of the characteristics of the key points, for example, chinese patent CN110874849A provides a non-rigid point set registration method based on local transformation consistency.
In addition, the chinese patent CN112819867A also provides a fundus image registration method based on a key point matching network, the method adopts a full image registration mode, and the corresponding scheme adopts a mode of directly predicting rigid transformation parameters by using a rigid registration network training model, which has the problems that the model itself has a more complex architecture, the corresponding model calculation pressure/training pressure is very large, the registration effect of the traditional full image rigid registration on the vertebral body image is not good, and the operation pressure is very large. In summary, the conventional full-image registration or local non-rigid registration method cannot well solve the problem of local rigid registration, and particularly in bone structure registration, based on the conventional feature point method, since the extracted local features of key points are too similar to the image features of adjacent vertebral bodies, mismatching is easy to occur, and calculation is complicated and time-consuming, the method tries to output the designated key points of the designated vertebral bodies end to end through a key point detection model based on a convolutional neural network, and can obtain the local rigid transformation relation between the same vertebral bodies without additional calculation in a way of matching point pairs with the same name. The method provided by the invention can be used for conveniently and automatically measuring the biological dynamic index of the vertebral motion position.
Disclosure of Invention
In a first aspect, an embodiment of the present application provides an automatic registration method for a local rigid part of an image, which automatically extracts key points of a same-name vertebral body from a reference image and an image to be registered by using a neural network model and establishes a matching point pair set, and registers a specific part in the image to be registered and the reference image by using a local rigid registration method according to a transformation matrix.
Specifically, the method comprises the following steps:
acquiring a group of same-position reference images and images to be registered, wherein the reference images and the images to be registered correspondingly comprise one or more local rigid positions; respectively inputting the reference image and the image to be registered into a key point detection model, and acquiring the homonymous key points of the corresponding target local rigid part in the reference image and the image to be registered, wherein the key point detection model is obtained by training based on the target local rigid part; establishing a matching point pair set based on the key points with the same name, and establishing a transformation matrix according to the matching point pair set; and registering the target local rigid part according to the transformation matrix.
In some embodiments, the image to be registered is detected by the key point detection model to obtain a key point to be registered corresponding to a target local rigid portion, the reference image is detected by the key point detection model to obtain a reference key point corresponding to the target local rigid portion, and the key point to be registered and the reference key point corresponding to the same position of the target local rigid portion are the same-name key points.
In some embodiments, the keypoint detection model intercepts a local reference image in the reference image corresponding to the target local rigid region and outputs the reference keypoints corresponding to the target local rigid region based on the local reference image; the key point detection model intercepts a local image to be registered corresponding to the target local rigid part in the image to be registered, and outputs the key point to be registered corresponding to the target local rigid part based on the local image to be registered; and the key point to be registered and the reference key point which correspond to the same position of the target local rigid part are the key points with the same name.
In some embodiments, the keypoint detection model outputs a reference segmentation result corresponding to the target local rigid part in the reference image, and intercepts the local reference image from the reference image after binarizing the reference segmentation result; and the key point detection model outputs a segmentation result to be registered corresponding to the target local rigid part in the image to be registered, and intercepts the local image to be registered from the image of the part to be registered after binarizing the segmentation result to be registered.
In some embodiments, the keypoint detection model outputs a reference thermodynamic diagram of the local reference image corresponding to the target local rigid part, and the reference keypoints are obtained from the reference thermodynamic diagram; and the key point detection model outputs a thermodynamic diagram to be registered corresponding to the target local rigid part in the local image to be registered, and the key points to be registered are obtained from the thermodynamic diagram to be registered.
In some embodiments, when the local rigid part is a cervical vertebra, the reference image is a cervical vertebra median lateral bitmap, and the image to be registered is a cervical vertebra hyperextension lateral bitmap or a cervical vertebra over flexion lateral bitmap.
In some embodiments, a registered image is obtained after registering the target local rigid site, and a biodynamic index is calculated according to the registered image and the reference image, wherein the biodynamic index at least comprises a site activity degree and a site rotation angle.
In a second aspect, an automatic registration apparatus for local rigid part of image is provided, which includes:
the device comprises an acquisition module, a registration module and a registration module, wherein the acquisition module is used for acquiring a group of reference images with the same position and images to be registered, and the reference images and the images to be registered correspondingly comprise one or more local rigid positions;
the key point detection module is used for respectively inputting the reference image and the image to be registered into a key point detection model and acquiring the key points with the same name of the corresponding target local rigid part in the reference image and the image to be registered, wherein the key point detection model is obtained based on the training of the target local rigid part;
the computing module is used for establishing a matching point pair set based on the key points with the same name and establishing a transformation matrix according to the matching point pair set;
and the registration module is used for registering the target local rigid part according to the transformation matrix.
In a third aspect, the present application provides an electronic apparatus, including a memory and a processor, where the memory stores a computer program, and the processor is configured to execute the computer program to perform the automatic registration method for local rigid parts of images as described in any of the above embodiments.
In a fourth aspect, an embodiment of the present application provides a computer program product, where the computer program product includes: a program or instructions which, when run on a computer, causes the computer to perform the method for automatic registration of a locally rigid portion of an image as described in any of the embodiments above.
In a fifth aspect, the present application provides a readable storage medium having stored therein a computer program comprising program code for controlling a process to execute a process, the process comprising an automatic registration method for local rigid regions of an image according to any one of the embodiments described above.
The method comprises the steps of automatically registering key points of a centrum with the same name on a reference image and an image to be registered through a trained neural network model, establishing a matching point pair set, establishing a transformation matrix based on the matching point pair set, and registering a specific part on the image to be registered to the reference image by a local rigid registration method.
It is worth mentioning that the traditional rigid registration method generally adopts the traditional cv algorithm such as SIFT operator and ORB operator to extract key points, and searches matching point pairs through the matching of key point features; in the embodiment of the application, the key point detection model based on the convolutional neural network is adopted to extract the key points, and the model learns the characteristics of the designated positions, so that the two extracted images can automatically obtain homonymy point pairs. In addition, the key point detection model in the embodiment of the application adopts a cascade network architecture, the first-stage convolutional neural network firstly segments the image to be detected, a specified part image is intercepted from the image, and then the second-stage convolutional neural network is used for detecting the key points, so that the attention of the neural network and the accuracy of key point detection can be improved.
The details of one or more embodiments of the application are set forth in the accompanying drawings and the description below to provide a more concise and understandable description of the application, and features, objects, and advantages of the application.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:
FIG. 1 is a flow chart of an automatic registration method for local rigid regions of an image according to an embodiment of the application;
FIG. 2 is a reference image and an image to be registered according to an embodiment of the application;
FIG. 3 is a schematic diagram of a keypoint detection model according to an embodiment of the present application;
FIG. 4 is a schematic view of a homonymous vertebral registration in accordance with an embodiment of the application;
FIG. 5 is a block diagram of an automatic registration apparatus for local rigid regions of an image according to an embodiment of the present application;
fig. 6 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present application.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. The following description refers to the accompanying drawings in which the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of the present specification. Rather, they are merely examples of apparatus and methods consistent with certain aspects of one or more embodiments of the specification, as detailed in the claims which follow.
It should be noted that: in other embodiments, the steps of the corresponding methods are not necessarily performed in the order shown and described herein. In some other embodiments, the methods may include more or fewer steps than those described herein. Moreover, a single step described in this specification may be broken down into multiple steps for description in other embodiments; multiple steps described in this specification may be combined into a single step in other embodiments.
Example one
In this embodiment, a cervical vertebrae medial-lateral X-ray image taken for cervical spondylosis is taken as an example. The arrangement of vertebral body facet joints, whether the physiological radian of the cervical vertebra is normal or not, and the narrow part and degree of the intervertebral space can be observed through the cervical vertebra median lateral bitmap, and whether osteophyte hyperplasia exists at the rear edge of the vertebral body or not can be particularly noticed.
When needed, the X-ray photographic image of the dynamic side position of the cervical vertebra can be shot, the dynamic side position of the cervical vertebra comprises two positions of the hyperextension side position of the cervical vertebra and the over-flexion side position of the cervical vertebra, namely, the hyperextension side position map of the cervical vertebra and the over-flexion side position map of the cervical vertebra are obtained. Wherein, the elastic change condition of the intervertebral disc can be seen in the cervical vertebra hyperextension lateral diagram and the cervical vertebra over-flexion lateral diagram.
In order to determine the lesion position, analyze the lesion condition, or specify an operation plan, in this embodiment, it is necessary to use the cervical spine median lateral bitmap as a reference image, use the cervical spine hyperextension lateral bitmap or the cervical spine hyperflexion lateral bitmap as an image to be registered, and locally register at least one vertebral body in the image to be registered as a target local rigid portion to the reference image.
The embodiment provides an automatic registration method for local rigid parts of images, which extracts key points of centrums with the same name from a reference image and an image to be registered by using a neural network model, and because the neural network model of the scheme learns the characteristics of specific parts, the key points extracted by the neural network model of the scheme are also corresponding, so that a matching point pair set can be directly established based on the key points of the centrums with the same name, matching according to the characteristics of the key points is not needed as in the prior art, then a transformation matrix is established based on the matching point pair set, and at least one centrum in the image to be registered is registered to the reference image by using a local rigid registration method according to the transformation matrix.
Referring to fig. 1, fig. 1 is a flowchart of an automatic registration method for local rigid sites of an image according to an embodiment of the present application. It should be noted that the specific example of the present solution is described by taking the medical image as the cervical dynamic lateral X-ray photographic image, but this is not the only limitation.
As shown in fig. 1, the method comprises steps S1-S4:
step S1: acquiring a group of same-position reference images and images to be registered, wherein the reference images and the images to be registered correspondingly comprise one or more local rigid positions.
The local rigid part is a vertebral body and can be a spine or a cervical vertebra, correspondingly, if the local rigid part is the cervical vertebra, the medical image can be a dynamic lateral position X-ray photographic image of the cervical vertebra, the cervical vertebra refers to cervical vertebra, the part below the head and above the thoracic vertebra, the cervical vertebra is positioned in a cervical spine, and the total number of the parts is 7, and a specific vertebral body in the cervical vertebra is selected as the target local rigid part in the embodiment. At this time, the reference image at least comprises a reference cone corresponding to the target local rigid part, and the image to be registered at least comprises a cone to be registered corresponding to the target local rigid part. The meaning of the vertebral body to be registered and the reference vertebral body is explained as follows: the reference vertebral body and the vertebral body to be registered are the same vertebral body, that is, the target local rigid part in the embodiment, and the reference vertebral body and the vertebral body to be registered are distinguished by names only because the reference vertebral body and the vertebral body to be registered are displayed as different angular positions in different images.
In the step, a cervical vertebra median lateral bitmap of the same patient is obtained as a reference image, and a cervical vertebra hyperextension lateral bitmap or a cervical vertebra hyperflexion lateral bitmap of the patient is obtained as an image to be registered. Referring to fig. 2, fig. 2 is a reference image and an image to be registered according to an embodiment of the present application. As shown in fig. 2, the left image is a reference image, and the right image is an image to be registered, as described above, since the reference vertebral body and the vertebral body to be registered of the present solution are the same vertebral body, the key points corresponding to the same vertebral body are the same, it can be seen that the C7 vertebral body in the left image correspondingly contains 7 key points, the C7 vertebral body in the right image also contains 7 corresponding key points, and each pair of key points is corresponding. Just because the key points of the same centrum are detected, the key points with the same name in the two images can be automatically extracted in the key point detection model subsequently.
It should be noted that, the "same-location reference image and image to be registered" referred to in step S1 means that the reference image and the image to be registered are from the same source, that is, from the same patient, and are directed to the same location, that is, the cervical vertebra of the patient in this embodiment.
Step S2: and respectively inputting the reference image and the image to be registered into a key point detection model, and acquiring the key points with the same name of the corresponding target local rigid part in the reference image and the image to be registered, wherein the key point detection model is obtained based on the training of the target local rigid part.
It should be noted that, as described above, the reference cone and the to-be-registered cone in the reference image and the to-be-registered image are substantially the same cone, so that the key points corresponding to the same cone, that is, the key points corresponding to the local rigid portion of the target, can be obtained when the reference image and the to-be-registered image are respectively input into the key point detection model. The image to be registered is detected by the key point detection model to obtain key points to be registered corresponding to the target local rigid part, the reference image is detected by the key point detection model to obtain reference key points corresponding to the target local rigid part, and the key points to be registered and the reference key points corresponding to the same position of the target local rigid part are the key points with the same name.
In addition, it is worth mentioning that the key point detection model of the scheme adopts the cascaded double-U-shaped convolutional neural network, and the key point detection model has the advantages that the interference of the thoracic vertebra and the skull on the detection of the key points of the vertebral body can be shielded, the attention function is achieved, and the key point detection in the actual scene is more accurate and robust.
Correspondingly, the step S2 further comprises the following steps:
the key point detection model intercepts a local reference image corresponding to the target local rigid part in the reference image, and outputs the reference key point corresponding to the target local rigid part based on the local reference image;
the key point detection model intercepts a local image to be registered corresponding to the target local rigid part in the image to be registered, and outputs the key point to be registered corresponding to the target local rigid part based on the local image to be registered;
and the key point to be registered and the reference key point corresponding to the same position of the target local rigid part are homonymous key points.
In the step of intercepting a local reference image corresponding to the target local rigid part in the reference image by the key point detection model, the key point detection model outputs a reference segmentation result corresponding to the target local rigid part in the reference image, and intercepts the local reference image from the reference image after binarizing the reference segmentation result.
In the step of intercepting the local image to be registered corresponding to the target local rigid part in the image to be registered by the key point detection model, the key point detection model outputs the segmentation result to be registered corresponding to the target local rigid part in the image to be registered, and intercepts the local image to be registered from the image to be registered after binarizing the segmentation result to be registered.
And the processing mode of the binarization of the reference segmentation result is specifically as follows: and binarizing the reference segmentation result into a binary image, and cutting the original reference image according to a circumscribed rectangular frame of the binary image to obtain the local reference image.
Correspondingly, the processing method of the binarization of the segmentation result to be registered specifically comprises the following steps of: and binarizing the segmentation result to be registered into a binary image, and cutting the original image to be registered according to the circumscribed rectangle frame of the binary image to obtain the local image to be registered.
The method has the advantages that other interference parts can be shielded through the centrum segmentation of the key point detection model, so that the detection of key points in the later period is always concentrated on the detection part, and the detection of the key points is more accurate.
In "outputting the reference keypoints corresponding to the target local rigid part based on the local reference image", the keypoint detection model outputs a reference thermodynamic diagram corresponding to the target local rigid part in the local reference image, and the reference keypoints are acquired from the reference thermodynamic diagram.
Similarly, in the step of outputting the key points to be registered corresponding to the target local rigid part based on the local image to be registered, the key point detection model outputs a thermodynamic diagram to be registered corresponding to the target local rigid part in the local image to be registered, and the key points to be registered are obtained from the thermodynamic diagram to be registered.
Correspondingly, in this embodiment, the keypoint detection model includes a first-stage convolutional neural network, a second-stage convolutional neural network, and a keypoint output layer. In particular, in this embodiment, the first stage convolutional neural network and the second stage convolutional neural network are both U-shaped convolutional neural networks. Referring specifically to fig. 3, fig. 3 is a schematic diagram of a keypoint detection model according to an embodiment of the present application.
As shown in fig. 3, first, a reference image or an image to be registered is input into a first-stage convolutional neural network as an image to be detected, the first-stage convolutional neural network is a vertebral body segmentation network, a position region of a vertebral body is extracted by the vertebral body segmentation network, a single-channel vertebral body semantic segmentation result is correspondingly output, an output result of the first-stage convolutional neural network is binarized, and then a vertebral body partial image is obtained from the image to be detected according to the position region of the vertebral body, namely, a circumscribed matrix of a cut binary image.
And then inputting the partial image of the cone into a second-stage convolutional neural network, and outputting cone key point thermodynamic diagrams, wherein each cone key point thermodynamic diagram represents a same locus. For example, the vertebral body partial image comprises 7 vertebral bodies named C1-C7, and then a point at the upper left corner of each vertebral body is a key point with the same name, and similarly, a point at the lower left corner, a point at the upper right corner and a point at the lower right corner of each vertebral body are key points with the same name respectively.
And finally, performing Gaussian fitting on the cone key point thermodynamic diagrams on a key point output layer, and marking the positions of extreme values of the cone key point thermodynamic diagrams as cone key points of all cones in the image to be detected.
In this step, the reference image is input into the key point detection model to obtain key points of the reference vertebral body, that is, the reference key points, the image to be registered is input into the key point detection model to obtain key points of the vertebral body to be registered, that is, the key points to be registered, and the reference key points and the key points to be registered are both key points for the target local rigid portion in this embodiment.
And step S3: and establishing a matching point pair set based on the key points with the same name, and establishing a transformation matrix according to the matching point pair set.
Based on the corresponding relationship between the key points with the same name obtained in step S2, a matching point pair set can be automatically established. It is worth mentioning that the matching point pair set at least includes two sets of key points with the same name, and preferably, the number of matched key points with the same name in the matching point pair set is not less than 4, which is beneficial to ensure accurate identification of the local rigid part.
That is to say, the homologous key points at the same position of the local rigid part of the target in the reference image and the image to be registered are a set of matching points, and the matching point pair set comprises at least two matching points.
This is one of the bright spots that is different from the prior art, and the present solution performs the key point detection on the same portion, so that the matching point pair set is obtained without performing the feature matching after obtaining the key point, but the matching point pair set can be directly and automatically generated. It is worth explaining that the effect is of great significance for matching of key points of the spine and the cervical vertebra, the key points of the spine and the cervical vertebra have the characteristics of being small, numerous and dense, a plurality of feature key points correspond to the same vertebral body, and the distance between every two feature key points is small, so that the problems of low efficiency and low matching precision are caused if a manual distinguishing and matching mode is adopted.
Still taking fig. 2 as an example, the local rigid part is taken as the vertebral body C7, and the detected vertebral body C7 includes seven key points, i.e., key point No. 1, key point No. 2, key point No. 3, key point No. 4, key point No. 5, key point No. 6, and key point No. 7. At this time, since the key point detection model substantially detects the same part, that is, the target local rigid part in this embodiment, the reference key point and the key point to be registered are in a one-to-one correspondence relationship, as shown in the left diagram of fig. 2, the above 7 key points will be obtained in the reference image input key point detection model, as shown in the right diagram of fig. 2, the above 7 key points will also be obtained in the key point detection model to be registered, and the corresponding 2 number 1 key points in the two diagrams are the key points with the same name, and the 2 key points with the same name form a group.
It should be noted that the present solution is applicable to a local rigid portion conforming to the rigid transformation characteristic, that is, the distance between the front point and the rear point remains unchanged after the local rigid portion undergoes rigid transformation.
Specifically, in step S3, three parameters in the transformation matrix are calculated based on the matching point pair set, and then the transformation matrix is applied to transform the coordinates of all pixels in the image to be registered, so that the transformation result can be registered with the reference image.
After the abnormal centrum key point pairs are removed through random sampling consistency, the matching point pair set calculates transformation matrix parameters according to the set error tolerance (maximum fitting error) of the matching point pairs and the quantity constraint of the minimum matching point pairs.
In this embodiment, since there are 3 parameters in the transformation matrix, that is, the horizontal offset, the vertical offset, and the rotation angle in the transformation matrix are unknown, in order to ensure that there is a solution for calculating the parameters of the transformation matrix, the number of the minimum matching point pairs is 4, and the vertical offset, the horizontal offset, and the rotation angle are calculated, where the transformation matrix is:
firstly, respectively substituting 4 groups of matching points into the transformation matrix, namely X 'represents an X-axis coordinate in a key point to be registered, Y' represents a Y-axis coordinate in the key point to be registered, X represents an X-axis coordinate of a reference key point in the same group, Y represents a Y-axis coordinate of the reference key point in the same group, deltax represents a horizontal offset, deltay represents a vertical offset, and theta represents a rotation angle. Solving transformation matrix parameters in the transformation matrix: horizontal offset, vertical offset, and rotation angle.
And step S4: and registering the target local rigid part according to the transformation matrix.
And after calculating the matrix transformation parameters, substituting each pixel coordinate in the image to be registered into the transformation matrix for coordinate transformation to obtain the changed coordinate. At this time, X represents the X-axis coordinate of the pixel in the image to be registered, Y represents the Y-axis coordinate of the corresponding pixel in the image to be registered, and X 'and Y' of the transformed coordinates are calculated. Specifically, the coordinates of each pixel in the image to be registered are obtained, and coordinate transformation is performed according to the transformation matrix and the transformation matrix parameters, wherein the transformation matrix parameters are obtained according to key points of the target local rigid part, so that the registration is performed only on the target local rigid part in the image to be registered.
In this step, referring to fig. 4, fig. 4 is a schematic view of the same-name vertebral body registration according to an embodiment of the present application. As shown in fig. 4, the C7 cervical vertebral bodies in the image to be registered and the C7 cervical vertebral bodies in the reference image are registered in one-to-one correspondence according to the vertebral body key points.
In other embodiments, the biodynamic measurement index of the dynamic lateral position of the cervical vertebrae can be calculated according to the registered image as shown in fig. 4. Such as the mobility of the cervical vertebrae, the rotation angle of the vertebral body, etc. For example: the angle formed by connecting lines of a point at the upper right corner of the C3 vertebral body in the cervical vertebra hyperextension side bitmap, a point at the upper right corner of the C3 vertebral body in the cervical vertebra hyperflexion side bitmap and a central point corresponding to the C1 vertebral body is the activity of the vertebral body. The specific other biodynamic measurement indicators can be calculated according to the existing literature, and are not described herein. The biodynamic measurement indexes can be used for analyzing the cervical vertebra movement range of a patient, evaluating the mobility of the whole or the segment of the vertebral body, and facilitating understanding of cervical vertebra pathology of the patient and improvement of surgical treatment of cervical vertebra degenerative diseases.
Example two
Based on the same concept, the present embodiment further provides an automatic registration apparatus for an image local rigid part, and referring to fig. 5, fig. 5 is a structural block diagram of the automatic registration apparatus for an image local rigid part according to an embodiment of the present application. The device realizes the automatic registration method for the local rigid part of the image, and comprises the following steps:
the device comprises an acquisition module, a registration module and a registration module, wherein the acquisition module is used for acquiring a group of reference images with the same position and images to be registered, and the reference images and the images to be registered correspondingly comprise one or more local rigid positions;
the key point detection module is used for respectively inputting the reference image and the image to be registered into a key point detection model and acquiring the key points with the same name of the corresponding target local rigid part in the reference image and the image to be registered, wherein the key point detection model is obtained based on the training of the target local rigid part;
the computing module is used for establishing a matching point pair set based on the key points with the same name and establishing a transformation matrix according to the matching point pair set;
and the registration module registers the target local rigid part according to the transformation matrix.
EXAMPLE III
The present embodiment further provides an electronic apparatus, specifically referring to fig. 6, including a memory 304 and a processor 302, where the memory 304 stores a computer program, and the processor 302 is configured to execute the computer program to perform any one of the above-mentioned automatic registration methods for a local rigid portion of an image.
Specifically, the processor 302 may include a Central Processing Unit (CPU), or A Specific Integrated Circuit (ASIC), or may be configured to implement one or more Integrated circuits of the embodiments of the present Application.
The memory 304 may be used to store or cache various initialization data files that need to be processed and/or used for communication, as well as possibly computer program instructions executed by the processor 302.
The processor 302 reads and executes the computer program instructions stored in the memory 304 to implement any one of the above-mentioned automatic registration methods for local rigid portions of images.
Optionally, the electronic apparatus may further include a transmission device 306 and an input/output device 308, where the transmission device 306 is connected to the processor 302, and the input/output device 308 is connected to the processor 302.
The transmitting device 306 may be used to receive or transmit data via a network. Specific examples of the network described above may include a wired or wireless network provided by a communication provider of the electronic device. In one example, the transmission device includes a Network adapter (NIC) that can be connected to other Network devices through a base station so as to communicate with the internet. In one example, the transmitting device 306 can be a Radio Frequency (RF) module, which is used to communicate with the internet via wireless.
The input/output device 308 is used to input or output information. For example, the input/output device may be a display screen, a mouse, a keyboard, or other devices. In this embodiment, the input device is used to input the acquired information, the input information may be data, tables, images, real-time videos, and the output information may be texts, charts, alarm information, etc. displayed by the service system.
Alternatively, in this embodiment, the processor 302 may be configured to execute the following steps by a computer program:
step S1: acquiring a group of same-position reference images and images to be registered, wherein the reference images and the images to be registered correspondingly comprise one or more local rigid positions;
s2, respectively inputting the reference image and the image to be registered into a key point detection model, and acquiring the key points with the same name of the corresponding target local rigid part in the reference image and the image to be registered, wherein the key point detection model is obtained based on the training of the target local rigid part;
s3, establishing a matching point pair set based on the key points with the same name, and establishing a transformation matrix according to the matching point pair set;
and S4, registering the target local rigid part according to the transformation matrix.
In addition, in combination with any one of the above embodiments, the present application may be implemented as a computer program product. The computer program product includes: a program or instructions which, when run on a computer, causes the computer to perform a method for automatic registration of a localized rigid region of an image implementing any one of the above embodiments.
Moreover, in combination with any one of the foregoing embodiments of the method for automatically registering a local rigid portion of an image, the present application may provide a readable storage medium to implement the method. The readable storage medium having stored thereon a computer program; the computer program, when executed by a processor, implements any one of the above-described embodiments of the method for automatic registration of a locally rigid region of an image.
In general, the various embodiments may be implemented in hardware or special purpose circuits, software, logic or any combination thereof. Some aspects of the invention may be implemented in hardware, while other aspects may be implemented in firmware or software which may be executed by a controller, microprocessor or other computing device, although the invention is not limited thereto. While various aspects of the invention may be illustrated and described as block diagrams, flow charts, or using some other pictorial representation, it is well understood that these blocks, apparatus, systems, techniques or methods described herein may be implemented in, as non-limiting examples, hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controller or other computing devices, or some combination thereof.
Embodiments of the invention may be implemented by computer software executable by a data processor of the mobile device, such as in a processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets and/or macros can be stored in any device-readable data storage medium and they include program instructions for performing particular tasks. The computer program product may include one or more computer-executable components configured to perform embodiments when the program is run. The one or more computer-executable components may be at least one software code or a portion thereof. Further in this regard it should be noted that any block of the logic flow as in the figures may represent a program step, or an interconnected logic circuit, block and function, or a combination of a program step and a logic circuit, block and function. The software may be stored on physical media such as memory chips or memory blocks implemented within the processor, magnetic media such as hard or floppy disks, and optical media such as, for example, DVDs and data variants thereof, CDs. The physical medium is a non-transitory medium.
It should be understood by those skilled in the art that various features of the above embodiments can be combined arbitrarily, and for the sake of brevity, all possible combinations of the features in the above embodiments are not described, but should be considered as within the scope of the present disclosure as long as there is no contradiction between the combinations of the features.
The above examples are merely illustrative of several embodiments of the present application, and the description is more specific and detailed, but not to be construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.
Claims (10)
1. An automatic registration method for local rigid parts of images is characterized by comprising the following steps:
step S1: acquiring a group of same-position reference images and images to be registered, wherein the reference images and the images to be registered correspondingly comprise one or more local rigid positions;
s2, respectively inputting the reference image and the image to be registered into a key point detection model, and acquiring the key points with the same name of the corresponding target local rigid part in the reference image and the image to be registered, wherein the key point detection model is obtained based on the training of the target local rigid part;
s3, establishing a matching point pair set based on the key points with the same name, and establishing a transformation matrix according to the matching point pair set;
and S4, registering the target local rigid part according to the transformation matrix.
2. The automatic registration method for the local rigid part of the image according to claim 1, wherein the image to be registered is detected by the key point detection model to obtain the key point to be registered corresponding to the local rigid part of the target, the reference image is detected by the key point detection model to obtain the reference key point corresponding to the local rigid part of the target, and the key point to be registered and the reference key point corresponding to the same position of the local rigid part of the target are the key points with the same name.
3. The automatic registration method for image local rigid parts according to claim 1, wherein the keypoint detection model intercepts a local reference image of the reference image corresponding to the target local rigid part and outputs the reference keypoints corresponding to the target local rigid part based on the local reference image;
the key point detection model intercepts a local image to be registered corresponding to the target local rigid part in the image to be registered, and outputs the key point to be registered corresponding to the target local rigid part based on the local image to be registered;
and the key point to be registered and the reference key point which correspond to the same position of the target local rigid part are homonymous key points.
4. The automatic registration method for the local rigid part of the image according to claim 3, wherein the keypoint detection model outputs a reference segmentation result corresponding to the target local rigid part in the reference image, and intercepts the local reference image from the reference image after binarizing the reference segmentation result; and the key point detection model outputs a segmentation result to be registered corresponding to the target local rigid part in the image to be registered, and intercepts the local image to be registered from the image of the part to be registered after binarizing the segmentation result to be registered.
5. The automatic registration method for local rigid parts of images according to claim 3, wherein the keypoint detection model outputs a reference thermodynamic diagram corresponding to the target local rigid part in the local reference image, and the reference keypoints are obtained from the reference thermodynamic diagram; and the key point detection model outputs a thermodynamic diagram to be registered corresponding to the target local rigid part in the local image to be registered, and the key points to be registered are obtained from the thermodynamic diagram to be registered.
6. The automatic registration method for the local rigid part of the image according to claim 1, wherein when the local rigid part is the cervical vertebra, the reference image is a cervical vertebra median lateral bitmap, and the image to be registered is a cervical vertebra hyperextension lateral bitmap or a cervical vertebra hyperflexion lateral bitmap.
7. The automatic registration method for the local rigid part of the image according to claim 1, wherein the registration image is obtained after the registration of the local rigid part of the target, and the biodynamic index is calculated according to the registration image and the reference image, wherein the biodynamic index at least comprises a part activity degree and a part rotation angle.
8. An automatic registration device for local rigid parts of images, which is characterized by comprising:
the device comprises an acquisition module, a registration module and a registration module, wherein the acquisition module is used for acquiring a group of reference images with the same position and images to be registered, and the reference images and the images to be registered correspondingly comprise one or more local rigid positions;
the key point detection module is used for respectively inputting the reference image and the image to be registered into a key point detection model and acquiring the key points with the same name of the corresponding target local rigid part in the reference image and the image to be registered, wherein the key point detection model is obtained based on the training of the target local rigid part;
the computing module is used for establishing a matching point pair set based on the key points with the same name and establishing a transformation matrix according to the matching point pair set;
and the registration module registers the target local rigid part according to the transformation matrix.
9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the computer program to perform the method for automatic registration of local rigid parts in images according to any one of claims 1 to 8.
10. A computer program product, the computer program product comprising: program or instructions for causing a computer to carry out the method for automatic registration of a localized rigid region of an image according to any one of claims 1 to 8 when said program or instructions are run on the computer.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110928176.3A CN115705640B (en) | 2021-08-13 | 2021-08-13 | Automatic registration method, device and application for local rigid part of image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110928176.3A CN115705640B (en) | 2021-08-13 | 2021-08-13 | Automatic registration method, device and application for local rigid part of image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115705640A true CN115705640A (en) | 2023-02-17 |
| CN115705640B CN115705640B (en) | 2023-07-07 |
Family
ID=85181119
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110928176.3A Active CN115705640B (en) | 2021-08-13 | 2021-08-13 | Automatic registration method, device and application for local rigid part of image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115705640B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101593351A (en) * | 2008-05-28 | 2009-12-02 | 中国科学院自动化研究所 | Ocular fundus image registration method based on range conversion and rigid transformation parameters estimation |
| CN102208117A (en) * | 2011-05-04 | 2011-10-05 | 西安电子科技大学 | Method for constructing vertebral three-dimensional geometry and finite element mixture model |
| US20150317821A1 (en) * | 2014-04-30 | 2015-11-05 | Seiko Epson Corporation | Geodesic Distance Based Primitive Segmentation and Fitting for 3D Modeling of Non-Rigid Objects from 2D Images |
| CN105869153A (en) * | 2016-03-24 | 2016-08-17 | 西安交通大学 | Non-rigid face image registering method integrated with related block information |
| CN109685838A (en) * | 2018-12-10 | 2019-04-26 | 上海航天控制技术研究所 | Image elastic registrating method based on super-pixel segmentation |
| CN111724364A (en) * | 2020-06-12 | 2020-09-29 | 深圳技术大学 | Method and device based on lung lobes and trachea trees, electronic equipment and storage medium |
| CN112001889A (en) * | 2020-07-22 | 2020-11-27 | 杭州依图医疗技术有限公司 | Medical image processing method and device and medical image display method |
-
2021
- 2021-08-13 CN CN202110928176.3A patent/CN115705640B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101593351A (en) * | 2008-05-28 | 2009-12-02 | 中国科学院自动化研究所 | Ocular fundus image registration method based on range conversion and rigid transformation parameters estimation |
| CN102208117A (en) * | 2011-05-04 | 2011-10-05 | 西安电子科技大学 | Method for constructing vertebral three-dimensional geometry and finite element mixture model |
| US20150317821A1 (en) * | 2014-04-30 | 2015-11-05 | Seiko Epson Corporation | Geodesic Distance Based Primitive Segmentation and Fitting for 3D Modeling of Non-Rigid Objects from 2D Images |
| CN105869153A (en) * | 2016-03-24 | 2016-08-17 | 西安交通大学 | Non-rigid face image registering method integrated with related block information |
| CN109685838A (en) * | 2018-12-10 | 2019-04-26 | 上海航天控制技术研究所 | Image elastic registrating method based on super-pixel segmentation |
| CN111724364A (en) * | 2020-06-12 | 2020-09-29 | 深圳技术大学 | Method and device based on lung lobes and trachea trees, electronic equipment and storage medium |
| CN112001889A (en) * | 2020-07-22 | 2020-11-27 | 杭州依图医疗技术有限公司 | Medical image processing method and device and medical image display method |
Non-Patent Citations (1)
| Title |
|---|
| 刘益含 等: "医学图像配准分类研究", 《计算机科学》, vol. 42, no. 11, pages 22 - 27 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115705640B (en) | 2023-07-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108520519B (en) | Image processing method and device and computer readable storage medium | |
| CN110599508B (en) | Artificial intelligence-based spine image processing method and related equipment | |
| CN110956635B (en) | Lung segment segmentation method, device, equipment and storage medium | |
| Neubert et al. | Automated detection, 3D segmentation and analysis of high resolution spine MR images using statistical shape models | |
| JP7221421B2 (en) | Vertebral localization method, device, device and medium for CT images | |
| EP2472473B1 (en) | Image analysis device | |
| CN102138827B (en) | Image display device | |
| US20200058098A1 (en) | Image processing apparatus, image processing method, and image processing program | |
| US8509502B2 (en) | Spine labeling | |
| Ryan et al. | Registration of digital retinal images using landmark correspondence by expectation maximization | |
| US20180182091A1 (en) | Method and system for imaging and analysis of anatomical features | |
| CN111080573B (en) | Rib image detection method, computer device and storage medium | |
| Fu et al. | Optic disc segmentation by U-net and probability bubble in abnormal fundus images | |
| CN109919903B (en) | Spine detection positioning marking method and system and electronic equipment | |
| CN111932492B (en) | Medical image processing method and device and computer readable storage medium | |
| CN104732520A (en) | Cardio-thoracic ratio measuring algorithm and system for chest digital image | |
| WO2019146358A1 (en) | Learning system, method, and program | |
| Meng et al. | An artificial intelligence powered platform for auto-analyses of spine alignment irrespective of image quality with prospective validation | |
| CN114842003A (en) | Medical image follow-up target pairing method, device and application | |
| Reddy et al. | Anatomical landmark detection using deep appearance-context network | |
| KR100942699B1 (en) | Method and system for extracting distal radius metaphysis | |
| Sha et al. | The improved faster-RCNN for spinal fracture lesions detection | |
| CN115705640B (en) | Automatic registration method, device and application for local rigid part of image | |
| KR101659056B1 (en) | Automated diagnosis system for craniosynostosis using a 2d shape descriptor and automated diagnosis method for craniosynostosis using the same | |
| Maeda et al. | Automatic measurement of the Cobb angle for adolescent idiopathic scoliosis using convolutional neural network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |