CN112001889A - Medical image processing method and device and medical image display method - Google Patents
Medical image processing method and device and medical image display method Download PDFInfo
- Publication number
- CN112001889A CN112001889A CN202010710056.1A CN202010710056A CN112001889A CN 112001889 A CN112001889 A CN 112001889A CN 202010710056 A CN202010710056 A CN 202010710056A CN 112001889 A CN112001889 A CN 112001889A
- Authority
- CN
- China
- Prior art keywords
- image
- key point
- target key
- target
- dimension
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 19
- 238000000034 method Methods 0.000 title claims description 39
- 230000008569 process Effects 0.000 claims description 16
- 238000012545 processing Methods 0.000 claims description 12
- 238000005520 cutting process Methods 0.000 claims description 9
- 210000000115 thoracic cavity Anatomy 0.000 claims description 8
- 210000000038 chest Anatomy 0.000 claims description 5
- 238000002591 computed tomography Methods 0.000 description 15
- 238000010586 diagram Methods 0.000 description 10
- 238000004590 computer program Methods 0.000 description 7
- 230000011218 segmentation Effects 0.000 description 7
- 230000006870 function Effects 0.000 description 5
- 238000012986 modification Methods 0.000 description 5
- 230000004048 modification Effects 0.000 description 5
- 230000008859 change Effects 0.000 description 3
- 238000013527 convolutional neural network Methods 0.000 description 3
- 238000003860 storage Methods 0.000 description 3
- 238000013473 artificial intelligence Methods 0.000 description 2
- 210000000481 breast Anatomy 0.000 description 2
- 238000003745 diagnosis Methods 0.000 description 2
- 201000010099 disease Diseases 0.000 description 2
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 2
- 241000237970 Conus <genus> Species 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 210000000988 bone and bone Anatomy 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000002059 diagnostic imaging Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000002595 magnetic resonance imaging Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
- G06T7/0014—Biomedical image inspection using an image reference approach
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/97—Determining parameters from multiple pictures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10081—Computed x-ray tomography [CT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10088—Magnetic resonance imaging [MRI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30008—Bone
Abstract
The invention discloses a medical image processing method, which comprises the following steps: acquiring a first image and a second image; acquiring position information of a first target key point and a second target key point in a first image and a second image, wherein the first target key point and the second target key point are bilaterally symmetrical points in a human body; and registering the first image and the second image based on the position information of the first target key point and the second target key point in the first image and the position information of the first target key point and the second target key point in the second image. According to the medical image processing method, the size, the angle and the position of the images at different times can be adjusted quickly and accurately.
Description
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to a medical image processing method and device based on artificial intelligence and a medical image display method.
Background
Today, medical imaging examination is becoming more popular, imaging doctors are often confronted with multiple images of the same patient, and in such a case, the image reading needs to compare the images before and after the examination to obtain a more accurate diagnosis (hereinafter, referred to as contrast image reading). Currently, only some of the image-aided diagnosis tools available from AI corporation are capable of automatically registering multiple images of a subject. However, the registration is only to directly present a plurality of images of the examinee on a multi-view film reading tool without any post-processing of the images.
Taking Computed Tomography (CT) as an example, a subject may need to perform CT examinations at different time periods to confirm a change state of a disease condition, and since an operator of a CT apparatus may be different at each examination and a physical condition of the subject may also change, there may be a difference in selection of a scan interest region and an angle at the time of scanning. Furthermore, when CT images of different time periods are presented in the radiographing tool, the size of the view is not changed, so that the relative position, size and angle of the region of interest in the image examination (for example, the region of interest is the breast when the breast CT is scanned) in the view are different, and if a doctor directly compares radiographing, it is difficult to intuitively determine the change of the disease condition from the comparison of the front and rear CT images. To solve this problem, a doctor usually manually adjusts the size of the region of interest, rotates the image angle, and adjusts the position of the region of interest in the view to make the multiple regions of interest appear approximately consistent in different views, and then performs a contrast reading. But this operation is cumbersome and not precise.
Disclosure of Invention
The invention provides a medical image-based processing method which is used for registering the views of a historical image and a current image based on target key points in the historical image and the current image.
In order to solve the problems in the prior art, the invention provides a medical image processing method, which comprises the following steps:
acquiring a first image and a second image;
acquiring position information of a first target key point and a second target key point in a first image and a second image, wherein the first target key point and the second target key point are bilaterally symmetrical points in a human body;
and registering the first image and the second image based on the position information of the first target key point and the second target key point in the first image and the position information of the first target key point and the second target key point in the second image.
Optionally, the obtaining of the position information of the first target key point and the second target key point in the first image and the second image includes:
inputting the first image and the second image into a first positioning model respectively to obtain the positions of first key points of the first image and the second image respectively;
respectively inputting the image layer of at least one dimension of the first image and the image layer of at least one dimension of the second image into a second positioning model so as to respectively obtain the position of each second key point of the first image on the at least one dimension and the position of each second key point of the second image on the at least one dimension; the image layer with any dimension comprises one or more frames of continuous 2D images obtained by cutting the first image/the second image by using the cutting plane with the dimension, and the cutting planes with different dimensions are not parallel;
taking the same first key point and second key point in each first key point of the first image and each second key point of the first image in the at least one dimension as target key points, and determining the position of the target key point in the first image based on the positions of the same first key point and second key point, wherein the target key point in the first image comprises a first target key point and a second target key point;
and taking the same first key point and second key point in each first key point of the second image and each second key point of the second image in the at least one dimension as target key points, and determining the position of the target key point in the second image based on the positions of the same first key point and second key point, wherein the target key points in the second image comprise the first target key point and the second target key point.
Optionally, the registering the first image and the second image based on the position information of the first target key point and the second target key point in the first image and the position information of the first target key point and the second target key point in the second image includes:
adjusting the display sizes of the first image and the second image according to the distance between the first target key point and the second target key point in the first image and the distance between the first target key point and the second target key point in the second image;
and adjusting the display angles of the first image and the second image through the inclination angle of the connecting line between the first target key point and the second target key point in the first image and the inclination angle of the connecting line between the first target key point and the second target key point in the second image.
Optionally, the medical image processing method further includes:
and adjusting the position of the first image/the second image in the display frame through the first target key point and the second target key point in the first image/the second image.
Optionally, the adjusting the display sizes of the first image and the second image according to the distance between the first target key point and the second target key point in the first image and the distance between the first target key point and the second target key point in the second image includes:
reading a preset distance between the prestored first target key point and the second target key point,
and adjusting the sizes of the first image and the second image based on the preset distance.
Optionally, the medical image processing method further includes:
identifying a patient bed in the first image and the second image,
when registering the first image and the second image, the patient's bed is located below the first target keypoint and the second target keypoint.
Optionally, the medical image processing method further includes:
and adjusting the display sizes of the first image and the second image according to the scale information of the first image and the second image in the Di com protocol.
Optionally, the first image and the second image are chest images, and the first target key point and the second target key point are two farthest end points on the transverse process of the thoracic vertebra respectively.
The present invention also provides a medical image processing apparatus, comprising:
the image acquisition unit is used for acquiring a first image and a second image;
the key point acquisition unit is used for acquiring position information of a first target key point and a second target key point in a first image and a second image, wherein the first target key point and the second target key point are key points which are bilaterally symmetrical in a human body;
and the registration unit is used for registering the first image and the second image based on the position information of the first target key point and the second target key point in the first image and the position information of the first target key point and the second target key point in the second image.
The invention also provides a medical image display method, which comprises the following steps:
responding to the first operation, and displaying a first image and a second image;
responding to a second operation, and displaying the matched first image and second image;
the matching of the first image and the second image refers to the matching of the image display size, the image display angle and the position in the image display frame.
According to the medical image processing method, the target key points in the historical image and the current image are identified, and the images at different times can be adjusted in size, angle and position quickly and accurately through the position information of the target key points, so that the consistency of the display of a plurality of interested parts in different views can be ensured in the display state after the images are registered, more visual contrast information is provided for a doctor, and the efficiency of the doctor in contrasting and reading the film is improved.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
The technical solution of the present invention is further described in detail by the accompanying drawings and embodiments.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention. In the drawings:
FIG. 1 is a flow chart of a medical image processing method according to an embodiment of the invention;
FIG. 2 is a CT image of the chest of an exemplary patient;
FIG. 3 is a flowchart illustrating a method for locating a target keypoint according to an embodiment of the present invention;
FIG. 4 is a schematic structural diagram of a medical image processing apparatus according to an embodiment of the present invention;
fig. 5 is a flowchart illustrating a medical image displaying method according to an embodiment of the invention.
Detailed Description
Various aspects and features of the present application are described herein with reference to the drawings.
It will be understood that various modifications may be made to the embodiments of the present application. Accordingly, the foregoing description should not be construed as limiting, but merely as exemplifications of embodiments. Those skilled in the art will envision other modifications within the scope and spirit of the application.
The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the application and, together with a general description of the application given above and the detailed description of the embodiments given below, serve to explain the principles of the application.
These and other characteristics of the present application will become apparent from the following description of preferred forms of embodiment, given as non-limiting examples, with reference to the attached drawings.
It should also be understood that, although the present application has been described with reference to some specific examples, a person of skill in the art shall certainly be able to achieve many other equivalent forms of application, having the characteristics as set forth in the claims and hence all coming within the field of protection defined thereby.
The above and other aspects, features and advantages of the present application will become more apparent in view of the following detailed description when taken in conjunction with the accompanying drawings.
Specific embodiments of the present application are described hereinafter with reference to the accompanying drawings; however, it is to be understood that the disclosed embodiments are merely exemplary of the application, which can be embodied in various forms. Well-known and/or repeated functions and constructions are not described in detail to avoid obscuring the application of unnecessary or unnecessary detail. Therefore, specific structural and functional details disclosed herein are not to be interpreted as limiting, but merely as a basis for the claims and as a representative basis for teaching one skilled in the art to variously employ the present application in virtually any appropriately detailed structure.
The specification may use the phrases "in one embodiment," "in another embodiment," "in yet another embodiment," or "in other embodiments," which may each refer to one or more of the same or different embodiments in accordance with the application.
Fig. 1 is a flow chart of a medical image processing method according to an embodiment of the invention, as shown in the figure, the method includes:
s11, a first image and a second image are acquired.
And S12, acquiring position information of a first target key point and a second target key point in the first image and the second image, wherein the first target key point and the second target key point are points which are symmetrical left and right in the human body.
S13, registering the first and second images based on the position information of the first and second target keypoints in the first image and the position information of the first and second target keypoints in the second image.
In step S11, a first image and a second image are acquired. The first image and the second image are medical images including the same portion and taken by the same patient at different times, the medical images may be Computed Tomography (CT) images, Magnetic Resonance Imaging (MRI) images and other 3D images, and fig. 2 exemplarily shows a chest CT image of one patient for a clearer description of the medical images. The left side is the current image (first image) of the patient, and the right side is the historical image (second image) of the patient. Here, the interpretation in the landscape mode is exemplified, and if the first image and the second image are arranged in the portrait mode, they are arranged in the portrait mode.
S102, acquiring position information of a first target key point and a second target key point in the first image and the second image. The process of acquiring the position information of the first target key point and the second target key point from the first image is similar to the process of acquiring the position information of the first target key point and the second target key point from the second image, and the following description will be given in detail by taking the example of acquiring the first target key point and the second target key point in the first image with reference to fig. 3.
Fig. 3 is a flowchart illustrating a target keypoint locating method according to an embodiment of the present invention. As shown in fig. 3, the method specifically includes the following steps:
s31, inputting the first image into a first positioning model to obtain the position of each first keypoint of the first image.
S32, inputting the image layer of at least one dimension of the first image into a second positioning model to obtain the position of each second keypoint of the first image in the at least one dimension.
S33, regarding the same first and second key points of the first image in the at least one dimension as target key points, and determining positions of the target key points in the first image based on the positions of the same first and second key points, where the target key points in the first image include the first and second target key points.
The target key point positioning method of this embodiment may position a target key point on a 3D image, taking a Computed Tomography (CT) image as an example, such as a brain CT image, a chest CT image, and the like, and accordingly, the target key point may be a point known by those skilled in the art, or may be a point set by those skilled in the art according to actual needs. In this embodiment, the target key points may be a pair of key points that are symmetric left and right in the human body, and further, the target key points are preferably located at the middle of the human body in the vertical direction of the cross-sectional view, such as the two farthest end points on the transverse process of the thoracic 1 conus. One of the endpoints on the transverse processes of the thoracic 1 vertebra may be a first target keypoint and the other endpoint may be a second target keypoint. The human vertebra, also called as spine bone, has 33 pieces, and can be divided into 7 cervical vertebrae, 12 thoracic vertebrae, 5 lumbar vertebrae, 5 sacral vertebrae and 4 caudal vertebrae according to the distribution position of the vertebra in the human body. The thoracic 1 vertebra is the first vertebra of the 12 thoracic vertebrae, however, in other embodiments, the two farthest end points on the transverse processes of other vertebrae may be used as the first target key point and the second target key point, for example, the two farthest end points of a pair of symmetrical ribs may be used as the first target key point and the second target key point, respectively, which is not limited herein.
Executing S31, in this embodiment, after the first image is obtained, the first image may be preprocessed, for example, the target image may be first divided from the first image, and then the divided target image is pasted to a pure black image that matches the length, width, and height of the target image; the target image may refer to an image where the region of interest is located, such as a pure vertebra image. In this way, by preprocessing the first image, the pixel attribute of the region of the first image irrelevant to the positioning of the target key point can be set as a preset value, and only the pixel attribute of the region relevant to the target key point is reserved, so that the efficiency of subsequent positioning of the target key point can be improved. And inputting the preprocessed first image into the first positioning model, so that the position of each first key point of the first image can be obtained.
And executing S32, inputting the image layer of at least one dimension of the first image into a second positioning model to obtain the position of each second key point of the first image in the at least one dimension.
In specific implementation, after the first image is obtained, the first image may be segmented, before the segmentation, the first image may be converted into an image in a D I COM format, and then a fixed window width and window level may be selected according to D I COM information of the image in the D I COM format to segment the first image. In this way, the first image may be sliced to obtain 2D images of a plurality of frames. In one example, the window width may be chosen to be W-80 and the window level may be chosen to be L-40.
Furthermore, after the first image is segmented in different dimensions to obtain multi-frame 2D images in different dimensions, normalization processing can be performed on the multi-frame 2D images in different dimensions. Specifically, the multi-frame 2D images with different dimensions may be scaled, for example, the multi-frame 2D images with different dimensions may all be scaled to the same size, or the multi-frame 2D images with the same dimension may also be scaled to the same size, and the multi-frame 2D images with different dimensions may be scaled to different sizes, which is not limited specifically. In the embodiment of the invention, the multi-frame 2D images with different dimensions are normalized, so that the multi-frame 2D images with the same dimension or the multi-frame 2D images with different dimensions have consistency, and the efficiency of positioning the target key point in the subsequent images can be improved.
For example, for the first image, a reference coordinate system may be set on the first image in advance, and the reference coordinate system may be composed of an origin o, an x-axis, a y-axis, and a z-axis; further, the first image can be segmented into a multi-frame 2D image of one dimension by taking the xoy plane (i.e., a transverse plane) as a segmentation plane, or the yoz plane (i.e., a coronal plane) as a segmentation plane, or the xoz plane (i.e., a sagittal plane) as a segmentation plane; alternatively, any number (i.e., two or more) of the xoy plane, the yoz plane, and the xoz plane may be taken as the splitting plane, thereby splitting the first image into the multi-frame 2D images of multiple dimensions. If the xoy plane, the yoz plane, and the xoz plane are 3 splitting planes, the xoy plane may be used to split the first image to obtain a multi-frame (e.g., 90 frames) of 2D images with a first dimension, the yoz plane may be used to split the first image to obtain a multi-frame (e.g., 90 frames) of 2D images with a second dimension, and the xoz plane may be used to split the first image to obtain a multi-frame (e.g., 90 frames) of 2D images with a third dimension. The 90 frames of the 2D image with the first dimension may be parallel to the xoy plane, the 90 frames of the 2D image with the second dimension may be parallel to the yoz plane, and the 90 frames of the 2D image with the third dimension may be parallel to the xoz plane.
Further, after the 90 frames of 2D images with the first dimension, the 90 frames of 2D images with the second dimension and the 90 frames of 2D images with the third dimension are obtained through segmentation, the 270 frames of 2D images can be zoomed; in one example, the 270 frames of 2D imagery may each be scaled to a fixed size, such as 512 x 512 pixels. Taking 90 frames of 2D images with the first dimension as an example, in order to ensure the integrity and consistency of the subsequently detected 2D images, before scaling the 90 frames of 2D images with the first dimension, black edges may be added around the 90 frames of 2D images with the first dimension, so that the length-width ratio of the 90 frames of 2D images with the first dimension is adjusted to 1: 1.
Further, after the image layers of the respective dimensions are obtained by segmentation, the image layer of at least one dimension of the first image may be input to the second positioning model, so as to obtain the position of each second key point of the first image in at least one dimension. For example, the image layer of the first dimension may be input to the second positioning model to obtain the position of each second keypoint of the first image in the first dimension; or the image layer of the first dimension and the image layer of the second dimension may also be input to the second positioning model to obtain the positions of the second keypoints of the first image in the first dimension and the positions of the second keypoints of the first image in the second dimension; or the image layer of the first dimension, the image layer of the second dimension, and the image layer of the third dimension may also be input to the second positioning model to obtain the positions of the second keypoints of the first image in the first dimension, the positions of the second keypoints of the first image in the second dimension, and the positions of the second keypoints of the first image in the third dimension, and so on.
The specific implementation process of S32 is described below by taking the determination of each second keypoint of the first image in the first dimension as an example, and the process of determining each second keypoint of the first image in the second dimension and/or the third dimension may be implemented by referring to this method, which is not described herein again.
In a possible implementation manner, in order to reduce the calculation amount of the model, a 2-dimensional convolutional neural network may be used to locate each second keypoint of the first image in the first dimension, where the second location model may be a trained Convolutional Neural Network (CNN) model, or may be another model, which is not limited. In a specific implementation, the second positioning model may include a classifier, a third positioning module, and a fourth positioning module, and the classifier may classify each group of first-dimensional image layers to determine a key frame image from multiple frames of first-dimensional 2D images included in each group of first-dimensional image layers, where the key frame image is a 2D image including a key point; in this way, after the first image is determined to have the keyframe images in the first dimension using the classifier, the keyframe images can be input to the third positioning module to determine the position of the initial second keypoints in each of the keyframe images. Further, each of the roughly divided regions including each of the initial second keypoints may be divided from the keyframe image corresponding to each of the initial second keypoints based on the position of each of the initial second keypoints, and each of the roughly divided regions may be input to the corresponding fourth positioning module, so as to determine the position of each of the second keypoints from each of the roughly divided regions. The fourth positioning module corresponding to the rough-divided region may refer to a depth residual error network in which the type of the training sample is the same as the type of the rough-divided region.
In the embodiment of the invention, the positions of the initial second key points are positioned by adopting the third positioning module, and then the second key points are positioned by adopting the fourth positioning module from the rough segmentation areas containing the initial second key points, so that the positioning range of the second key points can be reduced, and the positioning precision of the key points is improved.
In a possible implementation manner, the fourth positioning module may be determined after a plurality of roughly-divided regions in which key points are labeled in advance are used as training samples to train the depth residual error network, and the number of the fourth positioning modules may be determined according to the number of roughly-divided regions corresponding to each initial second key point.
And executing S33, taking the same first key point and second key point of each first key point of the first image and each second key point of the first image in the at least one dimension as target key points, and determining the position of the target key point in the first image based on the positions of the same first key point and second key point, wherein the target key points in the first image comprise the first target key point and the second target key point.
In the embodiment of the invention, when the first positioning model and the second positioning model are trained, the identification can be manually set on the key points in the training sample, and the identification can be information such as serial numbers, characters and the like; in this way, the keypoints in the first image located by the first location model and the second location model may include not only the positions of the keypoints, but also the identifications of the keypoints. Therefore, after the position and the identification of each first key point of the first image are determined, and the position and the identification of each second key point of the first image in at least one dimension are determined, the first key point and the second key point with the same identification can be determined according to the identification of each first key point and the identification of each second key point in at least one dimension, and the first key point and the second key point with the same identification information are taken as a target key point. Thus, one target keypoint may correspond to a plurality of positions, for example, if the position and the identifier of each first keypoint in the first image are determined, and the position and the identifier of each second keypoint of the first image in the first to third dimensions are determined, one target keypoint may correspond to 4 positions, the coordinate of the first keypoint, the position of the second keypoint in the first dimension, the position of the second keypoint in the second dimension, and the position of the second keypoint in the third dimension.
Further, there may be multiple ways to determine the position of the target key point, for example, an average value of multiple corresponding positions of the target key point may be used as the position of the target key point, or at least two positions that are closer to each other may be selected from the multiple positions of the target key point, and the average value of the at least two positions that are closer to each other may be used as the position of the target key point, which is not limited specifically. In one example, a weight corresponding to the position of the first keypoint and a weight corresponding to the position of each second keypoint in at least one dimension may be preset, and the sum of the weight corresponding to the position of the first keypoint and the weight corresponding to the position of each second keypoint in at least one dimension may be 1; in this way, the weighted average of the position of the first keypoint and the positions of the second keypoints in at least one dimension can be taken as the position of the target keypoint.
It should be noted that, the weight corresponding to the position of the first keypoint and the weight corresponding to each second keypoint in at least one dimension may be set by those skilled in the art according to an actual scene, and are not limited.
In this embodiment, the obtained plurality of target key points include a first target key point and a second target key point, and specific target key points are the first target key point, and specific target key points are the second target key point, which may be determined according to actual requirements, for example, one of two end points (target key points) on the transverse process of the thoracic 1 vertebra is taken as the first target key point, and the other end point is taken as the second target key point.
In the above embodiment of the present invention, the first image is input to the first positioning model to obtain the position of each first keypoint of the first image, and the image layer of at least one dimension of the first image is input to the second positioning model to obtain the position of each second keypoint of the first image in at least one dimension; the image layer of any dimension comprises one or more frames of continuous 2D images obtained by cutting the first image by using the cutting plane of the dimension, and the cutting planes of different dimensions are not parallel; further, the same first key point and second key point in each first key point and each second key point in at least one dimension are used as target key points, the positions of the target key points are determined based on the positions of the same first key point and second key point, and then the positions of the first target key point and the second target key point are obtained. In the embodiment of the invention, the positions of the key points are automatically determined by adopting the first positioning model and the second positioning model, so that the key points can be judged according to the first image without artificial subjectivity, and the efficiency of positioning the key points can be improved; and the positions of the first target key point and the second target key point are obtained through the joint analysis of the position of the first key point positioned by the first positioning model and the position of the second key point positioned by the second positioning model, so that the technical problem of inaccuracy caused by the error of a single positioning model can be avoided, and the positioning precision of the target key point can be improved.
In this embodiment, obtaining the position information of the first target key point and the second target key point in the second image is similar to obtaining the position information of the first target key point and the second target key point in the first image, and the above first image is only replaced by the second image, which is not described herein again.
S13, registering the first and second images based on the position information of the first and second target keypoints in the first image and the position information of the first and second target keypoints in the second image.
Wherein the process of registering comprises:
and adjusting the display sizes of the first image and the second image according to the distance between the first target key point and the second target key point in the first image and the distance between the first target key point and the second target key point in the second image.
And adjusting the display angles of the first image and the second image through the inclination angle of the connecting line between the first target key point and the second target key point in the first image and the inclination angle of the connecting line between the first target key point and the second target key point in the second image.
The process of adjusting the display sizes of the first image and the second image by the distance between the first target keypoint and the second target keypoint in the first image and the distance between the first target keypoint and the second target keypoint in the second image will be described in detail below with reference to fig. 2.
In step S13, the position information of the first target keypoint a1 and the second target keypoint a2 in the first image and the position information of the first target keypoint B1 and the second target keypoint B2 in the second image are obtained, and then the size of the first image and/or the second image is adjusted by the distance between a1 and a2 and the distance between B1 and B2. Specifically, if the distance between a1 and a2 is greater than the distance between B1 and B2, the size of the second image is enlarged such that the distance between B1 and B2 is equal to the distance between a1 and a2, or the size of the first image is reduced such that the distance between a1 and a2 is equal to the distance between B1 and B2.
In other embodiments, the sizes of the first image and the second image may be adjusted by reading a preset distance between the first target key point and the second target key point, and adjusting the sizes of the first image and the second image based on the preset distance. That is, the memory stores reference distances (preset distances) of the first target keypoints and the second target keypoints when displayed in advance, and the first images and the second images are adjusted to be the same as the preset distances in the distance between A1 and A2 and the distance between B1 and B2 respectively.
In this embodiment, the display size of the first image and/or the second image is adjusted according to the distance between the first target keypoint and the second target keypoint in the first image and the distance between the first target keypoint and the second target keypoint in the second image, and the distance between the first target keypoint and the second target keypoint of the subject is usually not changed regardless of the history image or the current image. Therefore, when displaying, the same distance is adjusted between the first target key point and the second target key point in the first image and the second image, and the same distance is also applied to other parts in the first image and the second image in the display window.
In other embodiments, the display sizes of the first image and the second image are adjusted according to the scale information of the first image and the second image in the D i com protocol. The image contains scale information in the D i com protocol, where the scale is the ratio of the actual size of the human body to the size in the displayed image, for example, in the first image of fig. 2, the ratio of the size of the image to the actual size is 5:1, and in the second image, the ratio of the size of the image to the actual size is 7:1, and at this time, the scale information of the two images can be the same by adjusting the scale of the first image to 7:1, or adjusting the scale of the second image to 5:1, or adjusting the scale information of the first image and the second image to a specific value.
For image registration, in addition to adjusting the size of the image, the display angle of the image is also adjusted. In this embodiment, the display angles of the first image and the second image are adjusted by the inclination angle of the connection line between the first target key point and the second target key point in the first image and the second image. Specifically, the display angle of the image is adjusted by adjusting the angle formed by the connecting line between the first target key point and the second target key point and the edge of the display frame. In this embodiment, the display angle of the first image/the second image is adjusted such that the connection line between the first target key point and the second target key point is parallel to the bottom edge of the display frame or perpendicular to the side edge of the display frame. The display angles of the first image and the second image are sequentially adjusted through the method, so that the human body in the first image and the human body in the second image keep the same display angle.
Further, in this embodiment, the registration process of the first image and the second image further includes: and adjusting the position of the image in the display frame through the first target key point and the second target key point. Specifically, a central point on a connecting line between a first target key point and a second target key point is obtained, and the central point is located at the central position of a display frame. Through the steps, the interested part of the doctor in the image can be positioned in the center of the display frame, so that the doctor can read the film conveniently.
Further, in this embodiment, the registration process of the first image and the second image further includes: and identifying the sickbed in the first image and the second image, and when the first image and the second image are registered, the sickbed is positioned below a connecting line between the first target key point and the second target key point. The patient bed identification may be implemented by an AI image identification method, or may be implemented by a conventional image segmentation method, which is not limited herein. The sickbed is identified and is positioned below the first target key point and the second target key point, so that the problem that images are vertically inverted due to the fact that the images are registered only by the first target key point and the second target key point when the image rotation angle is larger than 90 degrees is solved.
The present invention further provides a medical image processing apparatus, fig. 4 is a schematic structural diagram of a medical image processing apparatus according to an embodiment of the present invention, and as shown in fig. 4, the medical image processing apparatus includes:
the image acquisition unit is used for acquiring a first image and a second image.
The key point acquisition unit is used for acquiring the position information of a first target key point and a second target key point in a first image and a second image, wherein the first key point and the second key point are points which are symmetrical left and right in a human body.
And the registration unit is used for registering the first image and the second image based on the position information of the first target key point and the second target key point in the first image and the position information of the first target key point and the second target key point in the second image.
Fig. 5 is a schematic flow chart of a medical image display method according to an embodiment of the present invention, and as shown in fig. 5, the method includes:
s51, displaying the first image and the second image in response to the first operation;
and S52, responding to the second operation, and displaying the matched first image and second image, wherein the matching of the first image and the second image refers to the matching of the image display size, the image display angle and the position in the image display frame.
The medical image display method is applied to a medical image reading system, a doctor has a requirement for comparing and checking a historical image and a current image in the reading process of the medical image, the first operation is that the doctor starts command input for comparing and reading the medical image, the command input can be triggered by a keyboard, a mouse, touch and the like, and when the reading system receives the first operation, the historical image of the patient is retrieved and displayed together with the current image (refer to fig. 2). Further, for the convenience of comparing and reading, the current image and the historical image need to be registered, the registration includes the registration of the image size, the registration of the image display angle and the registration of the image position, similarly, a doctor can trigger a second operation through a keyboard, a mouse, a touch and the like, and when the reading system receives the second operation, the medical image processing method according to the embodiment of the invention registers the current image and the historical image, that is, the first image and the second image are registered to be consistent in the size, the angle and the position of the display window.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.
Claims (10)
1. A method of medical image processing, comprising:
acquiring a first image and a second image;
acquiring position information of a first target key point and a second target key point in a first image and a second image, wherein the first target key point and the second target key point are bilaterally symmetrical points in a human body;
and registering the first image and the second image based on the position information of the first target key point and the second target key point in the first image and the position information of the first target key point and the second target key point in the second image.
2. The method of claim 1, wherein the obtaining the position information of the first and second target keypoints in the first and second images comprises:
inputting the first image and the second image into a first positioning model respectively to obtain the positions of first key points of the first image and the second image respectively;
respectively inputting the image layer of at least one dimension of the first image and the image layer of at least one dimension of the second image into a second positioning model so as to respectively obtain the position of each second key point of the first image on the at least one dimension and the position of each second key point of the second image on the at least one dimension; the image layer with any dimension comprises one or more frames of continuous 2D images obtained by cutting the first image/the second image by using the cutting plane with the dimension, and the cutting planes with different dimensions are not parallel;
taking the same first key point and second key point in each first key point of the first image and each second key point of the first image in the at least one dimension as target key points, and determining the position of the target key point in the first image based on the positions of the same first key point and second key point, wherein the target key point in the first image comprises a first target key point and a second target key point;
and taking the same first key point and second key point in each first key point of the second image and each second key point of the second image in the at least one dimension as target key points, and determining the position of the target key point in the second image based on the positions of the same first key point and second key point, wherein the target key points in the second image comprise the first target key point and the second target key point.
3. The medical image processing method of claim 1, wherein the registering the first and second images based on the position information of the first and second target keypoints in the first image and the position information of the first and second target keypoints in the second image comprises:
adjusting the display sizes of the first image and the second image according to the distance between the first target key point and the second target key point in the first image and the distance between the first target key point and the second target key point in the second image;
and adjusting the display angles of the first image and the second image through the inclination angle of the connecting line between the first target key point and the second target key point in the first image and the inclination angle of the connecting line between the first target key point and the second target key point in the second image.
4. A medical image processing method as claimed in claim 3, further comprising:
and adjusting the position of the first image/the second image in the display frame through the first target key point and the second target key point in the first image/the second image.
5. The method of claim 1, wherein the adjusting the display sizes of the first image and the second image by the distance between the first target keypoint and the second target keypoint in the first image and the distance between the first target keypoint and the second target keypoint in the second image comprises:
reading a preset distance between the prestored first target key point and the second target key point,
and adjusting the sizes of the first image and the second image based on the preset distance.
6. A medical image processing method as claimed in claim 3, further comprising:
identifying a patient bed in the first image and the second image,
when registering the first image and the second image, the patient's bed is located below the first target keypoint and the second target keypoint.
7. The medical image processing method according to claim 1, further comprising:
and adjusting the display sizes of the first image and the second image according to the scale information of the first image and the second image in the Di com protocol.
8. The medical image processing method according to claim 1,
the first image and the second image are chest images, and the first target key point and the second target key point are respectively two farthest end points on the transverse process of the thoracic vertebra.
9. A medical image processing apparatus, comprising:
the image acquisition unit is used for acquiring a first image and a second image;
the key point acquisition unit is used for acquiring position information of a first target key point and a second target key point in a first image and a second image, wherein the first target key point and the second target key point are key points which are bilaterally symmetrical in a human body;
and the registration unit is used for registering the first image and the second image based on the position information of the first target key point and the second target key point in the first image and the position information of the first target key point and the second target key point in the second image.
10. A method for displaying medical images, comprising:
responding to the first operation, and displaying a first image and a second image;
responding to a second operation, and displaying the matched first image and second image;
the matching of the first image and the second image refers to the matching of the image display size, the image display angle and the position in the image display frame.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010710056.1A CN112001889A (en) | 2020-07-22 | 2020-07-22 | Medical image processing method and device and medical image display method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010710056.1A CN112001889A (en) | 2020-07-22 | 2020-07-22 | Medical image processing method and device and medical image display method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112001889A true CN112001889A (en) | 2020-11-27 |
Family
ID=73468120
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010710056.1A Pending CN112001889A (en) | 2020-07-22 | 2020-07-22 | Medical image processing method and device and medical image display method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112001889A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113139954A (en) * | 2021-05-11 | 2021-07-20 | 上海杏脉信息科技有限公司 | Medical image processing device and method |
| WO2022178997A1 (en) * | 2021-02-25 | 2022-09-01 | 平安科技（深圳）有限公司 | Medical image registration method and apparatus, computer device, and storage medium |
| CN115018795A (en) * | 2022-06-09 | 2022-09-06 | 北京医准智能科技有限公司 | Method, device and equipment for matching focus in medical image and storage medium |
| CN115705640A (en) * | 2021-08-13 | 2023-02-17 | 杭州健培科技有限公司 | Automatic registration method, device and application for local rigid part of image |
| CN116740309A (en) * | 2022-03-04 | 2023-09-12 | 武汉迈瑞科技有限公司 | Medical image processing system, medical image processing method and computer equipment |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101739675A (en) * | 2009-12-11 | 2010-06-16 | 重庆邮电大学 | Method and device for registration and synthesis of non-deformed images |
| US20160034781A1 (en) * | 2014-07-31 | 2016-02-04 | International Business Machines Corporation | Method for Accurately Determining the Position and Orientation of Each of a Plurality of Identical Recognition Target Objects in a Search Target Image |
| CN109544623A (en) * | 2018-10-11 | 2019-03-29 | 百度在线网络技术（北京）有限公司 | The measurement method and device in vehicle damage region |
| CN110533639A (en) * | 2019-08-02 | 2019-12-03 | 杭州依图医疗技术有限公司 | A kind of key independent positioning method and device |
| CN111222448A (en) * | 2019-12-31 | 2020-06-02 | 深圳云天励飞技术有限公司 | Image conversion method and related product |
| CN111429354A (en) * | 2020-03-27 | 2020-07-17 | 贝壳技术有限公司 | Image splicing method and device, panorama splicing method and device, storage medium and electronic equipment |
-
2020
- 2020-07-22 CN CN202010710056.1A patent/CN112001889A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101739675A (en) * | 2009-12-11 | 2010-06-16 | 重庆邮电大学 | Method and device for registration and synthesis of non-deformed images |
| US20160034781A1 (en) * | 2014-07-31 | 2016-02-04 | International Business Machines Corporation | Method for Accurately Determining the Position and Orientation of Each of a Plurality of Identical Recognition Target Objects in a Search Target Image |
| CN109544623A (en) * | 2018-10-11 | 2019-03-29 | 百度在线网络技术（北京）有限公司 | The measurement method and device in vehicle damage region |
| CN110533639A (en) * | 2019-08-02 | 2019-12-03 | 杭州依图医疗技术有限公司 | A kind of key independent positioning method and device |
| CN111222448A (en) * | 2019-12-31 | 2020-06-02 | 深圳云天励飞技术有限公司 | Image conversion method and related product |
| CN111429354A (en) * | 2020-03-27 | 2020-07-17 | 贝壳技术有限公司 | Image splicing method and device, panorama splicing method and device, storage medium and electronic equipment |
Non-Patent Citations (1)
| Title |
|---|
| 王毅等: "扩散磁共振成像及其影像处理", 28 February 2017, 西北工业大学出版社, pages: 99 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022178997A1 (en) * | 2021-02-25 | 2022-09-01 | 平安科技（深圳）有限公司 | Medical image registration method and apparatus, computer device, and storage medium |
| CN113139954A (en) * | 2021-05-11 | 2021-07-20 | 上海杏脉信息科技有限公司 | Medical image processing device and method |
| CN113139954B (en) * | 2021-05-11 | 2023-06-20 | 上海杏脉信息科技有限公司 | Medical image processing device and method |
| CN115705640A (en) * | 2021-08-13 | 2023-02-17 | 杭州健培科技有限公司 | Automatic registration method, device and application for local rigid part of image |
| CN116740309A (en) * | 2022-03-04 | 2023-09-12 | 武汉迈瑞科技有限公司 | Medical image processing system, medical image processing method and computer equipment |
| CN115018795A (en) * | 2022-06-09 | 2022-09-06 | 北京医准智能科技有限公司 | Method, device and equipment for matching focus in medical image and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108520519B (en) | Image processing method and device and computer readable storage medium | |
| CN112001889A (en) | Medical image processing method and device and medical image display method | |
| WO2021017297A1 (en) | Artificial intelligence-based spine image processing method and related device | |
| US8150132B2 (en) | Image analysis apparatus, image analysis method, and computer-readable recording medium storing image analysis program | |
| Douglas | Image processing for craniofacial landmark identification and measurement: a review of photogrammetry and cephalometry | |
| US6055326A (en) | Method for orienting electronic medical images | |
| JP5337845B2 (en) | How to perform measurements on digital images | |
| US8423124B2 (en) | Method and system for spine visualization in 3D medical images | |
| US8355553B2 (en) | Systems, apparatus and processes for automated medical image segmentation using a statistical model | |
| US8150120B2 (en) | Method for determining a bounding surface for segmentation of an anatomical object of interest | |
| US20010036302A1 (en) | Method and apparatus for cross modality image registration | |
| US7822254B2 (en) | Automatic positioning of matching multi-planar image reformatting (MPR) views of multiple 3D medical images | |
| US6249590B1 (en) | Method for automatically locating image pattern in digital images | |
| EP2572332A1 (en) | Visualization of medical image data with localized enhancement | |
| US7346199B2 (en) | Anatomic triangulation | |
| JP2017067489A (en) | Diagnosis assistance device, method, and computer program | |
| US10078906B2 (en) | Device and method for image registration, and non-transitory recording medium | |
| WO2013135812A1 (en) | Method, apparatus and system for localizing a spine | |
| JP2015530155A (en) | Analysis Morphomics: High-speed medical image automatic analysis method | |
| EP4156096A1 (en) | Method, device and system for automated processing of medical images to output alerts for detected dissimilarities | |
| EP3047455B1 (en) | Method and system for spine position detection | |
| CN110580948A (en) | Medical image display method and display equipment | |
| CN112349391A (en) | Optimized rib automatic labeling method | |
| US9576353B2 (en) | Method for verifying the relative position of bone structures | |
| CN111445575A (en) | Image reconstruction method and device of Wirisi ring, electronic device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |