CN111242877A - Mammary X-ray image registration method and device - Google Patents
Mammary X-ray image registration method and device Download PDFInfo
- Publication number
- CN111242877A CN111242877A CN201911407248.9A CN201911407248A CN111242877A CN 111242877 A CN111242877 A CN 111242877A CN 201911407248 A CN201911407248 A CN 201911407248A CN 111242877 A CN111242877 A CN 111242877A
- Authority
- CN
- China
- Prior art keywords
- rectangular
- ray image
- breast
- mammary gland
- registration
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 58
- 210000000481 breast Anatomy 0.000 claims abstract description 98
- 210000005075 mammary gland Anatomy 0.000 claims abstract description 74
- 238000006073 displacement reaction Methods 0.000 claims abstract description 26
- 210000002445 nipple Anatomy 0.000 claims abstract description 17
- 238000003062 neural network model Methods 0.000 claims description 16
- 238000013528 artificial neural network Methods 0.000 claims description 15
- 230000008569 process Effects 0.000 claims description 15
- 230000011218 segmentation Effects 0.000 claims description 15
- 238000003860 storage Methods 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 13
- 230000000877 morphologic effect Effects 0.000 claims description 10
- 230000006870 function Effects 0.000 claims description 8
- 238000013519 translation Methods 0.000 claims description 7
- 230000004913 activation Effects 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 claims description 5
- 238000002372 labelling Methods 0.000 claims description 5
- 238000010606 normalization Methods 0.000 claims description 5
- 238000012549 training Methods 0.000 claims description 5
- 238000004891 communication Methods 0.000 claims description 4
- 230000008030 elimination Effects 0.000 claims description 4
- 238000003379 elimination reaction Methods 0.000 claims description 4
- 210000004907 gland Anatomy 0.000 abstract description 4
- 238000010191 image analysis Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 8
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012805 post-processing Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 238000009826 distribution Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 206010070834 Sensitisation Diseases 0.000 description 1
- 238000007792 addition Methods 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000011976 chest X-ray Methods 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 238000002059 diagnostic imaging Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000003703 image analysis method Methods 0.000 description 1
- 238000009776 industrial production Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000009533 lab test Methods 0.000 description 1
- 238000009607 mammography Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000010827 pathological analysis Methods 0.000 description 1
- 230000006798 recombination Effects 0.000 description 1
- 238000005215 recombination Methods 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000008313 sensitization Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration by the use of local operators
- G06T5/30—Erosion or dilatation, e.g. thinning
-
- G06T5/70—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10116—X-ray image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30068—Mammography; Breast
Abstract
The embodiment of the application provides a breast X-ray image registration method and device, and solves the problem that the image alignment of a breast gland cannot be realized in the conventional image analysis mode. The breast X-ray image registration method comprises the following steps: acquiring a rectangular external frame comprising a mammary gland region according to a mammary gland X-ray image; obtaining a scaling ratio according to the height of the rectangular external frame and the height of the rectangular registration window, and scaling the mammary X-ray image according to the scaling ratio; moving the mammographic X-ray image in a vertical direction such that a top end of the rectangular circumscribing frame is aligned with a top end of the rectangular registration window; calculating a horizontal displacement amount required when a vertical border of the rectangular circumscribing frame far away from the nipple is moved to a horizontal direction edge of the rectangular registration window based on the scaling; and moving the mammary gland X-ray image based on the horizontal displacement amount.
Description
Technical Field
The application relates to the technical field of image analysis, in particular to a breast X-ray image registration method, a breast X-ray image registration device, electronic equipment and a computer-readable storage medium.
Background
The mammary gland X-ray image is based on the medical diagnosis of X-ray, because the X-ray passes through the human body, the absorbed degree is different, the X-ray quantity after passing through the human body is different, thus the formed image carries the information of the density distribution of each part of the human body, the intensity of the fluorescence action or the sensitization action caused on the fluorescent screen or the photographic film has larger difference, and the shadow with different density is displayed on the fluorescent screen or the photographic film (after development and fixation). According to the contrast of shade, combine clinical manifestation, laboratory test result and pathological diagnosis, can judge whether a certain part of the human body is normal.
The mammary gland X-ray image is different from the chest X-ray and hand X-ray of routine physical examination, outpatient service, etc. In order to ensure that the double breasts can successfully complete the screening and comparison work in the mammary gland X, the shot image is generally divided into 4 projection positions, including the axial position (LCC) of the left mammary gland, the axial position (RCC) of the right mammary gland, the oblique position (LMLO) of the left mammary gland and the oblique position (RMLO) of the right mammary gland.
In a general medical image interpretation tool, four breast images cannot be displayed in such a manner that the glands are aligned. In the clinical level of medical imaging, the position, distribution, symmetry and the like of a focus are judged by a mammary gland X-ray image, and a plurality of images are required to be compared and read simultaneously. Because the traditional image analysis software cannot align the image of the breast gland and cannot effectively support clinical work, manual alignment is required to be manually performed in a zooming and moving mode, and partial workload of doctors is increased.
Disclosure of Invention
In view of this, the embodiment of the present application provides a breast X-ray image registration method and apparatus, which solve the problem that the image alignment of a breast gland cannot be realized in the existing image analysis method.
According to an aspect of the present application, an embodiment of the present application provides a breast X-ray image registration method, including: acquiring a rectangular external frame comprising a mammary gland region according to a mammary gland X-ray image; obtaining a scaling ratio according to the height of the rectangular external frame and the height of the rectangular registration window, and scaling the mammary X-ray image according to the scaling ratio; moving the mammographic X-ray image in a vertical direction such that a top end of the rectangular circumscribing frame is aligned with a top end of the rectangular registration window; calculating a horizontal displacement amount required when a vertical border of the rectangular circumscribing frame far away from the nipple is moved to a horizontal direction edge of the rectangular registration window based on the scaling; and moving the mammary gland X-ray image based on the horizontal displacement amount.
In an embodiment of the present application, the method further comprises: acquiring coordinates of a first vertex and coordinates of a second vertex at two ends of a diagonal of the rectangular external frame, wherein the coordinates of the first vertex are closer to the nipple than the coordinates of the second vertex in the horizontal direction; wherein, before the obtaining a scaling ratio according to the height of the rectangular bounding box and the height of the rectangular registration window and scaling the mammary X-ray image according to the scaling ratio, the method further comprises: and obtaining the height of the rectangular external frame according to the coordinates of the first vertex and the coordinates of the second vertex.
In an embodiment of the present application, the calculating the required horizontal displacement amount when moving the vertical border of the rectangular circumscribing frame away from the nipple to the horizontal edge of the rectangular registration window includes: based on the scaling, calculating a corresponding first registration position of the first vertex in the rectangular registration window; and obtaining the amount of horizontal displacement from a horizontal distance between the first vertex and the first registration location.
In an embodiment of the present application, the acquiring a rectangular bounding box including a breast region according to a breast X-ray image includes: inputting the mammary gland X-ray image into a mammary gland region segmentation model to obtain a mammary gland region output by the mammary gland region segmentation model, wherein the mammary gland region segmentation model is a pre-established deep neural network model; post-treating the breast area, wherein the post-treatment process comprises one or more of the following operations in combination: morphological open operation, morphological close operation and noise elimination; and selecting the minimum external rectangular frame of the maximum connected region as the rectangular external frame of the mammary gland region.
In an embodiment of the present application, the deep neural network model includes a plurality of convolutional layers, the plurality of convolutional layers including: a plurality of downsampled convolutional layers, a plurality of upsampled convolutional layers, and a plurality of normal convolutional layers; wherein each convolutional layer of the plurality of convolutional layers is connected with a corresponding normalization layer and activation function layer.
In an embodiment of the present application, a striding connection is used between different downsampled convolutional layers, and/or a striding connection is used between different upsampled convolutional layers.
In an embodiment of the present application, the deep neural network model is pre-established by the following process: inputting a mammary gland X-ray image sample including a doctor labeling mammary gland area into an initial deep neural network for training; comparing the predicted breast area output by the initial deep neural network with the doctor-labeled breast area to calculate a loss value; and updating the parameters of the initial deep neural network in a gradient feedback mode based on the loss value.
According to an aspect of the present application, an embodiment of the present application provides a breast X-ray image registration apparatus including: a first acquisition module configured to acquire a rectangular circumscribing frame including a breast region from a breast X-ray image; the second acquisition module is configured to acquire a scaling ratio according to the height of the rectangular external frame and the height of the rectangular registration window, and scale the mammary X-ray image according to the scaling ratio; a vertical translation module configured to move the mammographic X-ray image in a vertical direction such that a top end of the rectangular circumscribing frame is aligned with a top end of the rectangular registration window;
a first calculation module configured to calculate, based on the scaling, an amount of horizontal displacement required to move a vertical bounding box of the rectangular bounding box away from a nipple to a horizontal edge of the rectangular registration window; and a horizontal translation module configured to move the mammary X-ray image based on the horizontal displacement amount.
In an embodiment of the present application, the apparatus further comprises: a third obtaining module configured to obtain coordinates of a first vertex and coordinates of a second vertex at both ends of a diagonal of the rectangular circumscribing frame, wherein the coordinates of the first vertex are closer to the nipple than the coordinates of the second vertex in the horizontal direction; wherein the apparatus further comprises: and the second calculation module is configured to obtain a scaling ratio according to the height of the rectangular outer frame and the height of the rectangular registration window, and obtain the height of the rectangular outer frame according to the coordinates of the first vertex and the coordinates of the second vertex before scaling the mammary X-ray image according to the scaling ratio.
In an embodiment of the application, the first computing module is further configured to: based on the scaling, calculating a corresponding first registration position of the first vertex in the rectangular registration window; and obtaining the amount of horizontal displacement from a horizontal distance between the first vertex and the first registration location.
In an embodiment of the present application, the first obtaining module includes: the input unit is configured to input the mammary gland X-ray image into a mammary gland region segmentation model so as to obtain a mammary gland region output by the mammary gland region segmentation model, wherein the mammary gland region segmentation model is a pre-established deep neural network model; a post-processing unit configured to post-process the breast region, wherein the post-processing procedure comprises one or more of the following operations in combination: morphological open operation, morphological close operation and noise elimination; and the selecting unit is configured to select the minimum external rectangular frame of the maximum communication area as the rectangular external frame of the mammary gland area.
In an embodiment of the present application, the deep neural network model includes a plurality of convolutional layers, the plurality of convolutional layers including: a plurality of downsampled convolutional layers, a plurality of upsampled convolutional layers, and a plurality of normal convolutional layers; wherein each convolutional layer of the plurality of convolutional layers is connected with a corresponding normalization layer and activation function layer.
In an embodiment of the present application, a striding connection is used between different downsampled convolutional layers, and/or a striding connection is used between different upsampled convolutional layers.
In an embodiment of the present application, the deep neural network model is pre-established by the following process: inputting a mammary gland X-ray image sample including a doctor labeling mammary gland area into an initial deep neural network for training; comparing the predicted breast area output by the initial deep neural network with the doctor-labeled breast area to calculate a loss value; and updating the parameters of the initial deep neural network in a gradient feedback mode based on the loss value.
According to another aspect of the present application, an embodiment of the present application provides an electronic device, including: a processor; a memory; and computer program instructions stored in the memory, which when executed by the processor, cause the processor to perform the method of breast X-ray image registration according to any one of the preceding claims.
According to another aspect of the present application, an embodiment of the present application provides a computer-readable storage medium having stored thereon computer program instructions, which, when executed by a processor, cause the processor to execute the breast X-ray image registration method according to any one of the preceding claims.
According to another aspect of the present application, an embodiment of the present application provides a computer program product, which includes computer program instructions, when executed by a processor, cause the processor to execute the breast X-ray image registration method as described in any one of the above.
According to the breast X-ray image registration method, the breast X-ray image registration device, the electronic equipment and the computer readable storage medium, the rectangular outer frame comprising the breast area and the scaling ratio between the breast X-ray image and the rectangular registration window for displaying the breast image are obtained, so that the top end alignment and the edge alignment in the rectangular registration window can be realized by using the rectangular outer frame, the workload before the doctor reads the film is greatly reduced, and the working efficiency of the doctor for reading the film can be obviously improved.
Drawings
Fig. 1 is a schematic flowchart illustrating a breast X-ray image registration method according to an embodiment of the present disclosure.
Fig. 2 is a schematic view of a breast X-ray image according to an embodiment of the present application.
Fig. 3 is a schematic flow chart illustrating a process of calculating a horizontal displacement in a breast X-ray image registration method according to an embodiment of the present application.
Fig. 4 is a schematic combination diagram of a registered breast X-ray image according to an embodiment of the present application.
Fig. 5 is a schematic specific flowchart illustrating a method for acquiring a rectangular bounding box including a breast region according to a breast X-ray image in a breast X-ray image registration method according to an embodiment of the present application.
Fig. 6 is a schematic structural diagram of a breast X-ray image registration apparatus according to an embodiment of the present application.
Fig. 7 is a schematic structural diagram of a breast X-ray image registration apparatus according to another embodiment of the present application.
Fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Fig. 1 is a schematic flowchart illustrating a breast X-ray image registration method according to an embodiment of the present disclosure. As shown in fig. 1, the breast X-ray image registration method includes:
step 101: a rectangular circumscribing frame including a breast region is acquired from the breast X-ray image.
The breast area is an area corresponding to the breast image in the breast X-ray image, and the rectangular bounding box is a rectangular area including the following area, such as the rectangular area shown in fig. 2. The position of the breast image can be represented by acquiring the rectangular external frame comprising the breast area, so that the position registration of the breast image can be realized in the subsequent process of moving the breast X-ray image. In addition, as the rectangular registration window is also adopted as the registration window for displaying the breast image, the operability of realizing the registration of the breast image in the rectangular registration window by utilizing the rectangular external frame is higher and more accurate.
Step 102: and obtaining a scaling ratio according to the height of the rectangular external frame and the height of the rectangular registration window, and scaling the mammary X-ray image according to the scaling ratio.
When the mammary gland X-ray image is placed in the rectangular registration window, in order to enable the rectangular external frame to be capable of filling the rectangular registration window in the vertical direction, scaling of the mammary gland X-ray is required to a certain degree, and at the moment, the scaling can be obtained according to the height of the rectangular external frame and the height of the rectangular registration window. And then, scaling the mammary gland X-ray image according to the scaling, so that the rectangular outer frame can be ensured to just fill the rectangular registration window in the vertical direction in the subsequent vertical direction translation process.
In an embodiment of the present application, as shown in fig. 2, coordinates [ x1, y1] of a first vertex and coordinates [ x2, y2] of a second vertex at both ends of a diagonal of the rectangular bounding box may be further obtained, wherein the coordinates of the first vertex are closer to the nipple than the coordinates of the second vertex in the horizontal direction. Thus, before the scaling is obtained according to the height of the rectangular outer frame and the height of the rectangular registration window and the mammary X-ray image is scaled according to the scaling, the height of the rectangular outer frame can be obtained according to the coordinates of the first vertex and the coordinates of the second vertex. Specifically, the absolute value of the difference between y1 and y2 becomes the height of the rectangular bounding box.
Step 103: the mammography X-ray image is moved in the vertical direction so that the top of the rectangular circumscribing frame is aligned with the top of the rectangular registration window.
Since the initial position of the mammary gland X-ray image may be shifted when the mammary gland X-ray image is placed in the rectangular registration window, but since the scaling is already calculated and scaled through the foregoing steps, the mammary gland X-ray image is directly moved in the vertical direction so that the top end of the rectangular outer frame is aligned with the top end of the rectangular registration window.
Step 104: based on the scaling, the amount of horizontal displacement required to move the vertical bounding box of the rectangular bounding box away from the nipple to the horizontal edge of the rectangular registration window is calculated.
Although the rectangular bounding box has been vertically aligned at the top of the rectangular registration window, the rectangular bounding box is not horizontally aligned with the edges of the rectangular registration window. Since the mammary gland X-ray image is zoomed at this time, it is necessary to calculate the horizontal displacement amount required to move the vertical border of the rectangular outer frame far from the nipple to the horizontal edge of the rectangular registration window based on the zoom ratio, so as to know how much distance the zoomed mammary gland X-ray image needs to move in the horizontal direction to align the vertical border of the rectangular outer frame far from the nipple with the horizontal edge of the rectangular registration window.
In an embodiment of the present application, as shown in fig. 3, when the coordinates [ x1, y1] of the first vertex and the coordinates [ x2, y2] of the second vertex at both ends of the diagonal of the rectangular circumscribing frame are acquired, calculating the horizontal displacement amount required to move the vertical border of the rectangular circumscribing frame away from the nipple to the horizontal direction edge of the rectangular registration window may specifically include:
step 301: based on the scaling, a corresponding first registration position of the first vertex in the rectangular registration window is calculated.
Step 302: the horizontal displacement amount is obtained from the horizontal distance between the first vertex and the first registration position.
Therefore, the position of the rectangular circumscribing frame is represented by the first vertex, and the edge alignment of the rectangular circumscribing frame in the rectangular registration window can be realized by calculating the first registration position corresponding to the first vertex and referring to the moving distance of the first vertex. Specifically, based on the scaling, the distance from the horizontal edge of the rectangular registration window to the first vertex can be calculated, and the first configuration position can be obtained, where the horizontal distance between the first vertex and the first registration position becomes the displacement of the breast X-ray image that should be horizontally translated.
Step 105: the mammary gland X-ray image is moved based on the amount of horizontal displacement.
And moving the mammary X-ray image based on the calculated horizontal displacement to realize the horizontal edge alignment of the rectangular external frame and the rectangular registration window.
Therefore, according to the mammary gland X-ray image registration method provided by the embodiment of the application, the rectangular external frame comprising the mammary gland area and the scaling ratio between the mammary gland X-ray image and the rectangular registration window for displaying the mammary gland image are obtained, so that the top end alignment and the edge alignment in the rectangular registration window can be realized by using the rectangular external frame, the workload before the doctor reads the film is greatly reduced, and the work efficiency of the doctor to read the film can be obviously improved.
In an embodiment of the present application, as described above, since the X-ray image of the breast is generally divided into 4 projection positions, including the axial position (LCC) of the left breast, the axial position (RCC) of the right breast, the oblique position (LMLO) of the left breast, and the oblique position (RMLO) of the right breast, in order to enable the images of the projection positions to be displayed and compared more intuitively, the X-ray images of the breast at the four projection positions may be arranged in an array, as shown in fig. 4, the image of the axial position (LCC) of the left breast is located at the upper left after the registration process, the image of the axial position (RCC) of the right breast is located at the upper right after the registration process, the image of the oblique position (LMLO) of the left breast is located at the lower left after the registration process, and the image of the oblique position (RMLO) of the right breast is located at the lower left after the registration process.
Fig. 5 is a schematic specific flowchart illustrating a method for acquiring a rectangular bounding box including a breast region according to a breast X-ray image in a breast X-ray image registration method according to an embodiment of the present application. As shown in fig. 5, the rectangular circumscribing frame can be obtained by the following steps:
step 501: inputting the mammary gland X-ray image into a mammary gland region segmentation model to obtain a mammary gland region output by the mammary gland region segmentation model, wherein the mammary gland region segmentation model is a pre-established deep neural network model.
In an embodiment of the present application, the deep neural network model includes a plurality of convolutional layers, the plurality of convolutional layers including: a plurality of downsampled convolutional layers, a plurality of upsampled convolutional layers, and a plurality of normal convolutional layers; wherein each convolutional layer of the plurality of convolutional layers is connected with the corresponding normalization layer and activation function layer.
In a further embodiment of the present application, a cross-over connection is used between different downsampled convolutional layers, and/or a cross-over connection is used between different upsampled convolutional layers. Therefore, the deep neural network model can learn information with different resolutions.
In an embodiment of the present application, the deep neural network model may be pre-established by the following processes: inputting a mammary gland X-ray image sample including a doctor labeling mammary gland area into an initial deep neural network for training; comparing the predicted mammary gland region output by the initial deep neural network with the doctor-labeled mammary gland region to calculate a loss value; and updating the parameters of the initial deep neural network in a gradient return mode based on the loss value.
Step 502: subjecting the mammary gland region to post-treatment, wherein the post-treatment process comprises one or more of the following operations in combination: morphological on operation, morphological off operation, and noise cancellation. However, it should be understood that the specific content of the post-processing procedure performed on the mammary gland region can be adjusted according to the requirements of the actual application scenario, and the present application is not limited thereto.
Step 503: and selecting the minimum external rectangular frame of the maximum connected region as the rectangular external frame of the mammary gland region. After the breast area is segmented, there will be some small noise, and the most connected area is generally the breast area.
Fig. 6 is a schematic structural diagram of a breast X-ray image registration apparatus according to an embodiment of the present application. As shown in fig. 6, the breast X-ray image registration apparatus 60 includes:
a first acquisition module 601 configured to acquire a rectangular circumscribing frame including a breast area from a breast X-ray image;
a second obtaining module 602, configured to obtain a scaling ratio according to the height of the rectangular outer frame and the height of the rectangular registration window, and scale the mammary X-ray image according to the scaling ratio;
a vertical translation module 603 configured to move the mammographic X-ray image in a vertical direction such that a top end of the rectangular circumscribing frame is aligned with a top end of the rectangular registration window;
a first calculating module 604 configured to calculate, based on the scaling, an amount of horizontal displacement required to move a vertical border of the rectangular circumscribing frame away from the nipple to a horizontal edge of the rectangular registration window; and
a horizontal translation module 605 configured to move the mammographic X-ray image based on the amount of horizontal displacement.
According to the mammary gland X-ray image registration device provided by the embodiment of the application, the rectangular external frame comprising the mammary gland area and the scaling ratio between the mammary gland X-ray image and the rectangular registration window for displaying the mammary gland image are obtained, so that the top alignment and the edge alignment in the rectangular registration window can be realized by using the rectangular external frame, the workload before the doctor reads the film is greatly reduced, and the work efficiency of the doctor in reading the film can be obviously improved.
In an embodiment of the present application, as shown in fig. 7, the breast X-ray image registration apparatus 60 further includes:
a third obtaining module 606 configured to obtain coordinates of a first vertex and coordinates of a second vertex at both ends of a diagonal of the rectangular circumscribing frame, wherein the coordinates of the first vertex are closer to the nipple than the coordinates of the second vertex in the horizontal direction;
wherein, the breast X-ray image registration device 60 further comprises:
a second calculation module 607 configured to obtain the height of the rectangular bounding box according to the coordinates of the first vertex and the coordinates of the second vertex before obtaining the scaling according to the height of the rectangular bounding box and the height of the rectangular registration window and scaling the mammographic image according to the scaling.
In an embodiment of the present application, the first calculation module 604 is further configured to: calculating a corresponding first registration position of the first vertex in the rectangular registration window based on the scaling; and acquiring the horizontal displacement amount according to the horizontal distance between the first vertex and the first registration position.
In an embodiment of the present application, as shown in fig. 7, the first obtaining module 601 includes:
an input unit 6011 configured to input the X-ray breast image into the breast region segmentation model to obtain a breast region output by the breast region segmentation model, wherein the breast region segmentation model is a pre-established deep neural network model;
a post-treatment unit 6012 configured to post-treat the breast area, wherein the post-treatment process comprises one or more of the following operations in combination: morphological open operation, morphological close operation and noise elimination; and
the selecting unit 6013 is configured to select a minimum bounding rectangle of the maximum connected region as a rectangle bounding box of the breast region.
In an embodiment of the present application, the deep neural network model includes a plurality of convolutional layers, the plurality of convolutional layers including: a plurality of downsampled convolutional layers, a plurality of upsampled convolutional layers, and a plurality of normal convolutional layers; wherein each convolutional layer of the plurality of convolutional layers is connected with the corresponding normalization layer and activation function layer.
In an embodiment of the present application, a striding connection is used between different downsampled convolutional layers, and/or a striding connection is used between different upsampled convolutional layers.
In an embodiment of the present application, the deep neural network model is pre-established by the following process: inputting a mammary gland X-ray image sample including a doctor labeling mammary gland area into an initial deep neural network for training; comparing the predicted mammary gland region output by the initial deep neural network with the doctor-labeled mammary gland region to calculate a loss value; and updating the parameters of the initial deep neural network in a gradient return mode based on the loss value.
The detailed functions and operations of the respective modules in the mammographic image registration apparatus 60 are described in detail in the mammographic image registration method described above with reference to fig. 1 to 4. Therefore, a repetitive description thereof will be omitted herein.
It should be noted that the breast X-ray image registration apparatus 60 according to the embodiment of the present application may be integrated into the electronic device 80 as a software module and/or a hardware module, in other words, the electronic device 80 may include the breast X-ray image registration apparatus 60. For example, the breast X-ray image registration device 60 may be a software module in the operating system of the electronic device 80, or may be an application developed for it; of course, the breast X-ray image registration device 60 can also be one of the hardware modules of the electronic apparatus 80.
In another embodiment of the present application, the mammographic image registration apparatus 60 and the electronic device 80 can be separate devices (e.g., a server), and the mammographic image registration apparatus 60 can be connected to the electronic device 80 through a wired and/or wireless network and transmit interactive information according to an agreed data format.
Fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present application. As shown in fig. 8, the electronic apparatus 80 includes: one or more processors 801 and memory 802; and computer program instructions stored in the memory 802 which, when executed by the processor 801, cause the processor 801 to perform the breast X-ray image registration method according to any of the embodiments described above.
The processor 801 may be a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.
Memory 802 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. Volatile memory can include, for example, Random Access Memory (RAM), cache memory (or the like). The non-volatile memory may include, for example, Read Only Memory (ROM), a hard disk, flash memory, and the like. One or more computer program instructions may be stored on a computer readable storage medium and executed by the processor 801 to implement the steps of the breast X-ray image registration method of the various embodiments of the present application above and/or other desired functions. Information such as light intensity, compensation light intensity, position of the filter, etc. may also be stored in the computer readable storage medium.
In one example, the electronic device 80 may further include: an input device 803 and an output device 804, which are interconnected by a bus system and/or other form of connection mechanism (not shown in fig. 8).
For example, when the electronic device is a robot in an industrial production line, the input device 803 may be a camera for capturing the position of the part to be processed. When the electronic device is a stand-alone device, the input means 803 may be a communication network connector for receiving the collected input signal from an external removable device. The input device 803 may also include, for example, a keyboard, a mouse, a microphone, and the like.
The output device 804 may output various information to the outside, and may include, for example, a display, speakers, a printer, and a communication network and its connected remote output devices, among others.
Of course, for simplicity, only some of the components of the electronic device 80 relevant to the present application are shown in fig. 8, and components such as buses, input devices/output interfaces, and the like are omitted. In addition, the electronic device 80 may include any other suitable components depending on the particular application.
In addition to the above-described methods and apparatuses, embodiments of the present application may also be a computer program product comprising computer program instructions which, when executed by a processor, cause the processor to perform the steps of the breast X-ray image registration method according to any of the above-described embodiments.
The computer program product may include program code for carrying out operations for embodiments of the present application in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server.
Furthermore, embodiments of the present application may also be a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, cause the processor to perform the steps in the breast X-ray image registration method according to various embodiments of the present application described in the above section "exemplary breast X-ray image registration method" of the present specification.
A computer-readable storage medium may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory ((RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The foregoing describes the general principles of the present application in conjunction with specific embodiments, however, it is noted that the advantages, effects, etc. mentioned in the present application are merely examples and are not limiting, and they should not be considered essential to the various embodiments of the present application. Furthermore, the foregoing disclosure of specific details is for the purpose of illustration and description and is not intended to be limiting, since the foregoing disclosure is not intended to be exhaustive or to limit the disclosure to the precise details disclosed.
The block diagrams of devices, apparatuses, systems referred to in this application are only given as illustrative examples and are not intended to require or imply that the connections, arrangements, configurations, etc. must be made in the manner shown in the block diagrams. These devices, apparatuses, devices, systems may be connected, arranged, configured in any manner, as will be appreciated by those skilled in the art. Words such as "including," "comprising," "having," and the like are open-ended words that mean "including, but not limited to," and are used interchangeably therewith. The words "or" and "as used herein mean, and are used interchangeably with, the word" and/or, "unless the context clearly dictates otherwise. The word "such as" is used herein to mean, and is used interchangeably with, the phrase "such as but not limited to".
It should also be noted that in the devices, apparatuses, and methods of the present application, the components or steps may be decomposed and/or recombined. These decompositions and/or recombinations are to be considered as equivalents of the present application.
The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the application. Thus, the present application is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
The foregoing description has been presented for purposes of illustration and description. Furthermore, the description is not intended to limit embodiments of the application to the form disclosed herein. While a number of example aspects and embodiments have been discussed above, those of skill in the art will recognize certain variations, modifications, alterations, additions and sub-combinations thereof.
The above description is only exemplary of the present application and should not be taken as limiting the present application, as any modifications, equivalents and the like that are within the spirit and principle of the present application should be included in the scope of the present application.
Claims (10)
1. A breast X-ray image registration method, comprising:
acquiring a rectangular external frame comprising a mammary gland region according to a mammary gland X-ray image;
obtaining a scaling ratio according to the height of the rectangular external frame and the height of the rectangular registration window, and scaling the mammary X-ray image according to the scaling ratio;
moving the mammographic X-ray image in a vertical direction such that a top end of the rectangular circumscribing frame is aligned with a top end of the rectangular registration window;
calculating a horizontal displacement amount required when a vertical border of the rectangular circumscribing frame far away from the nipple is moved to a horizontal direction edge of the rectangular registration window based on the scaling; and
and moving the mammary gland X-ray image based on the horizontal displacement.
2. The method of claim 1, further comprising:
acquiring coordinates of a first vertex and coordinates of a second vertex at two ends of a diagonal of the rectangular external frame, wherein the coordinates of the first vertex are closer to the nipple than the coordinates of the second vertex in the horizontal direction;
wherein, before the obtaining a scaling ratio according to the height of the rectangular bounding box and the height of the rectangular registration window and scaling the mammary X-ray image according to the scaling ratio, the method further comprises:
and obtaining the height of the rectangular external frame according to the coordinates of the first vertex and the coordinates of the second vertex.
3. The method of claim 2, wherein the calculating the amount of horizontal displacement required to move a vertical bounding box of the rectangular bounding box away from a nipple to a horizontally oriented edge of the rectangular registration window comprises:
based on the scaling, calculating a corresponding first registration position of the first vertex in the rectangular registration window; and
obtaining the amount of horizontal displacement from a horizontal distance between the first vertex and the first registration location.
4. The method of any one of claims 1 to 3, wherein said obtaining a rectangular bounding box including a breast region from the mammographic image comprises:
inputting the mammary gland X-ray image into a mammary gland region segmentation model to obtain a mammary gland region output by the mammary gland region segmentation model, wherein the mammary gland region segmentation model is a pre-established deep neural network model;
post-treating the breast area, wherein the post-treatment process comprises one or more of the following operations in combination: morphological open operation, morphological close operation and noise elimination; and
and selecting the minimum external rectangular frame of the maximum communication area as the rectangular external frame of the mammary gland area.
5. The method of claim 4, wherein the deep neural network model comprises a plurality of convolutional layers, the plurality of convolutional layers comprising: a plurality of downsampled convolutional layers, a plurality of upsampled convolutional layers, and a plurality of normal convolutional layers;
wherein each convolutional layer of the plurality of convolutional layers is connected with a corresponding normalization layer and activation function layer.
6. The method of claim 4, wherein a cross-over connection is used between different downsampled convolutional layers and/or between different upsampled convolutional layers.
7. The method of claim 4, wherein the deep neural network model is pre-established by:
inputting a mammary gland X-ray image sample including a doctor labeling mammary gland area into an initial deep neural network for training;
comparing the predicted breast area output by the initial deep neural network with the doctor-labeled breast area to calculate a loss value;
and updating the parameters of the initial deep neural network in a gradient feedback mode based on the loss value.
8. A breast X-ray image registration apparatus, comprising:
a first acquisition module configured to acquire a rectangular circumscribing frame including a breast region from a breast X-ray image;
the second acquisition module is configured to acquire a scaling ratio according to the height of the rectangular external frame and the height of the rectangular registration window, and scale the mammary X-ray image according to the scaling ratio;
a vertical translation module configured to move the mammographic X-ray image in a vertical direction such that a top end of the rectangular circumscribing frame is aligned with a top end of the rectangular registration window;
a first calculation module configured to calculate, based on the scaling, an amount of horizontal displacement required to move a vertical bounding box of the rectangular bounding box away from a nipple to a horizontal edge of the rectangular registration window; and
a horizontal translation module configured to move the mammographic X-ray image based on the horizontal displacement amount.
9. An electronic device, comprising:
a processor; and
memory having stored therein computer program instructions which, when executed by the processor, cause the processor to perform the method of any of claims 1 to 7.
10. A computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, cause the processor to perform the method of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911407248.9A CN111242877A (en) | 2019-12-31 | 2019-12-31 | Mammary X-ray image registration method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911407248.9A CN111242877A (en) | 2019-12-31 | 2019-12-31 | Mammary X-ray image registration method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111242877A true CN111242877A (en) | 2020-06-05 |
Family
ID=70874127
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911407248.9A Pending CN111242877A (en) | 2019-12-31 | 2019-12-31 | Mammary X-ray image registration method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111242877A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112465802A (en) * | 2020-12-10 | 2021-03-09 | 上海联影医疗科技股份有限公司 | Method, system, terminal and computer readable storage medium for processing tomographic image |
Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150154756A1 (en) * | 2013-12-03 | 2015-06-04 | Viewray Incorporated | Single- And Multi-Modality Alignment Of Medical Images In The Presence Of Non-Rigid Deformations Using Phase Correlation |
| CN105761217A (en) * | 2016-01-29 | 2016-07-13 | 上海联影医疗科技有限公司 | Image reconstruction method and device |
| CN107049345A (en) * | 2017-05-23 | 2017-08-18 | 深圳艾砾拓科技有限公司 | Exposure index and bias exponent computational methods in digital X-ray mammography system |
| CN107240131A (en) * | 2017-06-09 | 2017-10-10 | 大连理工大学 | A kind of breast image method for registering based on iteration texture Deformation Field |
| US20170337672A1 (en) * | 2016-05-20 | 2017-11-23 | Shenyang Neusoft Medical Systems Co., Ltd. | Image splicing |
| CN107424162A (en) * | 2017-05-05 | 2017-12-01 | 上海联影医疗科技有限公司 | A kind of image partition method and system |
| US20180182103A1 (en) * | 2016-12-23 | 2018-06-28 | International Business Machines Corporation | 3d segmentation reconstruction from 2d slices |
| CN108717707A (en) * | 2018-04-10 | 2018-10-30 | 杭州依图医疗技术有限公司 | A kind of tubercle matching process and device |
| WO2018222755A1 (en) * | 2017-05-30 | 2018-12-06 | Arterys Inc. | Automated lesion detection, segmentation, and longitudinal identification |
| US20180374193A1 (en) * | 2017-06-21 | 2018-12-27 | International Business Machines Corporation | Automatic Image Registration Quality Assessment |
| CN109512464A (en) * | 2018-11-23 | 2019-03-26 | 清华大学深圳研究生院 | A kind of disorder in screening and diagnostic system |
| CN109635835A (en) * | 2018-11-08 | 2019-04-16 | 深圳蓝韵医学影像有限公司 | A kind of breast lesion method for detecting area based on deep learning and transfer learning |
| CN109754387A (en) * | 2018-11-23 | 2019-05-14 | 北京永新医疗设备有限公司 | Medical image lesion detects localization method, device, electronic equipment and storage medium |
| CN109872332A (en) * | 2019-01-31 | 2019-06-11 | 广州瑞多思医疗科技有限公司 | A kind of 3 d medical images method for registering based on U-NET neural network |
| CN110473235A (en) * | 2019-09-12 | 2019-11-19 | 杭州依图医疗技术有限公司 | A kind of method and device being aligned two sides breast image |
| CN110599528A (en) * | 2019-09-03 | 2019-12-20 | 济南大学 | Unsupervised three-dimensional medical image registration method and system based on neural network |
-
2019
- 2019-12-31 CN CN201911407248.9A patent/CN111242877A/en active Pending
Patent Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150154756A1 (en) * | 2013-12-03 | 2015-06-04 | Viewray Incorporated | Single- And Multi-Modality Alignment Of Medical Images In The Presence Of Non-Rigid Deformations Using Phase Correlation |
| CN105761217A (en) * | 2016-01-29 | 2016-07-13 | 上海联影医疗科技有限公司 | Image reconstruction method and device |
| US20170337672A1 (en) * | 2016-05-20 | 2017-11-23 | Shenyang Neusoft Medical Systems Co., Ltd. | Image splicing |
| US20180182103A1 (en) * | 2016-12-23 | 2018-06-28 | International Business Machines Corporation | 3d segmentation reconstruction from 2d slices |
| CN107424162A (en) * | 2017-05-05 | 2017-12-01 | 上海联影医疗科技有限公司 | A kind of image partition method and system |
| CN107049345A (en) * | 2017-05-23 | 2017-08-18 | 深圳艾砾拓科技有限公司 | Exposure index and bias exponent computational methods in digital X-ray mammography system |
| WO2018222755A1 (en) * | 2017-05-30 | 2018-12-06 | Arterys Inc. | Automated lesion detection, segmentation, and longitudinal identification |
| CN107240131A (en) * | 2017-06-09 | 2017-10-10 | 大连理工大学 | A kind of breast image method for registering based on iteration texture Deformation Field |
| US20180374193A1 (en) * | 2017-06-21 | 2018-12-27 | International Business Machines Corporation | Automatic Image Registration Quality Assessment |
| CN108717707A (en) * | 2018-04-10 | 2018-10-30 | 杭州依图医疗技术有限公司 | A kind of tubercle matching process and device |
| CN109635835A (en) * | 2018-11-08 | 2019-04-16 | 深圳蓝韵医学影像有限公司 | A kind of breast lesion method for detecting area based on deep learning and transfer learning |
| CN109512464A (en) * | 2018-11-23 | 2019-03-26 | 清华大学深圳研究生院 | A kind of disorder in screening and diagnostic system |
| CN109754387A (en) * | 2018-11-23 | 2019-05-14 | 北京永新医疗设备有限公司 | Medical image lesion detects localization method, device, electronic equipment and storage medium |
| CN109872332A (en) * | 2019-01-31 | 2019-06-11 | 广州瑞多思医疗科技有限公司 | A kind of 3 d medical images method for registering based on U-NET neural network |
| CN110599528A (en) * | 2019-09-03 | 2019-12-20 | 济南大学 | Unsupervised three-dimensional medical image registration method and system based on neural network |
| CN110473235A (en) * | 2019-09-12 | 2019-11-19 | 杭州依图医疗技术有限公司 | A kind of method and device being aligned two sides breast image |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112465802A (en) * | 2020-12-10 | 2021-03-09 | 上海联影医疗科技股份有限公司 | Method, system, terminal and computer readable storage medium for processing tomographic image |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7945083B2 (en) | Method for supporting diagnostic workflow from a medical imaging apparatus | |
| Wang et al. | Smartphone-based wound assessment system for patients with diabetes | |
| JP5421756B2 (en) | Image display apparatus and method, and program | |
| CN110827335B (en) | Mammary gland image registration method and device | |
| US9619888B2 (en) | Method for defining a region of interest in a radiation image of a breast | |
| CN110969245A (en) | Target detection model training method and device for medical image | |
| CN110992376A (en) | CT image-based rib segmentation method, device, medium and electronic equipment | |
| CN111047610A (en) | Focal region presenting method and device | |
| US20130109953A1 (en) | Marking System for Computer-Aided Detection of Breast Abnormalities | |
| CN111476776B (en) | Chest lesion position determination method, system, readable storage medium and apparatus | |
| US20230125385A1 (en) | Auto-focus tool for multimodality image review | |
| US20150193948A1 (en) | Body motion detection device and method | |
| US20190125306A1 (en) | Method of transmitting a medical image, and a medical imaging apparatus performing the method | |
| US20150356757A1 (en) | System and method for reducing data transmission volume in tomosynthesis | |
| CN111161297B (en) | Method and device for determining edge of beam limiter and X-ray system | |
| CN111242877A (en) | Mammary X-ray image registration method and device | |
| CN113116365A (en) | Image acquisition method, device and system and storage medium | |
| CN111415333B (en) | Mammary gland X-ray image antisymmetric generation analysis model training method and device | |
| CN104605872A (en) | Radiation field control method and device for medical X-ray system | |
| CN110974286B (en) | Method and device for detecting breast X-ray image tumor | |
| CN111415332B (en) | Mammary gland X-ray image linkage method and device | |
| CN111429406B (en) | Mammary gland X-ray image lesion detection method and device combining multi-view reasoning | |
| CN111415741B (en) | Mammary gland X-ray image classification model training method based on implicit apparent learning | |
| WO2022127318A1 (en) | Scanning positioning method and apparatus, storage medium and electronic device | |
| JP7134678B2 (en) | Radiation image processing device, radiation image processing method and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |