WO2022141882A1 - 一种基于历史病理信息的病灶识别模型构建装置及系统 - Google Patents

一种基于历史病理信息的病灶识别模型构建装置及系统 Download PDF

Info

Publication number
WO2022141882A1
WO2022141882A1 PCT/CN2021/084408 CN2021084408W WO2022141882A1 WO 2022141882 A1 WO2022141882 A1 WO 2022141882A1 CN 2021084408 W CN2021084408 W CN 2021084408W WO 2022141882 A1 WO2022141882 A1 WO 2022141882A1
Authority
WO
WIPO (PCT)
Prior art keywords
contour
image data
lesion
image
data
Prior art date
Application number
PCT/CN2021/084408
Other languages
English (en)
French (fr)
Inventor
罗中宝
王海峰
唐章源
Original Assignee
上海睿刀医疗科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海睿刀医疗科技有限公司 filed Critical 上海睿刀医疗科技有限公司
Publication of WO2022141882A1 publication Critical patent/WO2022141882A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30096—Tumor; Lesion

Definitions

  • the invention belongs to the technical field of medical devices, and in particular relates to a lesion identification model building device and system based on historical pathological information.
  • embodiments of the present invention provide a lesion identification model building device and system based on historical pathological information.
  • a device for constructing a lesion identification model based on historical pathological information comprising:
  • a data acquisition module is used to acquire historical image image data and corresponding pathological slice image data, wherein the acquired image image data includes the first contour information of the organ, and the pathological slice image data includes the second contour information of the organ. contour information and lesion contour information;
  • a data registration module configured to register the imaging image data and the pathological slice image data based on the first contour information and the second contour information
  • a data mapping module configured to map the lesion outline information to the image image data after registration to form mapped image image data
  • a model building module is configured to build a model of the correspondence between the image data and the lesion contour information based on the mapped image data, where the model is used to predict the lesion contour of the new image data.
  • the device for constructing a lesion identification model further includes:
  • a feature verification module for verifying the accuracy of the model constructed by the model construction module.
  • the data registration module includes:
  • centroid alignment submodule for aligning the geometric centroids of the first contour and the second contour, or aligning the geometric centroids of the region within the first contour and the region within the second contour
  • the first registration sub-module is configured to perform rigid registration on the aligned second contour with the first contour as a reference to obtain a transformation matrix.
  • the data registration module further includes:
  • the second registration sub-module is configured to perform flexible registration on the rigidly registered second contour with the first contour as a reference.
  • the first registration sub-module includes:
  • a transformation part configured to obtain a rigid transformation matrix that maps the aligned second contour to the first contour
  • an interpolation unit configured to perform interpolation processing on the rigidly transformed second contour based on the first contour
  • an evaluation unit configured to evaluate the degree of matching between the second contour after the interpolation processing and the first contour
  • An optimization part configured to optimize the rigid transformation matrix in response to the matching degree not reaching a preset value; and using the rigid transformation matrix as the transformation matrix in response to the matching degree reaching a preset value.
  • the matching degree is characterized by the mean square error of the gray values of the second contour after interpolation and the first contour.
  • the model building module includes:
  • an extraction submodule for extracting a plurality of image features of the mapped image data
  • a determination sub-module for determining the correspondence between the plurality of image features and the lesion contour in the mapped image data
  • a construction submodule is used to construct the model based on the corresponding relationship.
  • the model is a convolutional neural network model, and the correspondence is represented by a network structure.
  • the network structure adopts a U-Net network structure or an optimized U-Net network structure
  • the construction submodule includes:
  • the training part is used to train the U-Net network structure and optimize the network parameters
  • the testing part is configured to test the network parameters based on the test data; in response to the test passing, the convolutional neural network model with the network parameters is used as the trained convolutional neural network model.
  • the testing unit tests the network parameters based on test data, including:
  • a system for identifying lesions using the device for constructing a lesion identification model based on historical pathological information as described in any preceding item including:
  • the lesion identification model construction device is used to construct a model of the correspondence between the image image data and the lesion outline information
  • the model application module is used for inputting new image data into the constructed model, and predicting lesion outline information or lesion position information in the new image data.
  • the device and system for constructing a lesion identification model based on historical pathological information proposed in the embodiment of the present invention by registering the pathological slice image data and the image image data, the lesions in the registered pathological slice image data are registered.
  • the contour information is mapped to the image image data, and the corresponding relationship between the image image data and the lesion contour information is constructed based on the mapped image image data, so that the constructed model can effectively and accurately predict the new image image data. Therefore, when real-time assistance is performed with the help of ultrasound images, the normal area of the tissue and the lesion area can be well distinguished from the ultrasound image, and the spatial position of the lesion can be accurately located.
  • FIG. 1 shows a schematic structural diagram of an apparatus for constructing a lesion identification model based on historical pathological information provided by an embodiment of the present invention
  • FIG. 2 shows a schematic structural diagram of an embodiment of a data registration module included in an apparatus for building a lesion identification model based on historical pathological information proposed by an embodiment of the present invention
  • FIG. 3 is a schematic structural diagram of an embodiment of a first registration submodule included in an apparatus for building a lesion identification model based on historical pathological information proposed by an embodiment of the present invention
  • FIG. 4 shows a schematic structural diagram of an embodiment of a model building module included in an apparatus for building a lesion identification model based on historical pathological information proposed by an embodiment of the present invention
  • FIG. 5 shows a schematic structural diagram of an embodiment of a construction submodule included in an apparatus for constructing a lesion identification model based on historical pathological information provided by an embodiment of the present invention
  • FIG. 6 shows a schematic structural diagram of a system for lesion identification using a lesion identification model construction device based on historical pathological information proposed by an embodiment of the present invention
  • FIG. 7 shows a schematic diagram of an embodiment in which the data acquisition module included in the device for constructing a lesion identification model based on historical pathological information according to an embodiment of the present invention saves patient data;
  • FIG. 8 shows a schematic diagram of the improved U-Net network structure adopted by an embodiment of the apparatus for constructing a lesion identification model based on historical pathological information proposed by an embodiment of the present invention.
  • FIG. 9 shows a schematic diagram of an improved U-Net network structure of a specific input size adopted by an embodiment of the apparatus for constructing a lesion identification model based on historical pathological information provided by an embodiment of the present invention.
  • the term “including” and its various variants can be understood as open-ended terms meaning “including but not limited to”.
  • the term “based on” may be understood as “based at least in part on”.
  • the term “one embodiment” may be understood to mean “at least one embodiment.”
  • the term “another embodiment” may be understood to mean “at least one other embodiment.”
  • the embodiments of the present invention propose a lesion identification model construction device and system based on historical pathological information.
  • FIG. 1 shows a schematic structural diagram of an apparatus for constructing a lesion identification model based on historical pathological information according to an embodiment of the present invention, the apparatus includes:
  • a data acquisition module is used to acquire historical image image data and corresponding pathological slice image data, wherein the acquired image image data includes the first contour information of the organ, and the pathological slice image data includes the second contour information of the organ. contour information and lesion contour information;
  • a data registration module configured to register the imaging image data and the pathological slice image data based on the first contour information and the second contour information
  • a data mapping module configured to map the lesion outline information to the image image data after registration to form mapped image image data
  • the model building module is configured to build a model of the correspondence between the image data and the lesion contour information based on the mapped image data, where the model is used to predict the lesion contour of the new image data.
  • the device for constructing a lesion identification model based on historical pathological information registers the image data of the historical patient and the corresponding pathological slice image data based on the organ contour information, and the pathological slice image data after the registration is registered.
  • the lesion contour information is mapped to the image image data, so that the image image data is mapped with the lesion contour information in the registered pathological slice image data, and the correspondence between the image image data and the lesion contour information is constructed based on the mapped image image data
  • the built model can effectively and accurately predict lesion contours in new imaging image data.
  • the image data includes but not limited to MRI (ie magnetic resonance imaging), PET (ie positron emission computed tomography), CT (ie X-ray electron computed tomography) and other images image data.
  • the contour of an organ can be identified from the image data, but it is difficult to accurately identify the contour of the lesion.
  • the first contour of the organ can be obtained from the image data through image recognition technology, or It can be obtained by manual delineation in the image image data, or obtained from the image image data by a combination of image recognition technology and manual correction. Of course, other acquisition methods can also be used, which will not be listed here.
  • the first contour of the organ is characterized by the first contour information.
  • the pathological slice image data is the image data obtained after sampling the organ tissue and performing slice processing.
  • the image data obtained by using the slice method can not only identify the contour of the organ, but also identify the contour of the lesion.
  • the second contour and lesion contour of the organ can be obtained from the pathological slice image data by using image recognition technology, manual manual delineation, or a combination of image recognition technology and manual correction.
  • the contour information is used to represent the second contour of the organ, and the lesion contour of the organ is represented by the lesion contour information.
  • the image data and corresponding pathological slices of an organ of a patient are collected, and a senior doctor or medical expert can directly outline the organ contour in the image image data, the organ contour in the pathological slice image data and the pathological slice image on the image image data.
  • the lesion contours of the data can be obtained to obtain the first contour, the second contour and the lesion contour of the organ.
  • One contour information, second contour information and lesion contour information are included in the image image data and corresponding pathological slices of an organ of a patient.
  • the first contour information, the second contour information and the lesion contour information of the organ can also be obtained through image recognition technology;
  • the first contour information, the second contour information and the lesion contour corresponding to the first contour information, the second contour information and the lesion contour information identified by the image recognition technology are manually corrected with the help of a senior doctor or medical expert, so as to effectively improve the obtained contour.
  • the accuracy of the information (including the first contour information, the second contour information and the lesion contour information), while reducing the workload of the personnel.
  • the data registration module passes The registration of the pathological slice image data and the image image data can help to accurately map the lesion contour included in the pathological slice image data to the image image data subsequently.
  • the data registration module realizes the registration of the imaging image data and the pathological slice image data through geometric centroid alignment and rigid registration.
  • the data registration module includes:
  • centroid alignment submodule for aligning the geometric centroids of the first contour and the second contour, or aligning the geometric centroids of the region within the first contour and the region within the second contour
  • the first registration sub-module is configured to use the first contour as a reference to rigidly register the aligned second contour to obtain a transformation matrix.
  • the transformation matrix reflects the correspondence between the pixel coordinates in the image image data and the pixel coordinates in the pathological slice image data.
  • the center of gravity alignment sub-module aligns the center of gravity of the pathological slice image data with the center of gravity of the image data through center-of-gravity alignment processing
  • the first registration sub-module aligns the center of gravity of the image data through rigid registration processing.
  • the second contour included in the pathological slice image data is aligned in size with the first contour included in the image image data through scaling, and the second contour is angularly aligned with the first contour through rotation, so that the pathological slice image is rigidly processed.
  • the second contour obtained after the transformation matrix is processed has a higher degree of matching with the first contour.
  • the geometric barycentric coordinates may be calculated based on the grayscale values of the pixel points of the image.
  • the aligning the geometric center of gravity of the first contour and the second contour includes: calculating the geometric center of gravity coordinates of the first contour based on the gray value of the pixel points on the first contour, and calculating the coordinates of the geometric center of gravity of the first contour based on the pixel points on the second contour Calculate the geometric barycentric coordinates of the second contour; then align the geometric barycentric coordinates of the first contour and the geometric barycentric coordinates of the second contour.
  • the aligning the geometric center of gravity of the region within the first contour and the region within the second contour includes: based on the grayscale values of pixels on the first contour and all pixels within the first contour Calculate the geometric barycentric coordinates of the first contour, and calculate the geometric barycentric coordinates of the second contour based on the gray values of the pixels on the second contour and all pixels in the second contour; then align the geometric barycentric coordinates of the first contour and the second contour.
  • the label image of the first contour and the label image of the second contour are obtained, and the gray value of the label image only takes 0 or the label value, that is, the gray value of the pixel at the contour position is the label value, and other positions
  • the gray value of the upper pixel is 0. Therefore, the geometric barycentric coordinates of the first contour are calculated according to the gray value of each pixel on the first contour, and the first contour is calculated according to the gray value of each pixel on the second contour.
  • the rigid registration of the image refers to determining a transformation matrix, the transformation matrix enables the same object to be mapped from the pathological slice image to the image image.
  • the rigid registration includes four steps of transformation, interpolation, evaluation and optimization.
  • the first registration sub-module includes:
  • a transformation part configured to obtain a rigid transformation matrix that maps the aligned second contour to the first contour
  • an interpolation unit configured to perform interpolation processing on the rigidly transformed second contour based on the first contour
  • an evaluation unit configured to evaluate the degree of matching between the second contour after the interpolation processing and the first contour
  • An optimization part configured to optimize the rigid transformation matrix in response to the matching degree not reaching a preset value; and using the rigid transformation matrix as the transformation matrix in response to the matching degree reaching a preset value.
  • the rigid transformation matrix with the highest matching degree is used as the transformation matrix.
  • the rigid transformation matrix can be continuously optimized by the optimization part, so that the matching degree between the second contour after interpolation and the first contour reaches the maximum, and when the matching degree reaches the maximum value, the A rigid transformation matrix is used as the transformation matrix.
  • the transformation part adopts a rigid transformation matrix for mapping the second contour to the first contour.
  • the interpolation unit may perform interpolation processing in a linear interpolation manner.
  • the evaluation unit may perform matching degree evaluation by means of mean square error evaluation.
  • the optimization unit may perform optimization processing in a gradient descent optimization manner.
  • the matching degree is represented by the mean square error of the gray values of the second contour after interpolation and the first contour. Since the grayscale values of different pathological slice images and image images are different, in this embodiment, in order to improve the accuracy of registration and the accuracy of mapping, during rigid registration, the original grayscale values of the pathological slice images and image images are not used. Instead, set the gray value of the contour as the label value, such as 1, and set the gray value of other areas as the label value 0, and then use the mean square error to evaluate the matching degree. The smaller the mean square error, the better the matching result. When the mean square error is 0, the two contours are completely aligned; the larger the mean square error, the worse the matching result.
  • a flexible registration link may be added on the basis of rigid registration.
  • the data registration module further includes: a second registration sub-module, configured to use the first contour as a reference to perform flexible registration on the rigidly registered second contour.
  • the flexible registration can use the B-spline deformation field technology, and the deformation field output by the registration acts on the second contour of the organ in the pathological slice image, the lesion contour of the organ in the pathological slice image, and the pathological slice image data to obtain the final registered data.
  • the data mapping module maps the lesion contour data of the organ in the registered pathological slice image to the image image data.
  • the image position of the organ in the newly obtained pathological slice image is completely aligned with the image position of the corresponding organ in the image slice image, based on The coordinate correspondence can map the lesion contour of the organ in the newly obtained pathological slice image to the image image data.
  • the contour feature corresponding to the mapped image image data will be the focus of follow-up attention.
  • image features may be obtained from the image image data by means of feature extraction for subsequent determination of the corresponding relationship.
  • the model building module includes:
  • an extraction submodule for extracting a plurality of image features of the mapped image data
  • a determination sub-module for determining the correspondence between the plurality of image features and the lesion contour in the mapped image data
  • a construction submodule is used to construct the model based on the corresponding relationship.
  • the model adopts a convolutional neural network model, and the corresponding relationship is represented by a network structure.
  • the model can be constructed not only by convolutional neural network, but also by other means, for example, by directly identifying specific features in the mapped image data, such as gray value, line density composed of pixels, The number of pixels in the region that meet a specific grayscale threshold, etc., and based on these specific features, it is determined which features are related to the lesion, thereby constructing the model and the structure of the model.
  • the network structure includes: a derivation formula and network parameters.
  • the derivation formula includes convolutional neural network structures such as convolutional structure, pooling structure, deconvolutional structure and softmax, and parameters used in these structures constitute network parameters.
  • a network structure with superior performance can be selected to represent the corresponding relationship.
  • the grid structure adopts a U-Net network structure or an optimized U-Net network structure, as shown in FIG. 5 , and the construction sub-modules include:
  • the training part is used to train the U-Net network structure and optimize the network parameters
  • the testing part is configured to test the network parameters based on the test data; in response to the test passing, the convolutional neural network model with the network parameters is used as the trained convolutional neural network model.
  • the trained convolutional neural network model has better prediction performance and can improve the accuracy of lesion contour prediction.
  • the overlap ratio is used to test whether the network parameters are qualified.
  • the test unit tests the network parameters based on the test data, including:
  • the lesion area here mainly refers to the area within the lesion outline, and of course, the lesion outline may also be included.
  • the constructed model can be verified for accuracy.
  • the device for building a lesion identification model further includes: a feature verification module for verifying the accuracy of the model constructed by the model building module. For example, input the image data of a new patient into the model, output the image position of the predicted lesion contour through the forward propagation of the model, and then take several biopsy samples from the inside and outside of the contour, and determine the current sample through pathological analysis. is positive. Experiments show that most of the samples inside the contour are positive, and most of the samples outside are negative, indicating that the constructed model is effective and the predicted contour is accurate.
  • the embodiment of the present invention also proposes a system for identifying lesions using a lesion identification model building device based on historical pathological information. As shown in FIG. 6 , the system includes:
  • the lesion identification model construction device is used to construct a model of the correspondence between the image image data and the lesion outline information
  • the model application module is used for inputting the new image data into the constructed relational model, and predicting the lesion outline information or the lesion position information in the new image data.
  • the information within the lesion outline can represent the lesion location information.
  • the result predicted by the model can be directly used for ablation.
  • the specific ablation process includes: firstly capturing image data for the subject, then inputting the captured image data into the trained model, and outputting the predicted lesion contour position. If the position is empty, it means that the subject is normal and there is no need to Ablated lesions, otherwise ablation can be performed for the predicted lesion contour.
  • the system for identifying lesions using the device for constructing a lesion identification model based on historical pathological information provided by the embodiment of the present invention has the same or similar technical content as the aforementioned device for constructing a lesion identification model based on historical pathological information.
  • the apparatus for constructing a lesion identification model based on historical pathological information similarly, the foregoing apparatus for constructing a lesion identification model based on historical pathological information can also refer to the description of the system provided in the embodiment of the present invention, and details are not repeated here.
  • the following will exemplify a specific embodiment proposed by the embodiments of the present invention by taking the organ as the prostate, the lesion as prostate cancer, and the model as a convolutional neural network model as an example , but should not be construed as a limitation to the embodiments of the present invention.
  • the data acquisition module collects the patient's image data (including but not limited to MRI, PET, CT, etc.) and the corresponding pathological slices, and a number of senior doctors outline the organ (prostate) contour of the pathological slice image data, and the pathological slice image data.
  • the data registration module registers the pathological slice image data and the image image data
  • the data mapping module maps the contour of the lesion (prostate cancer) of the pathological slice image data to the image image data, so that the precise position of the lesion (prostate cancer) is shown in the image image data;
  • the model building module builds a convolutional neural network based on the imaging image data showing the location of the lesion (prostate cancer), which can be used to determine the location of the lesion from the imaging image data that does not show the location of the lesion.
  • Constructing a convolutional neural network includes: extracting multiple image features from the obtained image image data showing the location of the lesion (prostate cancer), and determining the relationship between the multiple image features and the lesion, thereby constructing a convolution Neural network model.
  • the feature verification module verifies the accuracy of the constructed convolutional neural network model, for example, collects new patient imaging image data, determines the contour of the lesion (prostate cancer) in the imaging image data through the constructed convolutional neural network model, and then locates the contour inside the contour. And the outside of the needle biopsy, the results show that the constructed neural network is effective.
  • the model application module directly applies the convolutional neural network model to the new patient image data, determines the contour information of the lesion (prostate cancer), and performs subsequent ablation based on the determined contour information.
  • the data acquisition module acquires complete data of the patient including imaging image data and pathological slice image data.
  • the device for constructing a lesion identification model based on historical pathological information proposed in the embodiment of the present invention constructs a model based on historical pathological information. Therefore, each complete patient data should include image image data and pathological slice image data.
  • the image data can be stored in the dicom format, and the image data is desensitized data.
  • the image data is not desensitized, it will be stored after desensitization, for example, delete the patient sensitive information in dicom, such as address, phone number, etc.; then filter the data that does not contain the prostate, such as the lower abdomen, neck, and head; then divide the data containing the prostate into a sequence based on the coordinate system, and divide the sequences of the same coordinate system into a group, by A number of senior doctors select a sequence image with the most obvious prostate to delineate the outline of the prostate in the current group, and save the outline image of the prostate; The outline of the cancer is drawn and the outline information is saved.
  • the data of the same patient can be saved in the specified format.
  • Figure 7 shows a schematic diagram of the storage format of a patient data. From top to bottom, each layer represents a folder directory, as shown in the first layer. Folders are used to distinguish different patients, the second level of folders is to distinguish the image data and pathological slice data of the same patient, and so on.
  • the data registration module uses image image data as fixed data
  • the corresponding image image may also be called fixed image
  • pathological slice image data is used as moving data
  • the corresponding pathological slice image may also be called moving data image.
  • the coordinate system of the fixed data is used as the reference coordinate system and does not move or deflect; the moving data indicates that it needs to be moved or deflected to the reference coordinate system.
  • the image image data and pathological slice image data are obtained based on different time and different imaging equipment, in order to accurately map the prostate cancer area of the pathological slice image data to the image image data, it is necessary to firstly analyze the image image data and the pathological slice image data.
  • the data is registered, and after registration, the area of the prostate cancer is mapped.
  • the image data is tissue-level imaging
  • the pathological slice image data is cell-level imaging
  • direct registration of the two images is not effective.
  • the subsequent transformation matrix is applied to the pathological data to realize the registration of the pathological data.
  • the specific steps of registration can include:
  • Step 1 Align the geometric center of gravity of the prostate contour.
  • the geometric centers of gravity of the two prostates can be aligned first.
  • calculate the barycentric coordinates of the image V represents the gray value
  • (i, j) represents the coordinate
  • V(i, j) represents the gray value of the coordinate (i, j). The calculation formula is as follows:
  • the contour image is a label image, and the gray value of the image is only 0 or the label value, that is, V(i,j) here is equal to 0 or the label value, and the geometric center of gravity of the two prostate glands can be calculated according to the above formula, where the image image
  • the prostate geometric center of gravity is defined as (X fc , Y fc ), and the prostate geometric center of gravity of the pathological slice image is defined as (X pc , Y pc ).
  • the translation distance is equal to (X pc -X fc , Y pc -Y fc ), and the same translation and resampling operation needs to be performed on the pathological slice image data and the lesion outline of the pathological slice image.
  • Step 2 Prostate contour rigid registration.
  • the process of implementing rigid registration may include: performing rigid transformation on the points on the contour of the pathological slice image (moving image) through a rigid transformation matrix to generate a transformed contour, wherein the points on the transformed contour and the image image (fixed image) corresponds to the points on the contour; obtain the gray value of the points on the transformed contour through interpolation calculation; use the root mean square method for the information of all corresponding points to calculate the matching degree of these points; if If the matching degree does not meet the requirements, the transformation matrix is optimized by an optimization algorithm so that the matching degree satisfies the requirements (for example, the matching degree reaches the maximum), and the transformation matrix at this time is the required transformation matrix.
  • the prostate centroid of the pathological slice image data and the image data are relatively close.
  • the second step is to ensure that the two contours are aligned in size by scaling, and aligned in angle by rotation. , and the matching degree of the alignment needs to meet certain requirements.
  • the image images with the outline of the prostate and the pathological slice images with the outline of the prostate are input into the pre-configured registration framework of these four sub-modules with functions of transformation, interpolation, evaluation and optimization, and obtained through calculation. transformation matrix; and then resampling the pathological slice image based on the transformation matrix, so as to obtain a new pathological slice image after rotation, scaling, translation and other operations, and the new pathological slice image and the image image have a high degree of matching.
  • Step 3 Flexible registration of prostate contours. After completing the rigid registration in the second step, the prostate contour of the pathological slice image and the prostate contour of the imaging image are very close in the center of gravity, rotation angle and zoom. In order to improve the accuracy of subsequent lesion mapping, after rigid registration By adding flexible registration, the outline of the prostate in the pathological slice image and the outline of the prostate in the imaging image are completely coincident.
  • B-spline deformation field technology can be selected, and the deformation field output from the registration acts on the prostate contour of the pathological slice image, the lesion outline of the pathological slice image and the data of the pathological slice image, and finally the registered data is obtained.
  • the data mapping module maps the lesion outline data of the pathological slice image to the image image data.
  • the model constructed by the model building module adopts a convolutional neural network model.
  • Convolutional neural networks are widely used in image feature extraction, including semantic segmentation and object detection, due to their excellent model performance.
  • the network parameters are determined by training the network structure; the test data is used to verify whether the performance of the model meets the requirements; the trained network parameters are finally determined for subsequent feature verification modules and model applications module.
  • n the number of input images.
  • an effective way is to input all types of image data into the network, where all image data include T2, T1, DWI, ADC of MRI, and PET_5min, PET_1h, PET_2h of PET; another effective The method is to train the network separately for each type of image, and select the class of images with the best model performance for subsequent applications.
  • Wk x Hk represents the size of the feature map
  • Ck represents the number of feature maps
  • k is equal to 1, 2 or 3
  • the relationship between different image parameters is as follows:
  • the solid downward arrow in the figure represents the max pooling operation with a stride of 2 pixels.
  • the solid upward arrow in the figure represents the deconvolution operation, where the kernel size is 2*2 and the stride is 2.
  • a schematic diagram of the feature image size and the number of image feature maps for each layer of the network with a specific image input size (512, 512) is shown in Figure 9.
  • the network training process of the training department includes: first select the optimizer and loss function of the network parameters, for example, select Adam for the optimizer and dice loss for the loss function; then set the necessary hyperparameters, including setting the learning rate of the parameters to 0.0001, optimize The beta values of Adam were 0.9 and 0.999; the network parameters were initialized with kaiming distribution; then samples (image data) were randomly selected from the samples and sent to the network to obtain the prediction map of the network.
  • the testing department needs to test the network parameters, including: selecting a batch of samples that are not training data, respectively inputting them into the network loaded with the trained network parameters, outputting the predicted image, and combining the predicted image and
  • the gold standard (the outline of the lesion) is used to calculate the overlap rate. If the average overlap rate of the batch of samples is greater than the determined threshold, it means that the network training is effective. Otherwise, the number of samples for network training is increased, and the network parameters are retrained until the average overlap of the test data. rate is greater than a certain threshold. The parameters of the network structure are saved for subsequent use by the feature verification module.
  • the feature verification module can be verified by means of needle biopsy.
  • the model application module of the system can input the new image data into the network model to predict the lesion outline information or the lesion location in the new image data information, the results predicted by the network model can be directly used for ablation.
  • the specific ablation process includes: first, the patient captures image image data, then the image image data is input into the trained convolutional neural network, and the predicted lesion contour position is output. If the position is empty, it means that the current patient has no lesions that need to be ablated; If not empty, ablation can be performed for the predicted lesion outline.
  • a "computer-readable medium” can be any device that can contain, store, communicate, propagate, or transport the program for use by or in connection with an instruction execution system, apparatus, or apparatus.
  • computer readable media include the following: electrical connections with one or more wiring (electronic devices), portable computer disk cartridges (magnetic devices), random access memory (RAM), Read Only Memory (ROM), Erasable Editable Read Only Memory (EPROM or Flash Memory), Fiber Optic Devices, and Portable Compact Disc Read Only Memory (CDROM).
  • the computer readable medium may even be paper or other suitable medium on which the program may be printed, as the paper or other medium may be optically scanned, for example, followed by editing, interpretation, or other suitable medium as necessary process to obtain the program electronically and then store it in computer memory.
  • various parts of the present invention may be implemented in hardware, software, firmware or a combination thereof.
  • various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system.
  • a suitable instruction execution system For example, if implemented in hardware, as in another embodiment, it can be implemented by any one or a combination of the following techniques known in the art: Discrete logic circuits, application specific integrated circuits with suitable combinational logic gates, Programmable Gate Arrays (PGA), Field Programmable Gate Arrays (FPGA), etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Molecular Biology (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Medical Informatics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)
  • Measuring And Recording Apparatus For Diagnosis (AREA)

Abstract

本发明涉及一种基于历史病理信息的病灶识别模型构建装置及系统，其中装置包括：数据获取模块，用于获取历史的影像图像数据及对应的病理切片图像数据，其中，获取的影像图像数据包括器官的第一轮廓信息，病理切片图像数据包括器官的第二轮廓信息和病灶轮廓信息；数据配准模块，用于基于第一轮廓信息和第二轮廓信息配准影像图像数据和病理切片图像数据；数据映射模块，用于在配准后映射病灶轮廓信息到影像图像数据中；以及模型构建模块，用于基于映射后的影像图像数据构建影像图像数据与病灶轮廓信息之间对应关系的模型，以预测新影像图像数据的病灶轮廓。本发明实施例提出的方案可以通过构建的模型有效且准确地预测新影像图像数据中的病灶轮廓。

Description

一种基于历史病理信息的病灶识别模型构建装置及系统 技术领域
本发明属于医疗器械技术领域，具体涉及一种基于历史病理信息的病灶识别模型构建装置及系统。
背景技术
研究发现，癌症已成为危害人类健康的主要疾病之一。采用脉冲电场消融技术对病灶区域进行消融治疗，已经取得了可喜的进展。然而，由于超声图像和影像数据不能很好地区分出组织的正常区域和病灶区域，例如对于前列腺癌，现有的技术是借助穿刺活检来进行病灶定位，但是，穿刺活检也只能粗略地定位到病灶的大概位置而不能准确地确定病灶的轮廓。在实际治疗时，为了确保病灶被完全消融，往往会进行额外的消融，由此给病人带来不必要的伤害，并且耗时耗力。
发明内容
为了解决上述因不能准确地确定病灶的轮廓而导致额外消融的技术问题，本发明实施例提供了一种基于历史病理信息的病灶识别模型构建装置及系统。
在本发明的第一方面，提供一种基于历史病理信息的病灶识别模型构建装置，包括：
数据获取模块，用于获取历史的影像图像数据及对应的病理切片图像数据，其中，所述获取的影像图像数据包括器官的第一轮廓信息，所述病理切片图像数据包括所述器官的第二轮廓信息和病灶轮廓信息；
数据配准模块，用于基于所述第一轮廓信息和所述第二轮廓信息，配准所述影像图像数据和病理切片图像数据；
数据映射模块，用于在配准后，映射所述病灶轮廓信息到所述影像图像数据中，形成映射后的影像图像数据；以及
模型构建模块，用于基于所述映射后的影像图像数据，构建所述影像 图像数据与病灶轮廓信息之间对应关系的模型，所述模型用于预测新影像图像数据的病灶轮廓。
在某些实施例中，所述病灶识别模型构建装置还包括：
特征验证模块，用于验证所述模型构建模块构建的模型的准确性。
在某些实施例中，所述数据配准模块包括：
重心对齐子模块，用于对齐所述第一轮廓和所述第二轮廓的几何重心，或者对齐所述第一轮廓内区域和所述第二轮廓内区域的几何重心；以及
第一配准子模块，用于以所述第一轮廓作为基准，对所述对齐后的第二轮廓进行刚性配准，以获得变换矩阵。
在某些实施例中，所述数据配准模块还包括：
第二配准子模块，用于以所述第一轮廓作为基准，对所述刚性配准后的第二轮廓进行柔性配准。
在某些实施例中，所述第一配准子模块，包括：
变换部，用于获取将所述对齐后的第二轮廓映射到所述第一轮廓的刚性变换矩阵；
插值部，用于基于所述第一轮廓，对所述刚性变换后的第二轮廓进行插值处理；
评估部，用于对所述插值处理后的第二轮廓与所述第一轮廓进行匹配度评估；
优化部，用于响应于所述匹配度未达到预设值，对所述刚性变换矩阵进行优化；响应于所述匹配度达到预设值，将所述刚性变换矩阵作为所述变换矩阵。
在某些实施例中，所述匹配度通过所述插值处理后的第二轮廓与所述第一轮廓的灰度值的均方差来表征。
在某些实施例中，所述模型构建模块包括：
提取子模块，用于提取所述映射后的影像图像数据的多个图像特征；
确定子模块，用于确定所述多个图像特征与所述映射后的影像图像数据中病灶轮廓的对应关系；以及
构建子模块，用于基于所述对应关系构建所述模型。
在某些实施例中，所述模型为卷积神经网络模型，所述对应关系通过网络结构表征。
在某些实施例中，所述网络结构采用U-Net网络结构或优化后的U-Net网络结构，所述构建子模块包括：
训练部，用于训练U-Net网络结构，优化网络参数；以及
测试部，用于基于测试数据，对所述网络参数进行测试；响应于所述测试通过，将具有所述网络参数的卷积神经网络模型作为训练完成的卷积神经网络模型。
在某些实施例中，所述测试部基于测试数据，对所述网络参数进行测试，包括：
选取一组非训练使用的影像图像数据样本作为一组测试数据，分别输入至加载有所述网络参数的所述U-Net网络中，输出每个测试数据对应的预测图像；
分别将每个测试数据对应的预测图像与所述映射后的影像图像数据中的病灶区域进行重叠率计算，响应于该组内非训练使用的影像图像数据样本对应的平均重叠率大于预设阈值，所述测试通过。
在本发明的第二方面，提供一种采用如前任一项所述的基于历史病理信息的病灶识别模型构建装置进行病灶识别的系统，包括：
所述病灶识别模型构建装置，用于构建影像图像数据与病灶轮廓信息之间对应关系的模型；以及
模型应用模块，用于将新影像图像数据输入所述构建的模型，预测所述新影像图像数据中的病灶轮廓信息或病灶位置信息。
本发明的有益效果：本发明实施例提出的基于历史病理信息的病灶识别模型构建装置及系统，通过将病理切片图像数据与影像图像数据进行配准，配准后的病理切片图像数据中的病灶轮廓信息映射到影像图像数据中，并基于映射后的影像图像数据构建影像图像数据与病灶轮廓信息之间的对应关系构建模型，从而可以通过构建的模型有效且准确地预测新影像图像数据中的病灶轮廓，因此在借助超声图像进行实时辅助时，可以从超声图像很好地区分出组织的正常区域和病灶区域，准确定位病灶的空间位置。
附图说明
图1示出本发明实施例提出的基于历史病理信息的病灶识别模型构建装置的结构示意图；
图2示出本发明实施例提出的基于历史病理信息的病灶识别模型构建装置包括的数据配准模块的一个实施例的结构示意图；
图3示出本发明实施例提出的基于历史病理信息的病灶识别模型构建装置包括的第一配准子模块的一个实施例的结构示意图；
图4示出本发明实施例提出的基于历史病理信息的病灶识别模型构建装置包括的模型构建模块的一个实施例的结构示意图；
图5示出本发明实施例提出的基于历史病理信息的病灶识别模型构建装置包括的构建子模块的一个实施例的结构示意图；
图6示出本发明实施例提出的采用基于历史病理信息的病灶识别模型构建装置进行病灶识别的系统的结构示意图；
图7示出本发明实施例提出的基于历史病理信息的病灶识别模型构建装置包括的数据获取模块保存病人数据的一个实施例的示意图；
图8示出本发明实施例提出的基于历史病理信息的病灶识别模型构建装置的一个实施例所采用的改进的U-Net网络结构示意图；以及，
图9示出本发明实施例提出的基于历史病理信息的病灶识别模型构建装置的一个实施例所采用的具体输入尺寸的改进的U-Net网络结构示意图。
具体实施方式
为使本发明的目的、技术方案和优点更加清楚明白，以下结合具体实施例，并参照附图，对本发明进一步详细说明。但本领域技术人员知晓，本发明并不局限于附图和以下实施例。
如本文中所述，术语“包括”及其各种变体可以被理解为开放式术语，其意味着“包括但不限于”。术语“基于”可以被理解为“至少部分地基于”。术语“一个实施例”可以被理解为“至少一个实施例”。术语“另一实施例”可以被理解为“至少一个其它实施例”。
如前所述，由于超声图像和影像图像不能很好地区分出组织的正常区域和病灶区域，而现有的病灶定位技术也只能粗略地定位到病灶的大概位置而不能精确地确定病灶的轮廓，因此，在实际治疗时，为了确保病灶被完全消融，往往会进行额外的消融，基于此，本发明实施例提出了一种基于历史病理信息的病灶识别模型构建装置及系统。
下面结合附图对本发明实施例作进一步描述。图1示出了根据本发明的一个实施例的基于历史病理信息的病灶识别模型构建装置的结构示意图，所述装置包括：
数据获取模块，用于获取历史的影像图像数据及对应的病理切片图像数据，其中，所述获取的影像图像数据包括器官的第一轮廓信息，所述病理切片图像数据包括所述器官的第二轮廓信息和病灶轮廓信息；
数据配准模块，用于基于所述第一轮廓信息和所述第二轮廓信息，配准所述影像图像数据和病理切片图像数据；
数据映射模块，用于在配准后，映射所述病灶轮廓信息到所述影像图像数据中，形成映射后的影像图像数据；以及
模型构建模块，用于基于所述映射后的影像图像数据，构建所述影像图像数据与病灶轮廓信息之间对应关系的模型，所述模型用于预测新影像图像数据的病灶轮廓。
本发明实施例提出的基于历史病理信息的病灶识别模型构建装置，通过将历史病人的影像图像数据与其对应的病理切片图像数据基于器官轮廓信息进行配准，配准后的病理切片图像数据中的病灶轮廓信息映射到影像图像数据，使得影像图像数据中映射有配准后的病理切片图像数据中的病灶轮廓信息，并基于映射后的影像图像数据构建影像图像数据与病灶轮廓信息之间的对应关系构建模型，从而可以通过构建的模型有效且准确地预测新影像图像数据中的病灶轮廓。
其中，在一个实施例中，所述影像图像数据包括但不限于MRI(即磁共振成像)、PET(即正电子发射型计算机断层显像)、CT(即X线电子计算机断层扫描)等影像图像数据。通常，可以从影像图像数据中识别出器官的轮廓，但是很难准确地识别出病灶轮廓，在本实施例中，所述器官的 第一轮廓可以从影像图像数据中通过图像识别技术获取，也可以在影像图像数据中采用手动勾画的方式获取，还可以从影像图像数据中通过图像识别技术与人工修正相结合的方式获取，当然也可以采用其他获取方式，在此不一一列举。通过第一轮廓信息来表征器官的第一轮廓。
病理切片图像数据是对器官组织取样并进行切片处理后获得的图像数据，使用切片方法获得的图像数据不但可以识别出器官的轮廓，还能识别出病灶轮廓。在本实施例中，采用图像识别技术、人工手动勾画或图像识别技术与人工修正相结合的方法，可以在病理切片图像数据中获取所述器官的第二轮廓和病灶轮廓，其中，通过第二轮廓信息来表征器官的第二轮廓，通过病灶轮廓信息来表征器官的病灶轮廓。
例如，收集病人的某器官的影像图像数据和对应的病理切片，由资深的医生或医学专家在影像图像数据上直接勾画影像图像数据中的器官轮廓、病理切片图像数据的器官轮廓和病理切片图像数据的病灶轮廓，从而获得器官的第一轮廓、第二轮廓和病灶轮廓，通过实时采集勾画轨迹或者对器官的第一轮廓、第二轮廓和病灶轮廓进行信息化，由此可以获得器官的第一轮廓信息、第二轮廓信息和病灶轮廓信息。当然，也可以通过图像识别技术来获取器官的第一轮廓信息、第二轮廓信息和病灶轮廓信息；或者可以通过图像识别技术识别器官的第一轮廓信息、第二轮廓信息和病灶轮廓信息，同时借助资深的医生或医学专家对通过图像识别技术识别出的第一轮廓信息、第二轮廓信息和病灶轮廓信息对应的第一轮廓、第二轮廓和病灶轮廓进行人工修正，从而有效提高获取的轮廓信息(包括第一轮廓信息、第二轮廓信息和病灶轮廓信息)的准确度，同时减少人员的工作量。
由于影像图像数据和病理切片图像数据是在不同时间、不同成像设备所获得，并且影像图像数据是组织层面的成像，而病理切片图像数据是细胞层面的成像，因此，所述数据配准模块通过对病理切片图像数据和影像图像数据进行配准，可以有助于后续将病理切片图像数据包括的病灶轮廓准确地映射到影像图像数据中。
为了提高映射的准确性，在一个实施例中，所述数据配准模块通过几何重心对齐和刚性配准来实现所述影像图像数据和病理切片图像数据的配 准。具体地，如图2所示，所述数据配准模块包括：
重心对齐子模块，用于对齐所述第一轮廓和所述第二轮廓的几何重心，或者对齐所述第一轮廓内区域和所述第二轮廓内区域的几何重心；以及
第一配准子模块，用于以所述第一轮廓作为基准，刚性配准对齐后的所述第二轮廓，以获得变换矩阵。其中该变换矩阵反映了影像图像数据中像素点坐标与病理切片图像数据中像素点坐标之间的对应关系。
可见，本实施例中，所述重心对齐子模块通过重心对齐处理使得所述病理切片图像数据的重心与所述影像图像数据的重心对齐，第一配准子模块通过刚性配准处理使得所述病理切片图像数据包括的第二轮廓通过缩放与所述影像图像数据包括的第一轮廓在大小上对齐，并通过旋转使第二轮廓在角度上与第一轮廓对齐，从而使得病理切片图像经刚性变换矩阵处理后得到的第二轮廓与第一轮廓具有较高的匹配程度。
其中，在一个实施例中，可以基于图像的像素点的灰度值来计算几何重心坐标。具体地，所述对齐所述第一轮廓和所述第二轮廓的几何重心，包括：基于第一轮廓上像素点的灰度值计算第一轮廓的几何重心坐标，基于第二轮廓上像素点的灰度值计算第二轮廓的几何重心坐标；然后对齐第一轮廓的几何重心坐标和第二轮廓的几何重心坐标。在另一个实施例中，所述对齐所述第一轮廓内区域和所述第二轮廓内区域的几何重心，包括：基于第一轮廓上像素点和第一轮廓内所有像素点的灰度值计算第一轮廓的几何重心坐标，基于第二轮廓上像素点和第二轮廓内所有像素点的灰度值计算第二轮廓的几何重心坐标；然后对齐第一轮廓的几何重心坐标和第二轮廓的几何重心坐标。
在一个具体实施例中，获取第一轮廓的标签图像和第二轮廓的标签图像，标签图像的灰度值仅取0或者标签值，即轮廓位置上像素的灰度值为标签值，其他位置上像素的灰度值为0，因此，根据第一轮廓上每个像素点的灰度值计算出第一轮廓的几何重心坐标，根据第二轮廓上每个像素点的灰度值计算出第二轮廓的几何重心坐标。获得第一轮廓的几何重心坐标和第二轮廓的几何重心坐标之后，为了保证第一轮廓的几何重心坐标和第二轮廓的几何重心坐标的对齐，需要对第二轮廓的几何重心坐标进行平移重 采样等操作，相应地，在对第二轮廓的几何重心坐标进行平移重采样等操作时，第二轮廓上像素的坐标也需要相应地进行整体的平移重采样等操作。
在本发明实施例中，所述图像的刚性配准是指确定一个变换矩阵，该变换矩阵使得相同的目标从病理切片图像映射到影像图像。为了提高配准精度，在一个实施例中，所述刚性配准包括变换、插值、评估和优化四个环节。具体地，如图3所示，所述第一配准子模块，包括：
变换部，用于获取将所述对齐后的第二轮廓映射到所述第一轮廓的刚性变换矩阵；
插值部，用于基于所述第一轮廓，对所述刚性变换后的第二轮廓进行插值处理；
评估部，用于对所述插值处理后的第二轮廓与所述第一轮廓进行匹配度评估；
优化部，用于响应于所述匹配度未达到预设值，对所述刚性变换矩阵进行优化；响应于所述匹配度达到预设值，将所述刚性变换矩阵作为所述变换矩阵。
在一个可选实施例中，匹配度最大的刚性变换矩阵作为所述变换矩阵。具体地，可以通过优化部对所述刚性变换矩阵进行不断优化，使得所述插值处理后的第二轮廓与所述第一轮廓的匹配度达到最大，所述匹配度达到最大值时，所述刚性变换矩阵作为所述变换矩阵。
在一个可选实施例中，所述变换部采用刚性变换矩阵，用于将第二轮廓映射到第一轮廓。
经过刚性变换后，若经刚性变换后的第二轮廓像素点索引没有与之对应的经刚性变换前的第二轮廓像素点的索引，则通过插值的方式根据刚性变换前的第二轮廓的灰度值计算经刚性变换后的第二轮廓像素点的灰度值，在一个可选实施例中，所述插值部可以采用线性插值的方式进行插值处理。
在一个可选实施例中，所述评估部可以采用均方差评估的方式进行匹配度评估。
在一个可选实施例中，所述优化部可以采用梯度下降优化的方式进行 优化处理。
可选地，所述匹配度通过所述插值处理后的第二轮廓与所述第一轮廓的灰度值的均方差来表征。由于不同病理切片图像和影像图像的灰度值不同，本实施例为了提高配准的准确性，提高映射的精确度，在刚性配准时，不使用病理切片图像和影像图像各自原有的灰度值，而是将轮廓的灰度值设置为标签值，例如1，其他区域灰度值设置为标签值0，然后用均方差进行匹配度评估，均方差越小，说明匹配结果越好，在均方差为0时，两个轮廓完全对齐；均方差越大，说明匹配结果越差。
在一个实施例中，为了更进一步提高所述病灶轮廓信息映射到所述影像图像数据的精度，可以在刚性配准的基础上增加柔性配准的环节。具体地，所述数据配准模块还包括：第二配准子模块，用于以所述第一轮廓作为基准，对所述刚性配准后的第二轮廓进行柔性配准。在重心对齐子模块和第一配准子模块对病理切片图像进行处理后，病理切片图像中器官的第二轮廓与影像图像中器官的第一轮廓已经在重心、旋转角度和缩放上非常接近，再通过第二配准子模块进行柔性配准，可以使得病理切片图像中器官的第二轮廓与影像图像中器官的第一轮廓完全重合。在一个可选实施例中，柔性配准可以采用B样条变形场技术，通过配准输出的变形场作用于病理切片图像中器官的第二轮廓、病理切片图像中器官的病灶轮廓和病理切片图像数据，得出最后配准好的数据。
在本发明实施例中，所述数据映射模块将配准后的病理切片图像中器官的病灶轮廓数据映射到影像图像数据中。在一个实施例中，所述病理切片图像经过几何重心对齐、刚性配准和柔性配准后，新得到的病理切片图像中器官的图像位置和影像切片图像中对应器官的图像位置完全对齐，基于坐标对应关系，可以将新得到的病理切片图像中器官的病灶轮廓映射到影像图像数据中，此时映射后的影像图像数据对应的轮廓特征将是后续关注的重点。
在一个实施例中，为了提高运算的有效性，可以通过特征提取的方式从影像图像数据中获取图像特征用于后续对应关系的确定。具体地，如图4所示，所述模型构建模块包括：
提取子模块，用于提取所述映射后的影像图像数据的多个图像特征；
确定子模块，用于确定所述多个图像特征与所述映射后的影像图像数据中病灶轮廓的对应关系；以及
构建子模块，用于基于所述对应关系构建所述模型。
在一可选实施例中，所述模型采用卷积神经网络模型，所述对应关系通过网络结构表征。可以理解，所述模型不仅可以通过卷积神经网络构建模型，也可以通过其他方式构建模型，例如，直接识别映射后的影像图像数据中的特定特征，例如灰度值、像素构成的线条密度、区域内符合特定灰度阈值的像素个数等，基于这些特定特征判断哪些特征是与病灶有关的，由此构建所述模型以及模型的结构。
可选地，所述网络结构包括：推导公式和网络参数。所述推导公式包括：卷积结构、池化结构、反卷积结构和softmax等卷积神经网络结构，这些结构中所使用的参数构成网络参数。在一个可选实施例中，可以选用性能优越的网络结构来表征对应关系。具体地，所述网格结构采用U-Net网络结构或优化后的U-Net网络结构，如图5所示，所述构建子模块包括：
训练部，用于训练U-Net网络结构，优化网络参数；以及
测试部，用于基于测试数据，对所述网络参数进行测试；响应于所述测试通过，将具有所述网络参数的卷积神经网络模型作为训练完成的卷积神经网络模型。
所述训练完成的卷积神经网络模型具有更优的预测性能，可以提高病灶轮廓预测的准确性。
在一可选实施例中，采用重叠率来测试网络参数是否合格。具体地，所述测试部基于测试数据，对所述网络参数进行测试，包括：
选取一组非训练使用的影像图像数据样本作为一组测试数据，分别输入至加载有所述网络参数的所述U-Net网络中，输出每个测试数据对应的预测图像；
分别将每个测试数据对应的预测图像与所述映射后的影像图像数据中的病灶区域进行重叠率计算，响应于该组内非训练使用的影像图像数据样本对应的平均重叠率大于预设阈值，所述测试通过。
这里的所述病灶区域主要指的是病灶轮廓内的区域，当然也可以包括病灶轮廓。
为了验证模型的输出是否符合要求，以进一步提高模型预测的准确性，在一个可选实施例中，可以对构建的模型进行准确性验证。具体地，所述病灶识别模型构建装置还包括：特征验证模块，用于验证所述模型构建模块构建的模型的准确性。例如，将新病人的影像图像数据输入模型中，通过模型的前向传播输出预测的病灶轮廓所在的图像位置，然后在轮廓的内侧和外侧各取若干个活检样本，通过病理分析，判断当前样本是否为阳性。实验证明，轮廓内侧的大部分样本都是阳性，外侧的大部分样本都为阴性，表明构建的模型有效且预测的轮廓准确。
本发明实施例还提出了一种采用基于历史病理信息的病灶识别模型构建装置进行病灶识别的系统，如图6所示，所述系统包括：
所述病灶识别模型构建装置，用于构建影像图像数据与病灶轮廓信息之间对应关系的模型；以及
模型应用模块，用于将新影像图像数据输入所述构建的关系模型，预测所述新影像图像数据中的病灶轮廓信息或病灶位置信息。
可以理解，病灶轮廓内的信息可以表征病灶位置信息。
在一个实施例中，通过活检的方式验证模型的有效性后，可以将模型预测的结果直接用于消融。具体的消融流程包括：首先给被测者拍摄影像图像数据，然后将拍摄的影像图像数据输入训练好的模型中，输出预测的病灶轮廓位置，若位置为空，表示被测者正常，没有需要消融的病灶，否则可以针对预测的病灶轮廓进行消融。
为了节约篇幅，本发明实施例提供的采用基于历史病理信息的病灶识别模型构建装置进行病灶识别的系统与前述基于历史病理信息的病灶识别模型构建装置相同或相类似的技术内容，可参考前述基于历史病理信息的病灶识别模型构建装置的描述，同样，前述基于历史病理信息的病灶识别模型构建装置也可以参考本发明实施例提供的所述系统的描述，在此不再赘述。
下面以具体应用场景为例，对本发明实施例提出的技术方案进行示例 性说明，并非是对本发明实施例的限制。
为了更清楚地说明本发明实施例提出的技术方案，下面以器官为前列腺，病灶为前列腺癌，模型为卷积神经网络模型为例，对本发明实施例提出的一种具体实施例进行示例性描述，但不应理解为对本发明实施例的限制。
数据获取模块收集病人的影像图像数据(包括但不限于MRI、PET和CT等)和对应的病理切片，由多名资深的医生勾画病理切片图像数据的器官(前列腺)轮廓、病理切片图像数据的病灶(前列腺癌)轮廓以及影像图像数据中的器官(前列腺)轮廓；
数据配准模块对病理切片图像数据和影像图像数据进行配准；
数据映射模块将病理切片图像数据的病灶(前列腺癌)轮廓映射到影像图像数据中，从而使得影像图像数据中示出病灶(前列腺癌)的精准位置；
模型构建模块基于显示了病灶(前列腺癌)位置的影像图像数据，构建卷积神经网络，所述卷积神经网络可被用于从未显示病灶位置的影像图像数据中确定病灶的位置。
构建卷积神经网络包括：对获得的显示了病灶(前列腺癌)位置的影像图像数据进行多个图像特征的提取，并确定所述多个图像特征与病灶之间的关系，由此构建卷积神经网络模型。
特征验证模块验证构建的卷积神经网络模型的准确性，例如收集新的病人影像图像数据，通过构建的卷积神经网络模型确定影像图像数据中的病灶(前列腺癌)轮廓，然后在轮廓的内侧和外侧进行穿刺活检，结果表明构建的神经网络有效。
模型应用模块将卷积神经网络模型直接用于新的病人影像图像数据，确定病灶(前列腺癌)的轮廓信息，基于确定的轮廓信息进行后续消融。
在本实施例的一个示例中，数据获取模块获取病人的包括影像图像数据和病理切片图像数据的完整数据。本发明实施例提出的基于历史病理信息的病灶识别模型构建装置是基于历史病理信息构建模型，因此，对于每一个完整的病人数据，应该包括影像图像数据和病理切片图像数据。影像 图像数据可以采用dicom的格式进行存储，并且所述影像图像数据为脱敏数据，如果所述影像图像数据未脱敏，则在脱敏后存储，例如，删除dicom中的病人敏感信息，如地址，电话号码等；然后过滤不包含前列腺的数据，如下腹、颈部和头部等数据；接着基于坐标系对包含前列腺的数据进行序列划分，把相同坐标系的序列划分为一组，由多名资深的医生在当前组选择一个前列腺最明显的序列图像进行前列腺轮廓的勾画，将前列腺的轮廓图像保存；最后，由多名资深的医生在病理切片图像数据中进行前列腺的轮廓勾画和前列腺癌的轮廓勾画并保存轮廓信息。为了管理方便，可以将同一个病人的数据按照规定的格式进行保存，图7给出了一个病人数据的保存格式示意图，从上往下，每一层表示一个文件夹目录，如图第一层文件夹用于区分不同的病人，第二层文件夹是区分同一个病人的影像数据和病理切片数据，以此类推。
在本实施例的一个示例中，数据配准模块将影像图像数据作为固定数据，对应的影像图像也可以称为固定图像，病理切片图像数据作为移动数据，对应的病理切片图像也可以称为移动图像。固定数据的坐标系作为基准坐标系，不作移动或偏转；移动数据表示需要进行移动或偏转至基准坐标系。由于影像图像数据和病理切片图像数据是基于不同时间、不同成像设备所获得，为了能准确地将病理切片图像数据的前列腺癌区域映射到影像图像数据中，需要先对影像图像数据和病理切片图像数据进行配准，然后在配准之后，将前列腺癌的区域进行映射。考虑到影像图像数据是组织层面的成像，而病理切片图像数据是细胞层面的成像，直接对两个图进行配准效果不佳，为此，基于前列腺的轮廓特征进行配准，然后将配准之后的变换矩阵应用于病理数据，实现病理数据的配准。
配准的具体步骤可以包括：
第一步：前列腺轮廓的几何重心对齐。为了保证后续配准的过程中，使得两个前列腺轮廓的重叠率达到最大，可以先对两个前列腺的几何重心进行对齐。根据图像的几何矩，计算图像的重心坐标，V表示灰度值，(i,j)表示坐标，V(i,j)表示坐标(i,j)的灰度值，计算公式如下：
零阶矩：
Figure PCTCN2021084408-appb-000001
其中i，j为图像坐标索引
一阶矩：
Figure PCTCN2021084408-appb-000002
其中i，j为图像坐标索引
Figure PCTCN2021084408-appb-000003
其中i，j为图像坐标索引
图像重心：
Figure PCTCN2021084408-appb-000004
轮廓图像为标签图像，图像的灰度值仅为0或标签值，即此处的V(i,j)等于0或者标签值，根据如上公式可以计算出两个前列腺的几何重心，其中影像图像的前列腺几何重心定义为(X fc,Y fc)，病理切片图像的前列腺几何重心定义为(X pc,Y pc)，为了保证两者的坐标值一致，需要对病理切片图像的前列腺轮廓数据进行平移重采样操作，平移的距离等于(X pc-X fc,Y pc-Y fc),同时需要对病理切片图像数据和病理切片图像的病灶轮廓进行相同的平移重采样操作。
第二步：前列腺轮廓刚性配准。刚性配准实现的过程可以包括：通过刚性变换矩阵，对病理切片图像(移动图像)轮廓上的点进行刚性变换，以生成变换后的轮廓，其中，该变换后的轮廓上的点与影像图像(固定图像)轮廓上的点相对应；通过插值计算获得该变换后的轮廓上的点的灰度值；将所有对应点的信息采用均方根的方式，以计算这些点的匹配程度；如果匹配程度不满足要求，则通过优化算法对变换矩阵进行优化，使得匹配程度满足要求(例如匹配程度达到最大)，此时的变换矩阵为所求的变换矩阵。
在完成第一步的重心对齐处理之后，病理切片图像数据和影像图像数据的前列腺重心已经比较接近，第二步需要实现的是保证两个轮廓通过缩放在大小上对齐，通过旋转在角度上对齐，并且该对齐的匹配程度需要满足一定的要求。
具体的，将已勾画出前列腺轮廓的影像图像和已勾画出前列腺轮廓的病理切片图像输入预配置好的具有变换、插值、评估和优化功能的这四个子模块的配准框架中，通过计算获得变换矩阵；然后基于该变换矩阵对病 理切片图像进行重采样操作，从而获得经旋转、缩放、平移等操作后的新病理切片图像，该新病理切片图像与影像图像具有较高的匹配程度。
第三步：前列腺轮廓柔性配准。在完成第二步的刚性配准之后，病理切片图像的前列腺轮廓和影像图像的前列腺轮廓已经在重心、旋转角度和缩放上很接近了，为了提高后续病灶映射的精度，在刚性配准之后可以增加柔性配准，使得病理切片图像的前列腺轮廓和影像图像的前列腺轮廓完全重合。柔性配准可以选择B样条变形场技术，通过配准输出的变形场作用于病理切片图像的前列腺轮廓、病理切片图像的病灶轮廓和病理切片图像的数据，得出最后配准好的数据。
所述数据映射模块将病理切片图像的病灶轮廓数据映射到影像图像数据。通过第一步到第三步的处理，当前的病理切片图像中前列腺的图像位置和影像图像中前列腺的图像位置完全对齐，所以基于坐标对应关系，可以将当前的病理切片图像中病灶(前列腺癌)的轮廓映射到影像图像数据。
所述模型构建模块构建的模型采用卷积神经网络模型。卷积神经网络由于优秀的模型性能被广泛的应用于图像特征提取，包括语义分割和目标检测等。下面以改进的U-Net网络结构为例，通过训练网络结构，确定网络参数；通过测试数据验证模型的性能是否符合要求；最终确定训练好的网络参数，用于后续的特征验证模块和模型应用模块。
改进的U-Net网络结构如图8所示，图中的n表示输入的图像个数，在本实施例中，由于不同的影像图像数据包含的信息不一样，为了充分的利用不同影像图像数据的特点，一种有效的方式是将所有类型的影像图像数据都输入网络，这里的所有影像图像数据包括MRI的T2、T1、DWI、ADC，和PET的PET_5min、PET_1h、PET_2h；另一种有效的方法是对每一类型的图像进行单独的训练网络，选择模型性能最好的一类图像进行后续应用。
在图8所示的一个实施例中，Wk x Hk表示特征图的尺寸，Ck表示特征图的个数，k等于1、2或3，不同的图像参数的关系如下：
W1＝2*W2，W2＝2*W3
H1＝2*H2，H2＝2*H3
Figure PCTCN2021084408-appb-000005
图中的实心横向箭头表示卷积加上线性整流函数操作，其中卷积的核尺寸为3*3，图像向外填充1个像素，步长为1个像素，线性整流函数为f(x)＝max(0，x)。图中的实心向下箭头表示最大值池化操作，其中步长为2个像素。图中的实心向上箭头表示反卷积操作，其中核尺寸为2*2，步长为2。图中的空心横向箭头表示拼接过程，例如A＝[1,2],B＝[3,4]，则拼接的结果为C＝[1,2,3,4]。一个具体的图像输入尺寸(512,512)的网络每层的特征图像尺寸和图像特征图个数的示意图如图9所示。
训练部的网络训练过程包括：首先选择网络参数的优化器和损失函数，例如优化器选择Adam，损失函数选择dice loss；然后设定必要的超参数，包括参数的学习率设定为0.0001，优化器Adam的贝塔值为0.9和0.999；网络参数采用kaiming分布进行初始化；接着从样本中随机选取样本(影像图像数据)送入网络，获得网络的预测图，将预测图和金标准(病灶的轮廓)进行损失计算，把损失值输入优化器使优化器对网络参数进行更新；再次从样本中随机选取样本送入网络并更新网络参数，重复这个过程，直到损失值一直下降并趋于稳定(例如在某个很低的值附近进行小范围的波动)，表示网络训练完成，将网络参数进行保存。
测试部为了验证网络参数是否有效，需要对网络参数进行测试，包括：选取一批不是训练数据的样本，分别将它们输入加载训练好的网络参数的网络中，输出预测的图像，将预测图像和金标准(病灶的轮廓)进行重叠率计算，若该批样本的平均重叠率大于确定的阈值，则表示网络训练有效，否则增加网络训练的样本数量，重新训练网络参数，直到测试数据的平均重叠率大于确定的阈值。将网络结构的参数进行保存，用于后续的特征验证模块使用。
为了进一步验证卷积神经网络的输出是否符合要求，特征验证模块可以采用基于穿刺活检的方式进行验证。首先，将新病人的数据输入网络参数，通过网络的前向传播输出预测的病灶轮廓所在的图像位置，然后在轮 廓的内侧和外侧各取若干个活检样本，通过病理分析，判断当前样本是否为阳性，实验证明，轮廓内侧的大部分样本都是阳性，外侧的大部分样本都为阴性，表明当前的卷积神经网络参数有效且预测的轮廓准确。
通过活检的方式验证卷积神经网络参数的有效性之后，所述系统的模型应用模块可以将新影像图像数据输入所述网络模型，以预测所述新影像图像数据中的病灶轮廓信息或病灶位置信息，该网络模型预测的结果可直接用于消融。具体的消融流程包括：首先病人拍摄影像图像数据，然后将影像图像数据输入训练好的卷积神经网络，输出预测的病灶轮廓位置，若位置为空，表示当前病人没有需要消融的病灶；若位置不为空，可以针对预测的病灶轮廓进行消融。
本领域技术人员可以理解，在流程图中表示或在此以其他方式描述的逻辑和/或步骤，例如，可以被认为是用于实现逻辑功能的可执行指令的定序列表，可以具体实现在任何计算机可读介质中，以供指令执行系统、装置或设备(如基于计算机的系统、包括处理器的系统或其他可以从指令执行系统、装置或设备取指令并执行指令的系统)使用，或结合这些指令执行系统、装置或设备而使用。就本说明书而言，“计算机可读介质”可以是任何可以包含、存储、通信、传播或传输程序以供指令执行系统、装置或设备或结合这些指令执行系统、装置或设备而使用的装置。
计算机可读介质的更具体的示例(非穷尽性列表)包括以下：具有一个或多个布线的电连接部(电子装置)，便携式计算机盘盒(磁装置)，随机存取存储器(RAM)，只读存储器(ROM)，可擦除可编辑只读存储器(EPROM或闪速存储器)，光纤装置，以及便携式光盘只读存储器(CDROM)。另外，计算机可读介质甚至可以是可在其上打印所述程序的纸或其他合适的介质，因为可以例如通过对纸或其他介质进行光学扫描，接着进行编辑、解译或必要时以其他合适方式进行处理来以电子方式获得所述程序，然后将其存储在计算机存储器中。
应当理解，本发明的各部分可以用硬件、软件、固件或它们的组合来实现。在上述实施方式中，多个步骤或方法可以用存储在存储器中且由合适的指令执行系统执行的软件或固件来实现。例如，如果用硬件来实现， 和在另一实施方式中一样，可用本领域公知的下列技术中的任一项或它们的组合来实现：具有用于对数据信号实现逻辑功能的逻辑门电路的离散逻辑电路，具有合适的组合逻辑门电路的专用集成电路，可编程门阵列(PGA)，现场可编程门阵列(FPGA)等。
在本说明书的描述中，参考术语“一个实施例”、“一些实施例”、“示例”、“具体示例”、或“一些示例”等的描述意指结合该实施例或示例描述的具体特征、结构、材料或者特点包含于本发明的至少一个实施例或示例中。在本说明书中，对上述术语的示意性表述不一定指的是相同的实施例或示例。而且，描述的具体特征、结构、材料或者特点可以在任何的一个或多个实施例或示例中以合适的方式结合。
以上，对本发明的实施方式进行了说明。但是，本发明不限定于上述实施方式。凡在本发明的精神和原则之内，所做的任何修改、等同替换、改进等，均应包含在本发明的保护范围之内。

Claims (11)

  1. 一种基于历史病理信息的病灶识别模型构建装置，其特征在于，包括：
    数据获取模块，用于获取历史的影像图像数据及对应的病理切片图像数据，其中，所述获取的影像图像数据包括器官的第一轮廓信息，所述病理切片图像数据包括所述器官的第二轮廓信息和病灶轮廓信息；
    数据配准模块，用于基于所述第一轮廓信息和所述第二轮廓信息，配准所述影像图像数据和病理切片图像数据；
    数据映射模块，用于在配准后，映射所述病灶轮廓信息到所述影像图像数据中，形成映射后的影像图像数据；以及
    模型构建模块，用于基于所述映射后的影像图像数据，构建所述影像图像数据与病灶轮廓信息之间对应关系的模型，所述模型用于预测新影像图像数据的病灶轮廓。
  2. 根据权利要求1所述的基于历史病理信息的病灶识别模型构建装置，其特征在于，所述病灶识别模型构建装置还包括：
    特征验证模块，用于验证所述模型构建模块构建的模型的准确性。
  3. 根据权利要求1所述的基于历史病理信息的病灶识别模型构建装置，其特征在于，所述数据配准模块包括：
    重心对齐子模块，用于对齐所述第一轮廓和所述第二轮廓的几何重心，或者对齐所述第一轮廓内区域和所述第二轮廓内区域的几何重心；以及
    第一配准子模块，用于以所述第一轮廓作为基准，对所述对齐后的第二轮廓进行刚性配准，以获得变换矩阵。
  4. 根据权利要求3所述的基于历史病理信息的病灶识别模型构建装置，其特征在于，所述数据配准模块还包括：
    第二配准子模块，用于以所述第一轮廓作为基准，对所述刚性配准后的第二轮廓进行柔性配准。
  5. 根据权利要求3所述的基于历史病理信息的病灶识别模型构建装置，其特征在于，
    所述第一配准子模块，包括：
    变换部，用于获取将所述对齐后的第二轮廓映射到所述第一轮廓的刚性变换矩阵；
    插值部，用于基于所述第一轮廓，对所述刚性变换后的第二轮廓进行插值处理；
    评估部，用于对所述插值处理后的第二轮廓与所述第一轮廓进行匹配度评估；
    优化部，用于响应于所述匹配度未达到预设值，对所述刚性变换矩阵进行优化；响应于所述匹配度达到预设值，将所述刚性变换矩阵作为所述变换矩阵。
  6. 根据权利要求5所述的基于历史病理信息的病灶识别模型构建装置，其特征在于，所述匹配度通过所述插值处理后的第二轮廓与所述第一轮廓的灰度值的均方差来表征。
  7. 根据权利要求1所述的基于历史病理信息的病灶识别模型构建装置，其特征在于，所述模型构建模块包括：
    提取子模块，用于提取所述映射后的影像图像数据的多个图像特征；
    确定子模块，用于确定所述多个图像特征与所述映射后的影像图像数据中病灶轮廓的对应关系；以及
    构建子模块，用于基于所述对应关系构建所述模型。
  8. 根据权利要求7所述的基于历史病理信息的病灶识别模型构建装置，其特征在于，所述模型为卷积神经网络模型，所述对应关系通过网络结构表征。
  9. 根据权利要求8所述的基于历史病理信息的病灶识别模型构建装置， 其特征在于，所述网络结构采用U-Net网络结构或优化后的U-Net网络结构，所述构建子模块包括：
    训练部，用于训练U-Net网络结构，优化网络参数；以及
    测试部，用于基于测试数据，对所述网络参数进行测试；响应于所述测试通过，将具有所述网络参数的卷积神经网络模型作为训练完成的卷积神经网络模型。
  10. 根据权利要求9所述的基于历史病理信息的病灶识别模型构建装置，其特征在于，所述测试部基于测试数据，对所述网络参数进行测试，包括：
    选取一组非训练使用的影像图像数据样本作为一组测试数据，分别输入至加载有所述网络参数的所述U-Net网络中，输出每个测试数据对应的预测图像；
    分别将每个测试数据对应的预测图像与所述映射后的影像图像数据中的病灶区域进行重叠率计算，响应于该组内非训练使用的影像图像数据样本对应的平均重叠率大于预设阈值，所述测试通过。
  11. 一种采用如权利要求1-10中任一项所述的基于历史病理信息的病灶识别模型构建装置进行病灶识别的系统，其特征在于，包括：
    所述病灶识别模型构建装置，用于构建影像图像数据与病灶轮廓信息之间对应关系的模型；以及
    模型应用模块，用于将新影像图像数据输入所述构建的模型，预测所述新影像图像数据中的病灶轮廓信息或病灶位置信息。
PCT/CN2021/084408 2020-12-30 2021-03-31 一种基于历史病理信息的病灶识别模型构建装置及系统 WO2022141882A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011631255.XA CN112734710A (zh) 2020-12-30 2020-12-30 一种基于历史病理信息的病灶识别模型构建装置及系统
CN202011631255.X 2020-12-30

Publications (1)

Publication Number Publication Date
WO2022141882A1 true WO2022141882A1 (zh) 2022-07-07

Family

ID=75608379

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/084408 WO2022141882A1 (zh) 2020-12-30 2021-03-31 一种基于历史病理信息的病灶识别模型构建装置及系统

Country Status (2)

Country Link
CN (1) CN112734710A (zh)
WO (1) WO2022141882A1 (zh)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115063290A (zh) * 2022-08-17 2022-09-16 腾讯科技（深圳）有限公司 图像处理方法、装置、设备、系统及存储介质
CN115578437A (zh) * 2022-12-01 2023-01-06 武汉楚精灵医疗科技有限公司 肠体病灶深度数据获取方法、装置、电子设备及存储介质
CN116246019A (zh) * 2023-02-27 2023-06-09 上海迪派生物科技有限公司 一种病理切片的3d重建方法、装置、设备及介质
CN116458945A (zh) * 2023-04-25 2023-07-21 杭州整形医院有限公司 一种儿童面部美容缝合路线的智能引导系统及方法
CN116485846A (zh) * 2022-09-21 2023-07-25 数坤(上海)医疗科技有限公司 进针位置确定方法、装置、电子设备及可读存储介质
CN116597988A (zh) * 2023-07-18 2023-08-15 济南蓝博电子技术有限公司 一种基于医疗信息的智慧医院运营方法及系统
CN117036923A (zh) * 2023-10-08 2023-11-10 广东海洋大学 一种基于机器视觉的水下机器人目标检测方法
CN118037714A (zh) * 2024-03-29 2024-05-14 华伦医疗用品(深圳)有限公司 基于gpu医疗内窥镜图像的处理方法、系统和介质
CN118153594A (zh) * 2024-04-24 2024-06-07 华中科技大学同济医学院附属协和医院 一种基于epso-lm算法的病灶位姿实时动态跟踪方法及系统

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113628325B (zh) * 2021-08-10 2024-03-26 海盐县南北湖医学人工智能研究院 一种小器官肿瘤演化的模型建立方法及计算机可读存储介质
CN113902780B (zh) * 2021-08-31 2023-02-21 数坤(北京)网络科技股份有限公司 一种影像配准方法、装置、设备和可读存储介质
CN114187334B (zh) * 2021-10-12 2022-09-13 武汉兰丁云医学检验实验室有限公司 基于HE染色、Ki67、P16结合的相邻切片图像叠合对齐方法

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3273387A1 (en) * 2016-07-19 2018-01-24 Siemens Healthcare GmbH Medical image segmentation with a multi-task neural network system
CN109754387A (zh) * 2018-11-23 2019-05-14 北京永新医疗设备有限公司 医学图像病灶检测定位方法、装置、电子设备及存储介质
CN109993733A (zh) * 2019-03-27 2019-07-09 上海宽带技术及应用工程研究中心 肺部病灶的检测方法、系统、存储介质、终端及显示系统
CN111310841A (zh) * 2020-02-24 2020-06-19 中南大学湘雅医院 医学图像分类方法、装置、设备、计算机设备和存储介质
CN111402305A (zh) * 2020-06-05 2020-07-10 嘉兴太美医疗科技有限公司 医学影像配准方法、系统和计算机可读介质
CN111915609A (zh) * 2020-09-22 2020-11-10 平安科技（深圳）有限公司 病灶检测分析方法、装置、电子设备及计算机存储介质
CN112084930A (zh) * 2020-09-04 2020-12-15 厦门大学 一种全视野数字病理切片的病灶区域分类方法及其系统

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9984462B2 (en) * 2015-01-05 2018-05-29 Case Western Reserve University Disease characterization from fused pathology and radiology data
CN104835169A (zh) * 2015-05-15 2015-08-12 三爱医疗科技（深圳）有限公司 一种前列腺图像融合方法
CN105139382A (zh) * 2015-08-03 2015-12-09 华北电力大学（保定） 一种冠状动脉内超声图像序列的弹性配准方法
CN105167795A (zh) * 2015-09-07 2015-12-23 胡漫 一种pet/ct宏观数字信息与病理微观信息匹配的方法
US10198822B2 (en) * 2016-10-27 2019-02-05 International Business Machines Corporation Systems and user interfaces for determination of electro magnetically identified lesions as included in medical images of differing perspectives
CN109584201A (zh) * 2018-09-14 2019-04-05 新影智能科技(昆山)有限公司 医学图像配准方法、配准系统、存储介质、及电子设备
CN109829880A (zh) * 2018-12-07 2019-05-31 清影医疗科技（深圳）有限公司 一种基于深度学习的ct图像检测方法、装置和控制设备
CN110766730B (zh) * 2019-10-18 2023-02-28 上海联影智能医疗科技有限公司 图像配准及随访评估方法、存储介质及计算机设备

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3273387A1 (en) * 2016-07-19 2018-01-24 Siemens Healthcare GmbH Medical image segmentation with a multi-task neural network system
CN109754387A (zh) * 2018-11-23 2019-05-14 北京永新医疗设备有限公司 医学图像病灶检测定位方法、装置、电子设备及存储介质
CN109993733A (zh) * 2019-03-27 2019-07-09 上海宽带技术及应用工程研究中心 肺部病灶的检测方法、系统、存储介质、终端及显示系统
CN111310841A (zh) * 2020-02-24 2020-06-19 中南大学湘雅医院 医学图像分类方法、装置、设备、计算机设备和存储介质
CN111402305A (zh) * 2020-06-05 2020-07-10 嘉兴太美医疗科技有限公司 医学影像配准方法、系统和计算机可读介质
CN112084930A (zh) * 2020-09-04 2020-12-15 厦门大学 一种全视野数字病理切片的病灶区域分类方法及其系统
CN111915609A (zh) * 2020-09-22 2020-11-10 平安科技（深圳）有限公司 病灶检测分析方法、装置、电子设备及计算机存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
ZHAO QIAN, CHANGZHENG SHI, JIALI LUO, DEWEI ZENG, TING CHEN: "Non-rigid registration of MＲ image and pathological image based on Demons algorithm", HENAN MEDICAL RESEARCH, HE NAN SHENG YI XUE KE XUE YUAN, CN, vol. 23, no. 6, 30 June 2014 (2014-06-30), CN , pages 9 - 11, XP055948979, ISSN: 1004-437X, DOI: 10.3969/j.issn.1004-437X2014.06.003 *

Cited By (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115063290A (zh) * 2022-08-17 2022-09-16 腾讯科技（深圳）有限公司 图像处理方法、装置、设备、系统及存储介质
CN116485846A (zh) * 2022-09-21 2023-07-25 数坤(上海)医疗科技有限公司 进针位置确定方法、装置、电子设备及可读存储介质
CN116485846B (zh) * 2022-09-21 2024-01-23 数坤(上海)医疗科技有限公司 进针位置确定方法、装置、电子设备及可读存储介质
CN115578437A (zh) * 2022-12-01 2023-01-06 武汉楚精灵医疗科技有限公司 肠体病灶深度数据获取方法、装置、电子设备及存储介质
CN115578437B (zh) * 2022-12-01 2023-03-14 武汉楚精灵医疗科技有限公司 肠体病灶深度数据获取方法、装置、电子设备及存储介质
CN116246019B (zh) * 2023-02-27 2024-01-05 上海迪派生物科技有限公司 一种病理切片的3d重建方法、装置、设备及介质
CN116246019A (zh) * 2023-02-27 2023-06-09 上海迪派生物科技有限公司 一种病理切片的3d重建方法、装置、设备及介质
CN116458945A (zh) * 2023-04-25 2023-07-21 杭州整形医院有限公司 一种儿童面部美容缝合路线的智能引导系统及方法
CN116458945B (zh) * 2023-04-25 2024-01-16 杭州整形医院有限公司 一种儿童面部美容缝合路线的智能引导系统及方法
CN116597988A (zh) * 2023-07-18 2023-08-15 济南蓝博电子技术有限公司 一种基于医疗信息的智慧医院运营方法及系统
CN116597988B (zh) * 2023-07-18 2023-09-19 济南蓝博电子技术有限公司 一种基于医疗信息的智慧医院运营方法及系统
CN117036923B (zh) * 2023-10-08 2023-12-08 广东海洋大学 一种基于机器视觉的水下机器人目标检测方法
CN117036923A (zh) * 2023-10-08 2023-11-10 广东海洋大学 一种基于机器视觉的水下机器人目标检测方法
CN118037714A (zh) * 2024-03-29 2024-05-14 华伦医疗用品(深圳)有限公司 基于gpu医疗内窥镜图像的处理方法、系统和介质
CN118153594A (zh) * 2024-04-24 2024-06-07 华中科技大学同济医学院附属协和医院 一种基于epso-lm算法的病灶位姿实时动态跟踪方法及系统

Also Published As

Publication number Publication date
CN112734710A (zh) 2021-04-30

Similar Documents

Publication Publication Date Title
WO2022141882A1 (zh) 一种基于历史病理信息的病灶识别模型构建装置及系统
CN109741346B (zh) 感兴趣区域提取方法、装置、设备及存储介质
CN111160367B (zh) 图像分类方法、装置、计算机设备和可读存储介质
US9218661B2 (en) Image analysis for specific objects
WO2018120942A1 (zh) 一种多模型融合自动检测医学图像中病变的系统及方法
EP3365869B1 (en) System and method for image registration in medical imaging system
CN111292314B (zh) 冠脉分段方法、装置、图像处理系统和存储介质
CN107886508B (zh) 差分减影方法和医学图像处理方法及系统
Linguraru et al. Automated segmentation and quantification of liver and spleen from CT images using normalized probabilistic atlases and enhancement estimation
Nurmaini et al. Accurate detection of septal defects with fetal ultrasonography images using deep learning-based multiclass instance segmentation
CN110766730B (zh) 图像配准及随访评估方法、存储介质及计算机设备
CN111325714B (zh) 感兴趣区域的处理方法、计算机设备和可读存储介质
Oghli et al. Automatic fetal biometry prediction using a novel deep convolutional network architecture
CN111553892B (zh) 基于深度学习的肺结节分割计算方法、装置及系统
CN110288615B (zh) 一种基于深度学习的倾斜定位框定位方法
CN114092475B (zh) 病灶长径确定方法、图像标注方法、装置及计算机设备
CN116091466A (zh) 图像分析方法、计算机设备和存储介质
CN115661152B (zh) 基于模型预测的目标发展情况分析方法
CN115937163B (zh) 一种spect肺灌注成像的目标区域提取方法与系统
Zhang et al. CCS-net: cascade detection network with the convolution kernel switch block and statistics optimal anchors block in hypopharyngeal cancer MRI
Sun et al. Liver tumor segmentation and subsequent risk prediction based on Deeplabv3+
CN114010227B (zh) 一种右心室特征信息识别方法及装置
CN116205929A (zh) 一种超声融合成像方法及设备、存储介质
WO2012109641A2 (en) Systems, methods and computer readable storage mediums storing instructions for 3d registration of medical images
WO2024094088A1 (en) Systems and methods for image analysis

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21912652

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21912652

Country of ref document: EP

Kind code of ref document: A1