WO2023045284A1 - Procédé et appareil de traitement d'image, dispositif informatique, support de stockage et produit-programme informatique - Google Patents

Procédé et appareil de traitement d'image, dispositif informatique, support de stockage et produit-programme informatique Download PDF

Info

Publication number
WO2023045284A1
WO2023045284A1 PCT/CN2022/082195 CN2022082195W WO2023045284A1 WO 2023045284 A1 WO2023045284 A1 WO 2023045284A1 CN 2022082195 W CN2022082195 W CN 2022082195W WO 2023045284 A1 WO2023045284 A1 WO 2023045284A1
Authority
WO
WIPO (PCT)
Prior art keywords
object instance
image
attribute information
matching
instance
Prior art date
Application number
PCT/CN2022/082195
Other languages
English (en)
Chinese (zh)
Inventor
黄烨翀
陈翼男
Original Assignee
上海商汤智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海商汤智能科技有限公司 filed Critical 上海商汤智能科技有限公司
Publication of WO2023045284A1 publication Critical patent/WO2023045284A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30096—Tumor; Lesion

Definitions

  • the present disclosure relates to the technical field of image processing, and relates to but not limited to an image processing method, device, computer equipment, storage medium and program product.
  • Object detection related algorithms can detect regions of interest from input images, and are widely used in medical image analysis, such as lesion detection and other fields.
  • the existing process of providing medical images to determine the course of a lesion often involves multiple image sources. For example, multiple time-series images are required for follow-up, multiple images of different contrast stages are required for contrast imaging, and multiple images of different modalities are required for multimodal fusion diagnosis. images, etc., which involves the fusion of images; the current fusion method has the problem of large errors.
  • Embodiments of the present disclosure at least provide an image processing method, device, computer equipment, storage medium, and program product.
  • an embodiment of the present disclosure provides an image processing method, including:
  • the first object instance and the second object instance are fused.
  • the fusion of the first object instance and the second object instance can be performed with higher precision.
  • it also includes: acquiring a first original image and a second original image;
  • the first attribute information or the second attribute information includes at least one of the following:
  • the object instance includes: the first object instance and the second object instance.
  • the matching of the first object instance and the second object instance based on the first attribute information and the second attribute information includes:
  • the first attribute information and the second attribute information determine at least one of the following matching information between the first object instance and the second object instance: similarity, matching priority, distance, etc. effective radius;
  • a matching degree between the first object instance and the second object instance is obtained.
  • the obtaining the matching degree between the first object instance and the second object instance based on the matching information includes:
  • the matching degree between the first object instance and the second object instance can be obtained more accurately.
  • Matching the first object instance and the second object instance based on the first attribute information and the second attribute information to obtain the relationship between the first object instance and the second object instance match including:
  • the object instance pair For each object instance pair, according to the first attribute information corresponding to the first object instance included in the object instance pair and the second attribute information corresponding to the second object instance included in the object instance pair, the object instance pair The included first object instance is matched with the second object instance to obtain the matching degree of the object instance pair.
  • merging the first object instance and the second object instance based on the matching degree includes:
  • the object instance group For each object instance group, if the object instance group includes at least a first object instance and at least one second object instance, fuse the first object instance and the second object instance included in the object instance group.
  • the merging the first object instance and the second object instance includes:
  • the first attribute information of the first object instance and the second attribute information of the second object instance are fused.
  • an image processing device including:
  • a first determining module configured to determine first attribute information of a first object instance of the target object in the first image and determine second attribute information of a second object instance of the target object in the second image;
  • a matching module configured to match the first object instance and the second object instance based on the first attribute information and the second attribute information, to obtain the first object instance and the second object The degree of matching between instances;
  • a fusion module configured to fuse the first object instance and the second object instance based on the matching degree.
  • it also includes:
  • An acquisition module configured to acquire the first original image and the second original image
  • a second determination module configured to determine transformation relationship information between the first original image and the second original image
  • a transformation module configured to transform the first original image based on the transformation relationship information to obtain a first image, and use the second original image as the second image; or, based on the transformation relationship information Perform transformation processing on the second original image to obtain a second image, and use the first original image as the first image.
  • the attribute information includes at least one of the following:
  • the object instance includes: the first object instance and the second object instance.
  • the matching module includes:
  • a determining unit configured to determine at least one item of the following matching information between the first object instance and the second object instance based on the first attribute information and the second attribute information: similarity, matching priority level, distance, equivalent radius;
  • a matching unit configured to obtain a matching degree between the first object instance and the second object instance based on the matching information.
  • the matching unit is configured as:
  • the matching module is configured as:
  • the object instance pair For each object instance pair, according to the first attribute information corresponding to the first object instance included in the object instance pair and the second attribute information corresponding to the second object instance included in the object instance pair, the object instance pair The included first object instance is matched with the second object instance to obtain the matching degree of the object instance pair.
  • the fusion module is configured as:
  • the object instance group For each object instance group, if the object instance group includes at least a first object instance and at least one second object instance, fuse the first object instance and the second object instance included in the object instance group.
  • the fusion module is configured as:
  • the first attribute information of the first object instance and the second attribute information of the second object instance are fused.
  • an embodiment of the present disclosure further provides a computer device, including: a processor, a memory, and a bus, the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processing The processor communicates with the memory through a bus, and when the machine-readable instructions are executed by the processor, the above-mentioned first aspect, or the steps in any possible implementation manner of the first aspect are executed.
  • a computer device including: a processor, a memory, and a bus
  • the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processing
  • the processor communicates with the memory through a bus, and when the machine-readable instructions are executed by the processor, the above-mentioned first aspect, or the steps in any possible implementation manner of the first aspect are executed.
  • embodiments of the present disclosure further provide a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned first aspect, or any of the first aspects of the first aspect, may be executed. Steps in one possible implementation.
  • the embodiments of the present disclosure further provide a computer program product
  • the computer program product includes a non-transitory computer-readable storage medium storing the computer program
  • the computer program product may be a software installation package
  • the above computer program may be Operate to cause the computer to execute the steps in the above first aspect, or any possible implementation manner of the first aspect.
  • the image processing method, device, computer equipment, storage medium, and program product provided by the embodiments of the present disclosure, by matching the attribute information of the first object instance and the second object instance, the relationship between the first object instance and the second object instance is obtained. , and based on the matching degree, the first object instance and the second object instance are fused. In this way, by matching the attribute information of the first object instance and the second object instance, the matching degree between the first object instance and the object instance can be obtained, so that the first object instance and the second object instance can be matched according to the matching degree. more precise fusion.
  • FIG. 1 shows a flowchart of an image processing method provided by an embodiment of the present disclosure
  • FIG. 2 shows a schematic diagram of image fusion proposed by an embodiment of the present disclosure
  • FIG. 3 shows a flowchart of a method for determining transformation relationship information between a first original image and a second original image provided by an embodiment of the present disclosure
  • FIG. 4 shows a schematic diagram of multi-object instance matching proposed by an embodiment of the present disclosure
  • Fig. 5 shows a schematic diagram of an image processing device provided by an embodiment of the present disclosure
  • FIG. 6 shows a schematic diagram of another image processing device provided by an embodiment of the present disclosure.
  • FIG. 7 shows a schematic diagram of a matching module in an image processing device provided by an embodiment of the present disclosure
  • Fig. 8 shows a schematic diagram of a computer device provided by an embodiment of the present disclosure.
  • the detection performance and results of the target detection algorithm may be different. For example, lesions may disappear or increase during follow-up, and the appearance of lesions in different angiographic stages is different. In addition, the same patient is shooting When different images are used, there are spatial displacements and changes in their own background conditions; these two factors make it impossible to simply combine detection results from different sources.
  • For the detection of multi-source images there are currently two main solutions: one is to perform detection on the images from each source, and the reader establishes a connection by himself; the other is to use a registration algorithm to establish the spatial relationship between the two images , and then mapped from one source to another.
  • the first method does not solve the problem of multi-source image detection, and the second method partially solves the problem, but the current method of multi-source image fusion is generally to directly superimpose the registered images; this image fusion method causes The problem of poor fusion accuracy.
  • an embodiment of the present disclosure provides an image processing method, which obtains the matching degree between the first object instance and the object instance by matching the attribute information of the first object instance and the second object instance, so that the matching degree between the first object instance and the object instance can be obtained according to Matching degree, more accurate fusion of the first object instance and the second object instance.
  • an image processing method disclosed in the embodiment of the present disclosure is firstly introduced in detail.
  • the image processing method provided in the embodiment of the present disclosure is generally executed by a computer device with certain computing capabilities.
  • the image processing method may be implemented by a processor invoking computer-readable instructions stored in a memory.
  • FIG. 1 is a flowchart of an image processing method provided by an embodiment of the present disclosure, the method includes steps S101 to S103, wherein:
  • S101 Determine first attribute information of a first object instance of a target object in a first image and determine second attribute information of a second object instance of the target object in a second image.
  • the first image and the second image are images obtained by shooting the same target object.
  • the first image and the second image are images obtained by shooting the target object at different times;
  • the image obtained by shooting the target object can refer to the same diseased organ or the same body part of the same patient;
  • the first image and the second image can be the angles and distances taken at the same time for the same organ or the same body part Multiple images with different shooting parameters, or multiple images taken at different times for the same organ or the same body part.
  • the first object instance and the second object instance refer to a lesion that may exist on the first image and the second image respectively.
  • the attribute information includes at least one of the following:
  • the object instance includes: the first object instance in the first image and the second object instance in the second image.
  • an object instance is a result obtained through a target detection algorithm, and a certain post-processing and analysis may be performed on the detection result to obtain the object instance and its corresponding attribute information.
  • the target detection algorithm can be any method of detecting a specific target from an image, which is not limited here. When two images are detected, the corresponding target detection algorithms can be different, but the detected target types need to be the same, and the same Type attribute information.
  • the target detection algorithm can be Mask RCNN, Retina Net, etc.
  • the image may be detected by the above target detection algorithm, so as to obtain the attribute information corresponding to the object instance.
  • the position information may be the coordinate information of the object instance in the image, namely (x, y), which may be the coordinate information corresponding to the center point of the bounding box;
  • the size information may be the object instance The parameter information characterizing the size of an object instance, such as the size and radius of the instance;
  • the probability that the object instance belongs to the target object can be the probability or degree that the object instance has the target attribute determined based on a certain measurement standard, for example, a lesion corresponds to The degree of malignancy, etc.
  • the feature data can be coded for the hidden layer, that is, the feature vector corresponding to the object instance extracted by a certain network in the detection network;
  • the grayscale data of the image area corresponding to the object instance refers to the image area The gray value corresponding to each pixel;
  • the radiomics information is the description amount obtained by using the radiomics method to describe the features according to the image and range of the object instance.
  • other attribute information may also be included.
  • target detection processing may also be performed on the image, wherein the target detection method may be a method using semantic segmentation, that is, to determine which type of image each pixel in the image belongs to, and based on As a result of the target detection processing, the position information of the object instance in the image and the probability that the object instance belongs to the target object are obtained, and based on the position of the object instance in the image, determine and Grayscale data and/or radiomics information of the image region corresponding to the object instance.
  • semantic segmentation that is, to determine which type of image each pixel in the image belongs to, and based on
  • the position information of the object instance in the image and the probability that the object instance belongs to the target object are obtained, and based on the position of the object instance in the image, determine and Grayscale data and/or radiomics information of the image region corresponding to the object instance.
  • the first original image and the second original image are respectively original images taken for the target object. Since there is a certain positional deviation between the first original image and the second original image, for example, the overall translation of the second original image relative to the first original image is 10mm, etc., therefore, it is necessary to register the first original image and the second original image , so that the first original image and the second original image are in the same position in space, which facilitates the extraction and matching of later attribute information.
  • the second original image can be registered based on the position information of the first original image to obtain the first image and the transformed second image, or based on the position information of the second original image, the The first original image is registered to obtain the second image and the transformed first image.
  • FIG. 2 is a schematic diagram of image fusion proposed by an embodiment of the present disclosure.
  • Figure 2 it can be concluded that after object instance a is registered based on object instance A, the global spatial relationship 2 corresponding to object instance a is adjusted to the same global spatial relationship 1 as object instance A, and its corresponding coordinates, Dimensions are changed, but other properties not related to space are unchanged.
  • an embodiment of the present disclosure provides a flow chart of a method for determining transformation relationship information between a first original image and a second original image.
  • the method includes:
  • S301 Perform multi-level feature extraction on the first image to obtain the first target feature map corresponding to the multi-level feature extraction; and perform multi-level feature extraction on the second image to obtain the first target feature map corresponding to the multi-level feature extraction respectively The second target feature map of .
  • the first original image and the second original image may include different images taken for the same object at different times or at different angles.
  • the first original image and the second original image may be medical images obtained from multiple shots during one scan of the same lesion of the same patient , or, may be different medical images obtained during scanning the same lesion of the patient at different times, which is not limited here.
  • multi-level feature extraction can be performed on the two images respectively to obtain the first target feature map and the second target feature map respectively corresponding to the multi-level feature extraction.
  • target feature map can adopt the neural network self-learning method to realize the gradient return of the multi-level features of the image, and extract the high-level semantic features of the image.
  • multi-level feature extraction is performed on the image to obtain feature maps respectively corresponding to the multi-level feature extraction, including:
  • the first input data includes: the image, or the encoding feature output by the subsequent level of feature extraction Figure
  • the second input data includes: the decoding feature map output by the previous level of feature extraction
  • the decoding feature map corresponding to this level of feature extraction is determined as the target feature map corresponding to this level of feature extraction.
  • a feature extractor may be used to perform feature extraction on the first original image and the second original image respectively to obtain multi-level features corresponding to the two images.
  • the feature extractor is a feature pyramid network, which is divided into two stages of encoding and decoding, and connects the low-level and high-level networks by skip connections.
  • the feature extractor receives an image as input data, downsamples (also called “downsampling") and extracts features layer by layer in the encoding module, and upsamples (also called “upsampling”) and extracts features layer by layer in the decoding module.
  • the output of each decoding module is fed into an iterative registration network to form a multi-layer, coarse-to-fine pyramid feature.
  • the corresponding input data is the original image, that is, the first original
  • the corresponding input data is the encoding feature map corresponding to the output of the previous level, so that an iterative process of encoding feature extraction can be realized.
  • the lowest level that is, the most "thin” level in the “coarse to fine” structure of the feature "pyramid” network
  • the decoding feature map obtained by the corresponding lowest level encoding feature map
  • the corresponding input data is the decoding feature map corresponding to the output of the next level, and finally output the feature extraction result for the original image, so that the iterative process of decoding feature extraction can be realized.
  • the multi-level feature extraction described in the embodiments of the present disclosure indicates a multi-level feature extraction process from "fine” to "coarse", and each level of feature extraction includes an encoding process and a decoding process corresponding to the level of feature extraction, that is, From the bottom to the top of the "pyramid" network.
  • each level of feature extraction includes an encoding network corresponding to this level of feature extraction and a decoding network corresponding to this level of feature extraction;
  • each encoding network includes: an encoding module and a parallel domain adaptation module.
  • the decoding network corresponding to the first-level encoding network includes: a decoding module.
  • the decoding network corresponding to other encoding networks except the first-level encoding network includes: a decoding module and a gating fusion module.
  • the encoding process is a process from “coarse” to "fine", and the decoding process is sequentially decoded from “fine” to “coarse”.
  • the multi-level feature extraction described in the embodiments of the present disclosure corresponds to the decoding process.
  • the encoding network in each level of feature extraction it is used to perform encoding processing corresponding to the level of feature extraction on the first input data to obtain the encoding feature map corresponding to the level of feature extraction.
  • the decoding network in each level of feature extraction it is used to perform fusion processing on the encoding feature map corresponding to the level of feature extraction and the second input data to obtain a fusion feature map, and perform a fusion process on the fusion feature map with the level
  • the decoding process corresponding to the feature extraction obtains the decoding feature map corresponding to the feature extraction of this level.
  • encoding processing may be performed based on the first input data to obtain an encoding feature map corresponding to each level.
  • performing encoding processing corresponding to feature extraction of this level on the first input data to obtain an encoding feature map corresponding to feature extraction of this level includes:
  • the encoded feature map is obtained based on the downsampled feature map and the attention weights.
  • the encoding module When the first input data is input to the encoding module in each level of feature extraction, the encoding module will down-sample the first input data to obtain the corresponding down-sampling feature map, and input the output down-sampling feature map to each level In the parallel domain adaptation module corresponding to the encoding module.
  • the parallel domain adaptation module can enhance the expressive ability of the encoding network for some specific features in the encoding stage; for example, the image includes human organs; the feature extractor can be trained to enhance the expressive ability of organ texture features.
  • the parallel domain adaptation module After the downsampling feature map is input to the parallel domain adaptation module, the parallel domain adaptation module performs channel attention processing on the downsampling feature map to obtain the attention weights corresponding to each data channel in the downsampling feature map.
  • the above process includes:
  • the attention weights corresponding to each data channel in the downsampled feature map are obtained.
  • the parallel domain adaptation module includes two mechanisms: a channel attention mechanism and a domain awareness mechanism.
  • the channel attention mechanism determines the channel attention weights of multiple channels in the form of group convolution, thereby generating multiple candidate channel weights.
  • the domain-aware mechanism can combine the weights of multiple candidate channels according to the properties of the feature map to obtain the final result, so that when the performance of the first original image and the second original image are very different in the image domain, they can also be extracted through feature extraction.
  • the device obtains similar features, which is convenient for the subsequent registration process.
  • the parallel domain adaptation module is applied to the encoding stage of multi-level feature extraction, for example, in the encoding process from the i+1th level to the ith level, the corresponding parallel domain adaptation module is applied, for example, During the encoding process from level 3 to level 2, the parallel domain adaptation module 3 is applied.
  • FIG. 4 it is an example of a parallel domain adaptation module provided by an embodiment of the present disclosure.
  • the downsampling feature map corresponding to the encoding module of the i-th layer is expressed as (H, C, D, H, W), where N represents the number of feature maps included in a feature extraction (Number of instances in batch) , C represents the number of channels (Channel) of the feature map, D, H, and W represent the length, width and height (Depth, Height, Width) of the feature map, respectively, and B represents the branch of channel attention (Branches of attention).
  • the global average pooling process can be performed on the downsampled feature map, that is, the feature map Perform dimensionality reduction, average all pixels in the spatial dimensions of length, width, and height, and use it to reduce the dimensionality of the data, so that the feature map changes from (N, C, D, H, W) to (N, C), thus the overall information of the channel dimension can be extracted to obtain the first feature subgraph.
  • repeat Repeat
  • flatten Flatten
  • the first feature subgraph Transformed into the form of (N, BC, 1), where repetition refers to repeating the dimensionally reduced data B times, corresponding to B channel attention, where the repetition operation is to change the format of the data to facilitate calculation.
  • the feature map of this form is subjected to B-group convolution, activation and then B-group convolution processing, wherein the activation function can use a linear rectification function (Rectified Linear Unit, ReLU), and finally, the convolution is obtained
  • the channel reorganization is performed on the feature map of (N, C, B) to obtain the feature map of (N, C, B), so that the candidate attention weights corresponding to each data channel can be obtained.
  • the first feature submap is transformed into the form of (N, C), where flattening refers to rearranging the order of data, where flattening
  • the operation is to change the format of the data to facilitate calculation.
  • the feature map of this form is fully connected and activated.
  • the ReLU activation function and the activation function (Softmax) of the deep learning output layer can be used respectively to obtain a feature map of the form (N, B).
  • the feature domain weights of candidate attention corresponding to each data channel can be obtained.
  • the activation function can be an S-type growth curve (Sigmoid), so that the encoding feature map of the i-th level can be obtained.
  • the attention weights corresponding to each data channel in the downsampling feature map can be determined based on the feature domain weights of candidate attention corresponding to each data channel .
  • the image presented are different. Therefore, based on the pass domain
  • the feature domain weight determined by the perception module is used to match the feature domain weights of candidate attention corresponding to each data channel, highlighting the weight of the target position, that is, the weight of the liver position, and weakening other positions, such as muscles and blood. In this way, those positions can be prominently displayed for the target position. Therefore, even when the display effects of the first original image and the second original image are quite different, it can also be obtained by the feature extractor similar features for registration.
  • the channel attention mechanism can adopt multiple sets of parallel channel attention weight determination methods. For example, assuming that there are 12 channels, under normal circumstances, the above 12 channels can be convolved separately to obtain the candidate corresponding to the 12 channels. Attention weight, in order to improve the processing speed, the 12 channels can be divided into three channel attention, each of which includes 4 channels, so that the three channel attention can be convoluted at the same time, which improves the processing speed.
  • the encoding feature map can be obtained based on the downsampling feature map and attention weight, that is, the features of each channel in the downsampling feature map are calculated according to the corresponding
  • the attention weights are reorganized to obtain the encoded feature maps corresponding to each level.
  • the corresponding encoding feature map In the decoding process corresponding to each level of feature extraction, in addition to the second input data input to each level of decoding module, the corresponding encoding feature map also participates in the generation process of the encoding feature map.
  • the upsampled feature map of the previous decoding module has a lower spatial resolution but a higher degree of semantic information expression. Therefore, In order to combine the respective advantages of the upsampled feature map of the previous decoding module of the encoded feature map, in the embodiment of the disclosure, a gated fusion module that improves the fusion effect of high-level low-resolution features and low-level high-resolution features is used in the decoding stage .
  • performing fusion processing on the coded feature map corresponding to the level of feature extraction and the second input data to obtain the fusion feature map includes:
  • the encoding feature map of the encoding module data corresponding to the decoding module of this level is fused with the second feature data input to the decoding module to obtain the proportion of the encoding feature map corresponding to the feature extraction of this level.
  • the weight of the encoded feature map is multiplied by the weight to obtain a second feature submap, which is concatenated with the second feature data input to the decoding module to obtain a fusion feature map.
  • the encoded feature map from the encoding module is multiplied by the weight, and the decoding feature map of the decoding module is concatenated in the channel dimension, and sent to the next decoding module, so that the feature maps from the two sources can be fused more efficiently.
  • the function of the gated fusion module is mainly to determine the weights corresponding to each feature point in the encoding feature map, and extract the corresponding encoding feature map and the second input data based on the features of this level to obtain the The weights corresponding to each feature point in the coded feature map corresponding to level feature extraction, including:
  • weights corresponding to each feature point in the coded feature map corresponding to the level of feature extraction are obtained.
  • the eigenvalue of any feature point in the third eigensubgraph represents the autocorrelation coefficient of the image region corresponding to the feature point.
  • the third feature submap is convolved, normalized, and based on the linear rectification function (Rectified Linear Unit, ReLU) activation processing to obtain the fourth feature subgraph corresponding to the third feature subgraph, where convolution, normalization and activation processing are operations in a convolutional neural network (Convolutional Neural Network, CNN) Model (pattern), the number of occurrences of the above operations represents the depth of the network. The deeper the network, the stronger the expressive ability, and the larger the number of parameters. Here, in order to improve the expressive ability of the network, two or more layers can be selected. Convolution, normalization, and activation processing.
  • the encoding feature map and the local autocorrelation coefficient with the second input data can be obtained based on the fourth feature submap, including:
  • the maximum channel dimension and the average channel dimension are both aimed at the dimensionality reduction operation of the data, which is based on the channel dimension.
  • W becomes (N, 1, C, H, W), where the channel dimension maximum value and the channel dimension average value can be expressed as (N, 1, C, H, W).
  • the above two values can be spliced to obtain the spliced splicing result (N, 2, C, H, W), and then the splicing can be Convolution and normalization processing are performed on the result to obtain the local autocorrelation coefficient between the coded feature map and the second input data.
  • the activation function (sigmoid) can be used to activate the autocorrelation coefficient to obtain the gating activation value corresponding to each feature point in the encoding feature map; the gating activation value is used weights corresponding to each feature point in the coded feature map.
  • the fusion feature map can be obtained by multiplying the coded feature map by the gating activation value obtained through the activation function, and then concatenating it with the second input data.
  • the embodiment of the present disclosure also provides an example of a gating fusion module.
  • the encoded feature map output by the i-level encoding module and the decoded feature map of the data after the i-level decoding module has been down-sampled are spliced, where splicing refers to splicing in the channel dimension, requiring other channel dimensions
  • splicing refers to splicing in the channel dimension, requiring other channel dimensions
  • the same size, such as two data are (N, C1, D, H, W) and (N, C2, D, H, W), the size after splicing is (N, C1+C2, D, H , W).
  • the concatenated feature maps are processed by convolution, normalization, and activation, and then the channel dimension is maximized and the channel dimension is averaged.
  • the above two data are spliced, and convolution, normalization and activation processing are performed again, and after multiplication processing, the decoding feature of the i+1th level is obtained picture.
  • the fused feature map in the decoding stage is obtained, the fused feature map is subjected to decoding processing corresponding to the feature extraction of this level to obtain the decoding feature map corresponding to the feature extraction of this level, and the decoding feature map corresponding to the feature extraction of this level , to determine the target feature map corresponding to the feature extraction of this level, so as to facilitate the subsequent input of the target feature map corresponding to each level into the registration network.
  • the multi-level feature extraction method that is, the pyramid feature method
  • the parallel domain adaptation module and the gated fusion module are used in the encoding and decoding stages to enhance the generalization ability and robustness of the neural network and improve the abstraction ability of the feature extraction process.
  • the method for determining the transformation relationship information between the first original image and the second original image further includes:
  • S302 For each level of feature extraction, based on the first target feature map, the second target feature map corresponding to this level of feature extraction, and the first transformation relationship information corresponding to this level of feature extraction, determine the second transformation relationship corresponding to this level of feature extraction Information; wherein, the first transformation relationship information corresponding to this level of feature extraction includes: the second transformation relationship information corresponding to the previous level of feature extraction, or the original transformation relationship information between the first image and the second image.
  • the first transformation relationship information corresponding to this level of feature extraction includes: the second transformation relationship information corresponding to the previous level feature extraction, or the original transformation relationship information between the first original image and the second original image.
  • the original transformation relationship information is also called the initial deformation relationship.
  • the identity transformation can be used as the initial deformation relationship, that is, the initial image of the first original image and the second original image is used as input data ;
  • the previous linear registration network can solve the problem of poor accuracy and mismatch of Field of View (FoV) when the global deformation is too large.
  • FoV Field of View
  • the second target feature map and the first transformation relationship information corresponding to this level of feature extraction determine the level of feature extraction
  • the corresponding second transformation relationship information includes:
  • the iterative registration framework accepts an initial transformation relation in the form of a deformation field and several pairs of feature maps as input (each stage of registration corresponds to a stage of feature maps).
  • the iterative registration framework consists of multiple stages, and each stage includes a registration module and a combination module.
  • the registration module accepts the feature map pair of this stage and the transformation relationship of the previous stage (for the first stage, it is the initial transformation relationship) as input, and outputs the residual relative to the transformation relationship of the previous step, namely:
  • f( ⁇ ) is the registration module
  • ⁇ represents the applied transformation relationship
  • ⁇ i represents the cumulative transformation relationship in the i-th stage
  • ⁇ i represents the residual of ⁇ i relative to ⁇ i-1 .
  • the superscripts of ⁇ and ⁇ indicate the direction of the transformation relation, namely Represents the transformation relationship from the source image to the target image, Represents the transformation relationship from the target image to the source image.
  • the deformation field refers to adding the regular spatial grid to the predicted deformation field in the image registration to obtain the sampling grid, and using the sampling grid (sampling grid) containing deformation information for the floating image to obtain is the deformed image.
  • the size of the deformation field corresponding to a two-dimensional image with a size of [W, H] is [W, H, 2], where the size of the third dimension is 2, representing the displacement in the x-axis and y-axis directions, respectively.
  • the size of the deformation field corresponding to a three-dimensional image with a size of [D, W, H] is [D, W, H, 3], where the size of the third dimension is 3, respectively represented on the x-axis, y Axis and z-axis displacement.
  • Registration modules are divided into two types: linear transformation and deformation transformation, both of which are the same in input and output, so in application, you can freely combine linear and deformation transformation registration modules according to your needs, such as using a linear registration
  • the quasi-module is connected in series with three deformation transformation modules.
  • the two modules do not directly output ⁇ , but obtain ⁇ according to the transformation, that is, the output is the matrix corresponding to ⁇ .
  • the differential homeomorphism of the transformation process the calculation can be reduced The number of times, and because the output is a matrix, the process is reversible, providing a verification function.
  • the registration process proceeds as follows:
  • the original output of the linear registration module is the forward rotation, scaling, skew matrix Afw and the forward translation vector bfw.
  • Afw can be predicted directly through the network, or the parameters of rotation, scaling, and oblique cutting can be predicted separately, and then combined.
  • the reverse linear transformation relationship is obtained by inverting the linear transformation, namely:
  • the original output of the deformation transformation is the forward deformation relationship V fw in the form of velocity field, and on this basis, the deformation relationship ⁇ in the form of deformation field is obtained by integration, where the deformation relationship ⁇ satisfies the following formula (4) and formula (5) :
  • ⁇ represents the applied deformation relationship
  • the combination module accumulates the ⁇ of each previous step to obtain the cumulative deformation relationship ⁇ , namely (note that for forward and reverse deformations, the order of combination is reversed):
  • the registration neural network is used to perform multi-level fitting residuals, which not only has the advantage of fast registration speed of neural network learning, but also reduces the number of neural network errors. Learning the disadvantages of low accuracy when the global deformation is too large makes the registration speed and accuracy high.
  • the method for determining the transformation relationship information between the first original image and the second original image further includes:
  • the first original image and the second original image may be registered based on the second transformation relation information output after the last level of registration, ie final transformation information.
  • the last level of features can be used to extract the corresponding second transformation relationship information, and the first original image can be transformed to obtain the transformed image of the first original image , and then perform one-to-one position matching between the transformed image and the second original image to obtain a registration result.
  • the registration method involved is applied to a pre-trained registration neural network
  • the registration neural network includes two branch networks of a feature extraction neural network and a multi-level registration neural network
  • the feature extraction The neural network is used to perform multi-level feature extraction processing on the first original image and the second original image
  • the multi-level registration neural network is used to The multi-level features extracted from the image determine the target transformation relationship information between the first original image and the second original image, wherein the target transformation relationship information is used for the first original image and the second original image Two original images were registered.
  • the image processing method proposed by the embodiment of the present disclosure further includes:
  • S102 Based on the first attribute information and the second attribute information, match the first object instance and the second object instance to obtain the relationship between the first object instance and the second object instance match degree.
  • the first object instance and the second object instance may be matched based on the acquired attribute information. That is, based on the first attribute information and the second attribute information, determine at least one of the following matching information between the first object instance and the second object instance: similarity, matching priority, distance , the equivalent radius, and based on the matching information between the first object instance and the second object instance, the matching degree between the first object instance and the second object instance is obtained.
  • this embodiment of the present disclosure provides a method of obtaining the matching degree between the first object instance and the second object instance.
  • the matching information includes: similarity, matching priority, distance and equivalent radius ;
  • the obtaining the matching degree between the first object instance and the second object instance based on the matching information between the first object instance and the second object instance includes:
  • all possible object instances in the two images may be paired, and the similarity, matching priority, distance and equivalent radius between any two object instances may be calculated to form four matrices.
  • A represents the first object instance
  • a represents the second object instance, where:
  • Similarity is a measurement value. When the attributes of two object instances are closer, the value of similarity is greater. In order to be numerically controllable, it can be scaled or truncated to [0,1] or [-1, 1 Scope. Here, taking the radius of an object instance as an example, for the first object instance A and the second object instance a, the similarity can be:
  • the similarity of the radius is between [0,1], and the closer the two object instances are in the radius, the greater the similarity.
  • the similarity between other attribute information can also be calculated.
  • the attribute information includes: when the attribute information of the object instance includes a bounding box, the intersection-over-union (IoU) of the bounding boxes corresponding to the first object instance and the second object instance can be calculated, and This intersection is compared as the degree of similarity between the two; under the situation that the attribute information of object instance comprises contour, can calculate the Dika (Dice) coefficient of the contour corresponding to the first object instance and the second object instance respectively, and The Decca coefficient is used as the similarity between the two; when the attribute information of the object instance includes feature data, for example, in the case of a coded vector, the codes corresponding to the first object instance and the second object instance can be calculated respectively The cosine similarity of the vector, and the value of the cosine similarity is used as the similarity between the two, etc.
  • the similarities corresponding to multiple attributes can be weighted and summed, and the result of the weighted sum can be used as the similarity between the first object instance and the second object instance Spend.
  • the matching priority is also a kind of measurement value, and the higher the importance of the object instance, the greater its value.
  • the matching priority may represent the malignancy probability of the lesion, and the higher the malignancy probability and the larger the lesion, the higher the corresponding matching priority.
  • the matching priority can also be scaled and truncated to achieve numerical controllability. Taking the malignant probability of the lesion as an example, the matching priority can be expressed as:
  • the malignant probability is a value between [0,1]
  • the numerical distribution of the matching priority is between [0.5,1].
  • different matching priority metrics can be set for different attributes, or multiple matching priority metrics can be set for multiple attributes, or multiple matching priority metrics can be weighted, such as lesion deterioration speed, diffusion speed, and the like.
  • the distance is the distance between all possible paired object instances.
  • the Euclidean distance between the centers of the object instances can be used to represent the distance between the two object instances:
  • k represents the k-order norm
  • the second-order norm of the vector is the root sign after summing the squares of all components of the vector.
  • the equivalent radius is usually the average value of the radii or lengths of the diagonals of two object instances in a certain form.
  • its equivalent radius is The geometric mean of the radii of an object instance; when an object instance is expressed as a bounding box, its equivalent radius can be the length of a diagonal line, etc.: where, the equivalent radius is the geometric mean of the radii of two object instances It can be expressed as:
  • four matrices can be obtained, namely similarity matrix, matching priority matrix, distance matrix and equivalent radius matrix.
  • the i-th row and the j-th column represent the corresponding values of the i-th object instance in image 1 to the j-th object instance in image 2.
  • These four matrices can be used for further combinations in subsequent calculations.
  • the similarity matrix and matching priority matrix can be combined into a weighted similarity matrix.
  • the weighted similarity is the similarity after considering the matching priority, so the weighted similarity can be expressed as:
  • Weighted similarity similarity * matching priority.
  • the adjacency matrix is the relative positional relationship between two object instances calculated according to the distance and equivalent radius.
  • the adjacency relationship is used to measure whether two object instances are close or coincident in position. It is negatively related to the distance and positively related to the equivalent radius. For example, it can be expressed as follows:
  • a threshold can also be set. Due to the correction of the adjacency relationship, for example, it can be expressed as follows:
  • the final matching degree can be obtained.
  • the degree of matching determines how good or bad the match between two object instances is.
  • the degree of matching can be expressed as:
  • Matching degree weighted similarity * adjacency.
  • the first object instance Matching with the second object instance to obtain the matching degree between the first object instance and the second object instance includes:
  • the object instance pair For each object instance pair, according to the first attribute information corresponding to the first object instance included in the object instance pair and the second attribute information corresponding to the second object instance included in the object instance pair, the object instance pair The included first object instance is matched with the second object instance to obtain the matching degree of the object instance pair.
  • a plurality of first object instances and a plurality of second object instances may be grouped based on the corresponding matching degrees of the plurality of object instances and a preset matching degree threshold, so as to obtain multiple object instance groups; for each object instance group, if the object instance group includes at least a first object instance and at least one second object instance, the first object instance and the second object included in the object instance group Examples are merged.
  • the final matching result can be obtained from the matrix. That is, based on a threshold, select a pair whose matching degree is greater than the threshold as a feasible pairing, and transform the matrix into an undirected bipartite graph, each node on the graph is an object instance, and the weight of the edge is the matching degree; then According to the weight of the edge from high to low, pairing with replacement or without replacement is performed. Pairing with replacement will fuse all possible connected object instances into a new object instance; pairing without replacement will give priority to pairing two object instances with higher matching degrees. When an object instance is successfully paired, it will not be able to It is then paired with other object instances.
  • whether there is a return method is a configurable item, which is determined according to requirements.
  • one object instance can only be paired with another object instance at most, which is suitable for tracking the follow-up changes of pulmonary nodules, etc.; the pairing with replacement allows many-to-many matching, which is suitable for cross-modality Situations, such as tumor capsules that are not visualized in a particular modality, are not limited in the direction of application.
  • the successfully matched object instances will not continue to be matched, and can be removed from the pool of object instances that allow matching, but the object instances that have not yet been successfully matched. You can also continue to stay in the object instance pool and continue to wait for a match until all object instances in the object instance pool do not meet the matching conditions.
  • FIG. 4 is a schematic diagram of multi-object instance matching proposed by an embodiment of the present disclosure.
  • multiple object instance pairs that meet the threshold are screened out, including 0.7 between Aa, 0.6 between Ba, and 0.6 between Cc, and based on The methods with and without replacement get the final matching result.
  • the similarity, matching priority, position and size relationship, etc. are considered, and the matching accuracy is higher.
  • the image processing method proposed by the embodiment of the present disclosure further includes:
  • the first attribute information of the first object instance and the second attribute information of the second object instance may be fused.
  • attribute fusion can be performed.
  • image 1 is used as the reference space.
  • space-related attributes such as bounding box, center, contour, etc.
  • the coordinate transformation will be performed according to the registration relationship before fusion.
  • merging attributes different merging methods are used according to the type of attributes, which are not limited here. For example, for bounding boxes and outlines, etc., the union of all object instances is obtained; for dimensions, or based on the fused bounding box or outline. Calculate, or obtain the maximum value of all object instances; for the degree of malignancy, etc., obtain the maximum value of all object instances; for the hidden layer coding of the detection module, obtain the mean value of multiple object instances.
  • the coordinates and dimensions of the fused object instance are converted into coordinates and dimensions after registration according to the registration relationship, and the corresponding malignant degree is selected from the maximum value between object instance A and object instance a, and the feature The vectors are also averaged, thus completing the fusion process between the first object instance and the second object instance.
  • merging may be used for fusion to obtain a final result. For example, two-by-two object instances are fused, and then the two-by-two fusion results are fused; or, two object instances are arbitrarily selected for fusion, and the fusion result is fused with another object instance until the fusion with all object instances is completed.
  • the two merging methods depend on whether the multiple sources are in a parallel or hierarchical relationship.
  • multiple images can be linked, such as multi-modality, multi-phase contrast radiography, or multiple time-series follow-up results.
  • the matching method is more flexible, and it can better assist users in joint image reading and diagnosis.
  • the embodiments of the present disclosure are applied to the liver imaging diagnosis platform, for example.
  • Patients will be detected on the results of multi-phase angiography.
  • it can solve the problem of possible missed diagnosis in a single phase.
  • focal nodular hyperplasia is not obvious in the portal phase.
  • Metastatic tumors are often not obvious in the arterial phase, and the relationship of lesions in multiple phases is established for subsequent multi-phase combined diagnosis.
  • It can also be applied to the lung imaging diagnosis platform. According to multiple time-series lung images, nodule follow-up and analysis of nodule volume and sign changes are of great significance for the judgment of patients' condition.
  • the embodiments of the present disclosure first use a target detection algorithm to obtain the detection result corresponding to each image, that is, attribute information of an object instance, and then use a registration algorithm to obtain a spatial transformation relationship between any two images.
  • the similarity, matching priority, distance and equivalent radius of all possible detection target pairs in the two images are calculated sequentially, and the matching degree is calculated by combination to obtain the detection target matching of the two images
  • attribute fusion is performed again to obtain a fused object instance.
  • the matching degree between the first object instance and the object instance can be obtained, so that the first object instance and the second object instance can be matched according to the matching degree. more precise fusion.
  • the embodiment of the present disclosure also provides an image processing device corresponding to the image processing method. Since the problem-solving principle of the device in the embodiment of the present disclosure is similar to the above-mentioned image processing method in the embodiment of the present disclosure, the implementation of the device See the implementation of the method.
  • FIG. 5 is a schematic diagram of an image processing device provided by an embodiment of the present disclosure
  • FIG. 6 is a schematic diagram of another image processing device provided by an embodiment of the present disclosure
  • FIG. 7 It is a schematic diagram of a matching module in an image processing device provided by an embodiment of the present disclosure.
  • the device includes: a first determination module 510, a matching module 520, and a fusion module 530; wherein,
  • the first determination module 510 is configured to determine the first attribute information of the first object instance of the target object in the first image and determine the second attribute information of the second object instance of the target object in the second image;
  • the matching module 520 is configured to match the first object instance and the second object instance based on the first attribute information and the second attribute information to obtain the first object instance and the second object instance The degree of matching between object instances;
  • the fusion module 530 is configured to fuse the first object instance and the second object instance based on the matching degree.
  • An acquisition module 540 configured to acquire the first original image and the second original image
  • the second determination module 550 is configured to determine transformation relationship information between the first original image and the second original image
  • the transformation module 560 is configured to perform transformation processing on the first original image based on the transformation relationship information to obtain a first image, and use the second original image as the second image; or, based on the transformation relationship The information transforms the second original image to obtain a second image, and uses the first original image as the first image.
  • the attribute information includes at least one of the following:
  • the object instance includes: the first object instance and the second object instance.
  • the matching module 520 includes:
  • the determining unit 521 is configured to determine at least one item of the following matching information between the first object instance and the second object instance based on the first attribute information and the second attribute information: similarity, matching priority, distance, equivalent radius;
  • the matching unit 522 is configured to obtain a matching degree between the first object instance and the second object instance based on the matching information between the first object instance and the second object instance.
  • the matching unit 522 is configured as:
  • the matching module 520 is configured as:
  • the object instance pair For each object instance pair, according to the first attribute information corresponding to the first object instance included in the object instance pair and the second attribute information corresponding to the second object instance included in the object instance pair, the object instance pair The included first object instance is matched with the second object instance to obtain the matching degree of the object instance pair.
  • the fusion module 530 is configured to:
  • the object instance group For each object instance group, if the object instance group includes at least a first object instance and at least one second object instance, fuse the first object instance and the second object instance included in the object instance group.
  • the fusion module 530 is configured to:
  • the first attribute information of the first object instance and the second attribute information of the second object instance are fused.
  • the matching degree between the first object instance and the object instance is obtained by matching the attribute information of the first object instance and the second object instance, so that the first object instance and the second object instance can be matched according to the matching degree. Instances are more accurately fused.
  • FIG. 8 is a schematic structural diagram of the computer device provided by the embodiment of the present disclosure, including:
  • processor 801 memory 802, and bus 803; memory 802 is used to store execution instructions, including memory 8021 and external memory 8022; memory 8021 here is also called internal memory, and is used for temporarily storing the operation data in the processor 801, and The data exchanged by the external memory 8022 such as hard disk, the processor 801 exchanges data with the external memory 8022 through the memory 8021, when the computer device is running, the processor 801 communicates with the memory 802 through the bus 803, so that all The processor 801 executes the following instructions:
  • the first object instance and the second object instance are fused.
  • Embodiments of the present disclosure further provide a computer-readable storage medium, on which a computer program is stored, and when the computer program is run by a processor, the steps of the image processing method described in the foregoing method embodiments are executed.
  • the storage medium may be a volatile or non-volatile computer-readable storage medium.
  • the embodiment of the present disclosure also provides a computer program product, the computer program product carries a program code, and the instructions included in the program code can be used to execute the steps of the image processing method described in the above method embodiment, please refer to the above method implementation example.
  • the above-mentioned computer program product may be specifically implemented by means of hardware, software or a combination thereof.
  • the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK) etc. wait.
  • a software development kit Software Development Kit, SDK
  • the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.
  • the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium executable by a processor.
  • the technical solution of the present disclosure is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present disclosure.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disc and other media that can store program codes. .
  • Embodiments of the present disclosure provide an image processing method, device, computer equipment, storage medium, and program product, wherein the method includes: determining the first attribute information of the first object instance of the target object in the first image and the second attribute information of the second object instance of the target object in the second image; based on the first attribute information and the second attribute information, matching the first object instance and the second object instance, A matching degree between the first object instance and the second object instance is obtained; based on the matching degree, the first object instance and the second object instance are fused.
  • the matching degree between the first object instance and the object instance is obtained by matching the attribute information of the first object instance and the second object instance, so that the first object instance and the second object instance can be matched according to the matching degree. Instances are more accurately fused.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)

Abstract

Des modes de réalisation de la présente divulgation concernent un procédé et un appareil de traitement d'image, un dispositif informatique, un support de stockage et un produit-programme informatique. Le procédé consiste à : déterminer des premières informations d'attribut d'une première instance d'objet d'un objet cible dans une première image, et des secondes informations d'attribut d'une seconde instance d'objet de l'objet cible dans une seconde image ; mettre en correspondance la première instance d'objet avec la seconde instance d'objet sur la base des premières informations d'attribut et des secondes informations d'attribut pour obtenir le degré de correspondance entre la première instance d'objet et la seconde instance d'objet ; et fusionner la première instance d'objet et la seconde instance d'objet sur la base du degré de correspondance. Selon les modes de réalisation de la présente invention, les informations d'attribut de la première instance d'objet et les informations d'attribut de la seconde instance d'objet sont mises en correspondance pour obtenir le degré de correspondance entre la première instance d'objet et la seconde instance d'objet, de telle sorte que la première instance d'objet et la seconde instance d'objet puissent être fusionnées plus précisément selon le degré de correspondance.
PCT/CN2022/082195 2021-09-26 2022-03-22 Procédé et appareil de traitement d'image, dispositif informatique, support de stockage et produit-programme informatique WO2023045284A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111129299.7 2021-09-26
CN202111129299.7A CN114240809A (zh) 2021-09-26 2021-09-26 图像处理方法、装置、计算机设备及存储介质

Publications (1)

Publication Number Publication Date
WO2023045284A1 true WO2023045284A1 (fr) 2023-03-30

Family

ID=80743004

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/082195 WO2023045284A1 (fr) 2021-09-26 2022-03-22 Procédé et appareil de traitement d'image, dispositif informatique, support de stockage et produit-programme informatique

Country Status (2)

Country Link
CN (1) CN114240809A (fr)
WO (1) WO2023045284A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116453104A (zh) * 2023-06-15 2023-07-18 安徽容知日新科技股份有限公司 液位识别方法、装置、电子设备及计算机可读存储介质
CN116503603A (zh) * 2023-05-16 2023-07-28 北京交通大学 一种基于弱监督语义分割和特征弥补的类间遮挡目标检测网络模型的训练方法

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190043224A1 (en) * 2016-02-16 2019-02-07 Brainlab Ag Determination of Dynamic DRRs
CN112288708A (zh) * 2020-10-27 2021-01-29 推想医疗科技股份有限公司 Ct图像中淋巴结的检测方法、装置、介质及电子设备

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190043224A1 (en) * 2016-02-16 2019-02-07 Brainlab Ag Determination of Dynamic DRRs
CN112288708A (zh) * 2020-10-27 2021-01-29 推想医疗科技股份有限公司 Ct图像中淋巴结的检测方法、装置、介质及电子设备

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116503603A (zh) * 2023-05-16 2023-07-28 北京交通大学 一种基于弱监督语义分割和特征弥补的类间遮挡目标检测网络模型的训练方法
CN116503603B (zh) * 2023-05-16 2024-02-23 北京交通大学 一种基于弱监督语义分割和特征弥补的类间遮挡目标检测网络模型的训练方法
CN116453104A (zh) * 2023-06-15 2023-07-18 安徽容知日新科技股份有限公司 液位识别方法、装置、电子设备及计算机可读存储介质
CN116453104B (zh) * 2023-06-15 2023-09-08 安徽容知日新科技股份有限公司 液位识别方法、装置、电子设备及计算机可读存储介质

Also Published As

Publication number Publication date
CN114240809A (zh) 2022-03-25

Similar Documents

Publication Publication Date Title
Yin et al. [Retracted] U‐Net‐Based Medical Image Segmentation
Fu et al. LungRegNet: an unsupervised deformable image registration method for 4D‐CT lung
Lee et al. Cephalometric landmark detection in dental x-ray images using convolutional neural networks
Kroon et al. MRI modalitiy transformation in demon registration
Sotiras et al. Deformable medical image registration: A survey
WO2023045284A1 (fr) Procédé et appareil de traitement d'image, dispositif informatique, support de stockage et produit-programme informatique
Khalifa et al. State-of-the-art medical image registration methodologies: A survey
Pizer et al. Multiscale medial shape-based analysis of image objects
JP2018139693A (ja) 画像分類装置、方法およびプログラム
CN112102373A (zh) 一种基于强约束仿射形变特征学习的颈动脉多模态图像配准方法
WO2014120727A2 (fr) Procédé et appareil pour générer une image dérivée à l'aide d'images de différents types
Xu et al. 3D‐SIFT‐Flow for atlas‐based CT liver image segmentation
Corral Acero et al. SMOD-data augmentation based on statistical models of deformation to enhance segmentation in 2D cine cardiac MRI
Cheng et al. Fully automated prostate whole gland and central gland segmentation on MRI using holistically nested networks with short connections
CN110570394A (zh) 医学图像分割方法、装置、设备及存储介质
CN115082381A (zh) 一种基于u型网络的多目标分割方法及装置
Gaweł et al. Automatic spine tissue segmentation from MRI data based on cascade of boosted classifiers and active appearance model
Wu et al. A fully convolutional network feature descriptor: Application to left ventricle motion estimation based on graph matching in short-axis MRI
Palanivel et al. Mutifractals based multimodal 3D image registration
Yang et al. RAU-Net: U-Net network based on residual multi-scale fusion and attention skip layer for overall spine segmentation
Carvalho et al. Joint intensity‐and‐point based registration of free‐hand B‐mode ultrasound and MRI of the carotid artery
Haq et al. Deformable multisurface segmentation of the spine for orthopedic surgery planning and simulation
Javaid et al. Semantic segmentation of computed tomography for radiotherapy with deep learning: compensating insufficient annotation quality using contour augmentation
Olveres et al. Deformable models for segmentation based on local analysis
Preedanan et al. Urinary stones segmentation in abdominal X-ray images using cascaded U-net pipeline with stone-embedding augmentation and lesion-size reweighting approach

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22871349

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22871349

Country of ref document: EP

Kind code of ref document: A1