WO2021017481A1 - 图像处理方法及装置、电子设备和存储介质 - Google Patents

图像处理方法及装置、电子设备和存储介质 Download PDF

Info

Publication number
WO2021017481A1
WO2021017481A1 PCT/CN2020/079544 CN2020079544W WO2021017481A1 WO 2021017481 A1 WO2021017481 A1 WO 2021017481A1 CN 2020079544 W CN2020079544 W CN 2020079544W WO 2021017481 A1 WO2021017481 A1 WO 2021017481A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
feature map
target image
feature
image sequence
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2020/079544
Other languages
English (en)
French (fr)
Inventor
项磊
吴宇
赵亮
高云河
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Sensetime Intelligent Technology Co Ltd
Original Assignee
Shanghai Sensetime Intelligent Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Sensetime Intelligent Technology Co Ltd filed Critical Shanghai Sensetime Intelligent Technology Co Ltd
Priority to JP2021562337A priority Critical patent/JP2022529493A/ja
Priority to KR1020217031481A priority patent/KR20210134945A/ko
Publication of WO2021017481A1 publication Critical patent/WO2021017481A1/zh
Priority to US17/553,997 priority patent/US20220108452A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/174—Segmentation; Edge detection involving the use of two or more images
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/7715—Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/49—Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10081—Computed x-ray tomography [CT]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30008—Bone

Definitions

  • the present disclosure relates to the field of computer technology, and in particular to an image processing method and device, electronic equipment, and storage medium.
  • Bone injury has a relatively serious level of injury in accidents.
  • a high-intensity trauma caused by a fall from a height in an accident such as a traffic accident can cause bone damage such as fractures or cracks, and cause shock or even shock. death.
  • Medical imaging technology plays a very important role in the diagnosis and treatment of bones.
  • Three-dimensional Computed Tomography (CT) images can be used to show the anatomy and damage of the bone area. Based on the analysis of CT images, it is helpful for bone anatomy, surgical planning and postoperative recovery evaluation.
  • CT Computed Tomography
  • the analysis of CT images of bones can include the segmentation of the bone region, which requires manual positioning or manual segmentation of the bone region in each CT image.
  • the present disclosure proposes an image processing technical solution.
  • an image processing method including: acquiring an image sequence to be processed; determining an image sequence interval in which a target image is located in the image sequence to be processed, to obtain a target image sequence interval; The target image in the target image sequence interval is segmented, and the image area corresponding to at least one image feature class in the target image interval is determined. In this way, it is possible to automatically segment image regions of different image feature classes in the target image.
  • the determining the image sequence interval in which the target image is located in the image sequence to be processed to obtain the target image sequence interval includes: determining a sampling step of the image sequence; and according to the sampling step , Acquiring the images of the image sequence to obtain a sampled image; determining the sampled image with the characteristics of the target image according to the image characteristics of the sampled image; determining the position of the sampled image with the characteristics of the target image in the image sequence Describe the image sequence interval where the target image is located to obtain the target image sequence interval. In this way, it is possible to quickly determine the target image sequence interval where the target image is located, reduce the workload in the image processing process, and improve the efficiency of image processing.
  • the segmenting the target image in the target image sequence interval and determining the image area corresponding to at least one image feature class in the target image interval includes: based on the target image sequence interval
  • the target image and the preset relative position information are divided into the target image in the target image sequence interval, and the image area corresponding to at least one image feature class in the target image in the target image interval is determined.
  • the preset relative position information can be combined to reduce the error of the image area division.
  • the target image in the target image sequence interval is segmented based on the target image in the target image sequence interval and preset relative position information, and the target image in the target image interval is determined
  • the image area corresponding to at least one image feature class in the image includes: generating input information based on a preset number of continuous target images in the target image sequence interval and preset relative position information during an image processing cycle; The input information is subjected to at least one level of convolution processing to determine the image feature class to which each pixel in the target image in the target image interval belongs; and the target is determined according to the image feature class to which the pixel in the target image belongs The image area corresponding to at least one image feature class in the target image in the image interval.
  • the input target image is a continuous target image, which not only improves the efficiency of image processing, but also considers the associated information between the target images.
  • the convolution processing includes an up-sampling operation and a down-sampling operation, and the input information is subjected to at least one level of convolution processing to determine the image feature to which the pixel in the target image belongs
  • the class includes: obtaining a feature map of the down-sampling operation input based on the input information; performing a down-sampling operation on the feature map of the down-sampling operation input to obtain the first feature map output by the down-sampling operation; The first feature map output by the sampling operation is obtained to obtain the feature map input by the up-sampling operation; the up-sampling operation is performed on the feature map input by the up-sampling operation to obtain the second feature map output by the up-sampling operation; The second feature map output by the first-level up-sampling operation determines the image feature class to which each pixel in the target image belongs. In this way, through the up-sampling operation and the down-sampling operation, the image feature of the target image can be accurately extracted,
  • the convolution processing further includes a hole convolution operation;
  • the obtaining the feature map of the input of the upsampling operation based on the first feature map output by the downsampling operation includes:
  • the feature map of at least one level of hole convolution operation input is obtained; at least one level of hole convolution operation is performed on the feature map input by the at least one level hole convolution operation to obtain the The third feature map after the hole convolution operation; wherein the size of the third feature map obtained after the hole convolution operation decreases with the increase of the number of convolution processing stages; according to the hole convolution operation, it is obtained The third feature map of the upsampling operation is obtained. In this way, the local detailed information and global information of the target image can be combined to make the final image area more accurate.
  • the obtaining the feature map of the input of the upsampling operation according to the third feature map obtained after the hole convolution operation includes:
  • Feature fusion is performed on a plurality of third feature maps obtained after the at least one level of hole convolution operation to obtain a first fused feature map; based on the first fused feature map, a feature map input to the upsampling operation is obtained.
  • the feature map input by the upsampling operation can include more global information of the target image, which improves the accuracy of the image feature class to which the obtained pixel points belong.
  • the obtaining the feature map of the input of the upsampling operation based on the first feature map output by the downsampling operation includes:
  • the current upsampling operation is the first level of upsampling operation
  • the feature map of the current upsampling operation input is obtained; the current upsampling operation is greater than or equal to the first feature map.
  • the second feature map output by the previous upsampling is fused with the first feature map matching the same feature map size to obtain a second fused feature map; based on the second fused feature Figure to get the feature map of the current upsampling operation input. In this way, the feature map input by the current upsampling operation can be combined with the local detailed information and global information of the target image.
  • the method further includes: comparing the image feature class corresponding to the pixel in the target image in the target image interval with The labeled reference image feature classes are compared to obtain the comparison result; the first loss and the second loss in the image processing process are determined according to the comparison result; based on the first loss and the second loss, the image
  • the processing parameters used in the processing are adjusted so that the image feature class corresponding to the pixel in the target image is the same as the reference image feature class. In this way, the processing parameters used by the neural network can be adjusted through a variety of losses, so that the training of the neural network can have a better effect.
  • the adjusting processing parameters used in the image processing process based on the first loss and the second loss includes: obtaining a first weight corresponding to the first loss; A second weight corresponding to the second loss; based on the first weight and the second weight, weighting the first loss and the second loss to obtain a target loss; based on the target loss Adjust the processing parameters used in the image processing process.
  • the weight values of the first loss and the second loss can be set separately according to the actual application scenario, so that the training of the god network has a better effect.
  • the method before acquiring the image sequence to be processed, the method further includes: acquiring an image sequence formed by images acquired at a preset acquisition period; preprocessing the image sequence to obtain the image sequence to be processed Image sequence. In this way, irrelevant information of images in the image sequence can be reduced, and useful relevant information in the images can be enhanced.
  • the preprocessing the image sequence to obtain the image sequence to be processed includes: performing direction correction on the image of the image sequence according to the direction identifier of the image of the image sequence , Get the image sequence to be processed.
  • the images of the image sequence can be oriented according to the collection direction of the images, so that the collection direction of the images faces the preset direction.
  • the preprocessing the image sequence to obtain the image sequence to be processed includes: converting the images of the image sequence into an image of a preset size; The image in the center is cropped to get the image sequence to be processed. In this way, the center crop is performed on the image of the preset size, the irrelevant information in the image is deleted, and the useful and relevant information in the image is retained.
  • the target image is a pelvic computer tomography CT image
  • the image area includes one of a left hip area, a right hip area, a left femur area, a right femur area, and a spine area Or more.
  • the segmentation of one or more different regions among the left hip bone region, the right hip bone region, the left femur region, the right femur region and the spine region in the CT image can be realized.
  • an image processing apparatus including:
  • the obtaining module is used to obtain the image sequence to be processed; the determining module is used to determine the image sequence interval in which the target image is located in the image sequence to be processed to obtain the target image sequence interval; the segmentation module is used to compare the target image The target image in the sequence interval is segmented, and the image area corresponding to at least one image feature class in the target image interval is determined.
  • the determining module is specifically configured to determine the sampling step size of the image sequence; according to the sampling step size, the image of the image sequence is obtained to obtain the sampled image;
  • the image feature determines the sampled image with the target image feature; according to the arrangement position of the sampled image with the target image feature in the image sequence, the image sequence interval where the target image is located is determined to obtain the target image sequence interval.
  • the segmentation module is specifically configured to segment the target image in the target image sequence interval based on the target image in the target image sequence interval and preset relative position information, and determine An image area corresponding to at least one image feature class in the target image in the target image interval.
  • the segmentation module is specifically configured to generate input based on a preset number of continuous target images in the target image sequence interval and preset relative position information in an image processing cycle Information; perform at least one level of convolution processing on the input information to determine the image feature class to which pixels in the target image in the target image interval belong; determine the image feature class to which pixels in the target image belong An image area corresponding to at least one image feature class in the target image in the target image interval.
  • the convolution processing includes an up-sampling operation and a down-sampling operation
  • the segmentation module is specifically configured to obtain a feature map of the down-sampling operation input based on the input information; Perform a down-sampling operation on the feature map of the operation input to obtain the first feature map output by the down-sampling operation; obtain the feature map of the up-sampling operation input based on the first feature map output by the down-sampling operation;
  • the feature map input by the upsampling operation performs an upsampling operation to obtain the second feature map output by the upsampling operation; based on the second feature map output by the last-level upsampling operation, the image to which the pixel in the target image belongs is determined Feature class.
  • the convolution processing further includes a hole convolution operation;
  • the segmentation module is specifically configured to obtain at least one level of hole convolution based on the first feature map output by the last level of downsampling operation Operation input feature map; perform at least one level of hole convolution operation on the feature map input from at least one level of hole convolution operation to obtain a third feature map after the hole convolution operation; wherein, after the hole convolution operation The size of the obtained third feature map decreases as the number of convolution processing stages increases; according to the third feature map obtained after the hole convolution operation, the feature map input by the upsampling operation is obtained.
  • the segmentation module is specifically configured to perform feature fusion on a plurality of third feature maps obtained after the at least one level of hole convolution operation to obtain a first fused feature map; based on the first The feature maps are merged to obtain the feature maps input by the upsampling operation.
  • the segmentation module is specifically configured to obtain the current up-sampling operation according to the first feature map output by the last down-sampling operation when the current up-sampling operation is the first-stage up-sampling operation.
  • the feature map input by the sampling operation if the current up-sampling operation is greater than or equal to the second-level up-sampling operation, the second feature map output by the previous-level up-sampling and the first feature map matching the same feature map size Perform fusion to obtain a second fusion feature map; based on the second fusion feature map, obtain a feature map of the current upsampling operation input.
  • the device further includes: a training module, configured to compare image feature classes corresponding to pixels in the target image in the target image interval with the labeled reference image feature classes to obtain a comparison The result; the first loss and the second loss in the image processing process are determined according to the comparison result; the processing parameters used in the image processing process are adjusted based on the first loss and the second loss, so that all The image feature class corresponding to the pixel in the target image is the same as the reference image feature class.
  • a training module configured to compare image feature classes corresponding to pixels in the target image in the target image interval with the labeled reference image feature classes to obtain a comparison The result; the first loss and the second loss in the image processing process are determined according to the comparison result; the processing parameters used in the image processing process are adjusted based on the first loss and the second loss, so that all The image feature class corresponding to the pixel in the target image is the same as the reference image feature class.
  • the training module is specifically configured to obtain a first weight corresponding to the first loss and a second weight corresponding to the second loss; based on the first weight and the first weight Two weights, weighting the first loss and the second loss to obtain a target loss; adjust the processing parameters used in the image processing process based on the target loss.
  • the device further includes: a preprocessing module for acquiring an image sequence formed by images acquired at a preset acquisition period; preprocessing the image sequence to obtain an image to be processed sequence.
  • the preprocessing module is specifically configured to perform direction correction on the images of the image sequence according to the direction identification of the images of the image sequence to obtain the image sequence to be processed.
  • the preprocessing module is specifically configured to convert the images of the image sequence into an image of a preset size; perform center cropping on the image of the preset size to obtain the image to be processed sequence.
  • the target image is a pelvic computer tomography CT image
  • the image area includes one of a left hip area, a right hip area, a left femur area, a right femur area, and a spine area Or more.
  • an electronic device including:
  • a memory for storing processor executable instructions
  • the processor is configured to execute the above-mentioned image processing method.
  • a computer-readable storage medium having computer program instructions stored thereon, and the computer program instructions implement the above-mentioned image processing method when executed by a processor.
  • a computer program wherein the computer program includes computer-readable code, and when the computer-readable code runs in an electronic device, a processor in the electronic device executes To realize the above-mentioned image processing method.
  • the image sequence to be processed can be obtained, and then the image sequence interval in which the target image is located in the image sequence to be processed can be determined to obtain the target image sequence interval, so that the target image sequence can be performed on the determined target image sequence interval.
  • Image processing reduces the workload of image processing.
  • the target image in the target image sequence interval can be segmented to determine the image area corresponding to at least one image feature class in the target image sequence area. In this way, the image areas of different image feature classes in the target image can be automatically segmented, for example, The bone region in the CT image is segmented, saving human resources.
  • Fig. 1 shows a flowchart of an image processing method according to an embodiment of the present disclosure.
  • Fig. 2 shows a flowchart of preprocessing an image sequence according to an embodiment of the present disclosure.
  • Fig. 3 shows a flowchart of determining a target image sequence interval according to an embodiment of the present disclosure.
  • FIG. 4 shows a flowchart of determining the image area corresponding to each image feature class in the target image according to an embodiment of the present disclosure.
  • Fig. 5 shows a block diagram of an example of a neural network structure according to an embodiment of the present disclosure.
  • Fig. 6 shows a flowchart of an example of the above neural network training process according to an embodiment of the present disclosure.
  • Fig. 7 shows a block diagram of an image processing apparatus according to an embodiment of the present disclosure.
  • Fig. 8 shows a block diagram of an example of an electronic device according to an embodiment of the present disclosure.
  • the image processing solution provided by the embodiments of the present disclosure can determine the target image sequence interval in which the target image with the target image characteristics in the acquired image sequence is located, so that image processing can be performed on the target image in the target image sequence interval instead of the image Image processing for each image in the sequence can reduce the workload in the image processing process and improve the efficiency of image processing. Then segment the target image in the determined target image sequence interval to determine the image area corresponding to each image feature class in the target image.
  • the neural network can be used to process the target image, and the relative position information can be combined to make the image area corresponding to each image feature class determined in the target image Be more accurate and avoid obvious errors in segmentation results.
  • the image processing solution provided by the embodiments of the present disclosure can be applied to application scenarios such as image classification and image segmentation, and can also be applied to medical imaging in the medical field, for example, pelvic region annotation for CT images.
  • most of them are based on manually labeling the pelvic area, which is time-consuming and prone to errors.
  • This labeling method is also time-consuming. It takes more than ten minutes to label a three-dimensional CT image. .
  • the image processing solution provided by the embodiments of the present disclosure can quickly and accurately determine the pelvic area, and provide an effective reference for the diagnosis of the patient.
  • Fig. 1 shows a flowchart of an image processing method according to an embodiment of the present disclosure.
  • the image processing method can be executed by a terminal device, a server, or other image processing device, where the terminal device can be a user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital processing ( Personal Digital Assistant, PDA), handheld devices, computing devices, vehicle-mounted devices, wearable devices, etc.
  • the image processing method can be implemented by a processor calling computer-readable instructions stored in the memory.
  • the image processing method according to the embodiment of the present disclosure will be described below by taking the image processing device as the execution subject as an example.
  • the image processing method includes the following steps:
  • Step S11 Obtain the image sequence to be processed.
  • the image sequence may include at least two images, and each image in the image sequence may be sorted according to a preset arrangement rule to form an image sequence.
  • the preset arrangement rules can include time arrangement rules and/or space arrangement rules. For example, multiple images can be sorted according to the sequence of image acquisition time to form an image sequence, or they can be arranged in space according to the image acquisition position. The coordinates are sorted to form an image sequence.
  • the image sequence can be a set of CT images obtained by scanning a patient with CT.
  • the acquisition time of each CT image is different.
  • the acquired CT images can be formed into an image sequence according to the sequence of the acquisition time of the CT images.
  • the body part corresponding to each CT image can be different.
  • Fig. 2 shows a flowchart of preprocessing an image sequence according to an embodiment of the present disclosure.
  • the image sequence to be processed may be a preprocessed image sequence. As shown in Fig. 2, before the above step S11, the following steps may be further included:
  • S02 Preprocess the image sequence to obtain an image sequence to be processed.
  • images collected in a preset collection period can be acquired, and an image sequence can be formed from a set of images in the order of collection time.
  • each image can be preprocessed to obtain the preprocessed image sequence.
  • the preprocessing can include operations such as orientation correction, removal of abnormal pixel values, pixel normalization, and center clipping. After preprocessing, the irrelevant information of the images in the image sequence can be reduced, and the useful relevant information in the image can be enhanced.
  • the images of the image sequence may be oriented according to the direction identifier of the image sequence to obtain the image sequence to be processed.
  • Image sequence when the images in the image sequence are collected, they can carry collected related information. For example, they can carry collected related information such as the collection time of the image and the collection direction of the image, so that the images of the image sequence can be compared according to the collection direction of the image. Perform direction correction to make the image collection direction face the preset direction.
  • the CT image can be rotated to a preset direction.
  • the acquisition direction of the CT image is expressed as a coordinate axis
  • the x-axis or y-axis of the CT image can be parallel to the preset direction, so that the CT image can be Characterize the cross section of the human body.
  • the image of the image sequence when the image sequence is preprocessed to obtain the image sequence to be processed, the image of the image sequence may be converted into an image of a preset size, and then the image of the preset size may be centered Cut to get the image sequence to be processed.
  • the size of the images in the image sequence can be converted to a uniform size through image resampling or edge cropping, and then the center crop of the preset size image can be performed to delete irrelevant information in the image and retain useful related information in the image .
  • one or more CT images in the CT image sequence can be oriented to correct to ensure that the CT image represents the cross-section of the human body Structure; can remove the abnormal value in the CT image, make the pixel value of the pixel in the CT image in the interval of [-1024, 1024], and normalize the pixel value of the pixel in the CT image to [-1, 1]; Resample the CT image to a uniform scale, such as 0.8*0.8*1mm 3 ; Perform center cropping on the CT image, for example, perform center cropping to obtain a CT image with a size of 512*512 pixels, which is less than 512*512 pixels.
  • the pixel value of the image position can be set to a preset value, for example, set to -1.
  • Step S12 Determine the image sequence interval in which the target image is located in the image sequence to be processed, and obtain the target image sequence interval.
  • the image feature of each image in the image sequence can be extracted, or the image features of at least two images can be extracted, and then the target image with the target image feature in the image sequence can be determined according to the extracted image features, and then The arrangement position of the target image in the image sequence is determined. From the arrangement position of the two target images with the largest interval, the target image sequence interval in which the target image with the target image characteristics is located can be obtained.
  • a neural network can be used to perform image feature extraction on images in an image sequence, and a target image with target image features can be determined according to the extracted image features, and a target image sequence interval in which the target image is located can be further determined.
  • the target image sequence interval may be a part of the image sequence to be processed.
  • the image sequence includes 100 images, and the arrangement position of the target image is in the image sequence interval of 10-20, then the image sequence interval may be the target image Sequence interval.
  • Fig. 3 shows a flowchart of determining an image sequence interval according to an embodiment of the present disclosure.
  • step S12 may include the following steps:
  • Step S121 determining the sampling step size of the image sequence
  • Step S122 Obtain images of the image sequence according to the sampling step to obtain sampled images
  • Step S123 Determine a sampled image with target image characteristics according to the image characteristics of the sampled image
  • Step S124 Determine the image sequence interval in which the target image is located according to the arrangement position of the sampled image with the target image characteristics in the image sequence to obtain the target image sequence interval.
  • the sampling step size can be set according to the actual application scenario. For example, 30 images are used as the sampling step size, and every sampling step size, one image in the image sequence can be acquired, and the This image is used as a sample image.
  • the above-mentioned neural network can be used to extract the image features of the sampled images to determine the sampled images with target image features, and then the arrangement position of the sampled images with target image features in the image sequence can be determined.
  • two The arrangement position corresponding to the sampled images with target image characteristics may determine an image sequence interval, and the largest image sequence interval among the obtained multiple image sequence intervals may be used as the target image sequence interval where the target image is located.
  • the images in the target image sequence interval formed according to the sampled images with the target image characteristics also have the target image characteristics and are the target images.
  • the upper and lower boundaries of the image sequence interval may also be expanded, so that the final image sequence interval may include all target images.
  • the CT images in the CT image sequence can be sampled at equal intervals with a sampling step of 30, that is, it can be understood that every 30 CT images in the CT image sequence
  • a CT image is extracted from, and the extracted CT image is used as a sample image.
  • the neural network can be used to label different image areas of the sampled image to determine whether there is an image area with the target image characteristics in the sampled image, for example, to determine whether there is an image area characterizing the hip bone structure (image feature class) in the sampled image .
  • the start and end ranges of the CT image that characterizes the hip bone structure that is, the image sequence section where the target image is located can be quickly located.
  • the start and end ranges of the image sequence interval can also be appropriately increased to ensure that a complete CT image with a characteristic of the hip bone structure can be obtained.
  • the hip bone structure here may include the left femoral head structure, the right femoral head structure and the vertebral structure of the adjacent hip bone.
  • Step S13 segmenting the target image in the target image sequence interval, and determining the image area corresponding to each image feature class in the target image interval.
  • a neural network may be used to divide the target image in the target image sequence region to determine the image region corresponding to each image feature class in the target image in the target image interval.
  • one or more target images in the target image sequence interval can be used as the input of the neural network, and the neural network outputs the image feature class to which the pixel in the target image belongs, and then according to the pixel points corresponding to the multiple image feature classes, it can be determined The image area corresponding to one or more image feature classes in the target image.
  • the image feature class can represent each type of image feature of the target image, and the target image feature of the target image can include multiple types of image features, that is, it can be understood that the target image feature includes multiple sub-image features, and each sub-image feature corresponds to one Image feature class.
  • the target image can be a CT image with pelvic bone features
  • the image feature class can be left hip bone feature class, right hip bone feature class, left femur feature class, right femur feature class, and spine feature included in the pelvic bone feature. Class etc.
  • the CT image can be segmented into the left hip region (the image region formed by the pixels of the left hip bone feature class) and the right hip according to the pixels corresponding to one or more image feature classes.
  • Hip bone area image area formed by pixels of right hip bone characteristics
  • left femur area image area formed by pixels of left femur characteristics
  • right femur area image area formed by pixels of right femur characteristics
  • spine area the image area formed by the pixels of the spine feature
  • the target image in the process of segmenting the target image in the target image sequence interval to determine the image area corresponding to each image feature class in the target image interval, the target image may be based on the target image sequence interval in the target image sequence. And preset relative position information, segment the target image in the target image sequence interval, and determine the image area corresponding to each image feature class in the target image in the target image interval.
  • the preset relative position information can be combined to reduce the error of image region division.
  • the relative position information may indicate that the image area corresponding to an image feature class is located in the approximate orientation of the image, for example, the left hip bone structure is located in the left area of the image, and the right hip bone structure is located in the right area of the image, so according to the relative position relationship, If the obtained image area corresponding to the image feature class of the left hip bone structure is located in the right area of the image, it can be determined that the result is wrong.
  • the target image in the image sequence area may be matched with a preset image interval corresponding to one or more image feature classes in the preset image, and then one or more image features in the target image may be determined according to the matching result.
  • the image area corresponding to the class for example, when the matching result is greater than 75%, it can be considered that the image area of the target image corresponds to the image feature class of the preset image interval.
  • FIG. 4 shows a flowchart of determining the image area corresponding to each image feature class in the target image according to an embodiment of the present disclosure.
  • step S13 may include the following steps:
  • Step S131 In the image processing period, generate input information based on a preset number of consecutive target images and preset relative position information in the target image sequence interval;
  • Step S132 Perform at least one level of convolution processing on the input information to determine the image feature class to which pixels in the target image in the target image interval belong;
  • Step S133 Determine an image area corresponding to at least one image feature class in the target image in the target image interval according to the image feature class to which the pixel in the target image belongs.
  • the aforementioned neural network may be used to determine image regions corresponding to one or more image feature classes in the target image, so as to divide different image regions in the target image.
  • the target image and relative position information included in the image sequence interval can be used as the input of the neural network, so that the input information of the neural network can be generated from the target image and the relative position information, and then the neural network is used to perform at least one level of convolution processing on the input information .
  • the image feature class to which the pixel in the target image belongs can be the output of the neural network.
  • the image processing cycle may correspond to the processing cycle of the neural network for one input and output.
  • the target image input by the neural network may be a continuous preset number of target images, for example, five consecutive target images with a size of 512 *512*1cm 3 of the target image is used as the input of the neural network.
  • the continuity can be understood as the arrangement position of the target image in the image sequence is adjacent. Since the input target image is a continuous target image, compared to only one target image in one image processing cycle, it can not only improve the image processing Efficiency can also take into account the associated information between the target images. For example, the position of the image area corresponding to one image feature class is roughly the same in multiple target images, or the image area corresponding to one image feature class is in multiple target images. The position change in the image is continuous, which can improve the accuracy of target image segmentation.
  • the relative position information may include the relative position information in the x direction and the relative position in the y direction
  • the x map may be used to represent the relative position information in the x direction
  • the y map may be used to represent the relative position information in the y direction.
  • the size of the x image and the y image can be the same as the size of the target image
  • the feature value of the pixel in the x image can represent the relative position of the pixel in the x direction
  • the feature value of the pixel in the y image can represent the pixel
  • the relative position in the y direction so that the relative position information can be used, so that the image feature class determined by the neural network for multiple pixel points has a priori information, for example, if the feature value of a pixel in the x image is -1 , It can mean that the pixel is located on the left side of the target image, and the classification result should be the image feature class corresponding to the left image area.
  • the neural network may be a convolutional neural network, which may include multiple intermediate layers, and the first-level intermediate layer may correspond to the first-level convolution processing.
  • the neural network can be used to determine the image feature class to which the pixels in the target image belong, so that the image area formed by the pixels belonging to one or more image feature classes can be determined, and the different image areas of the target image can be segmented.
  • the convolution processing of the neural network may include down-sampling operations and up-sampling operations, the above-mentioned performing at least one level of convolution processing on the input information to determine the image feature class to which the pixels in the target image belong, It may include: obtaining a feature map of the down-sampling operation input based on the input information; performing a down-sampling operation on the feature map of the down-sampling operation input to obtain the first feature map after the down-sampling operation; and the first feature output based on the down-sampling operation Figure, get the feature map of the input of the upsampling operation; perform the upsampling operation on the feature map of the upsampling operation input to get the second feature map output by the upsampling operation; determine based on the second feature map output by the last-level upsampling operation The image feature class to which pixels in the target image belong.
  • the convolution processing of the aforementioned neural network may include a down-sampling operation and an up-sampling operation
  • the input of the down-sampling operation may be a feature map obtained based on the previous level of convolution processing.
  • the input feature map is processed for the first-level down-sampling operation, and after the down-sampling operation is performed on the feature map, the first feature map obtained after the down-sampling operation of this level can be obtained.
  • the size of the first feature map obtained after different levels of downsampling operations can be different.
  • the first feature map output by the next sampling operation of the last stage in the multi-level down-sampling processing can be used as the input feature map of the up-sampling operation, or the next sampling of the last stage
  • the feature map obtained after the first feature map output by the operation is subjected to convolution processing can be used as the feature map input to the last sampling operation.
  • the input of the up-sampling operation may be a feature map obtained based on the previous level of convolution processing.
  • the input feature map is processed for the first-level up-sampling operation, and after the up-sampling operation is performed on the feature map, the second feature map obtained after the up-sampling operation of this stage can be obtained.
  • the number of stages of the down-sampling operation and the number of stages of the up-sampling operation may be the same, and the neural network may adopt a symmetric structure. Then, according to the second feature map output by the last-level upsampling operation, the image feature class to which the pixels in the target image belong can be obtained. For example, the second feature map output by the last-level upsampling operation can be convolved and normalized. Other processing such as unified processing can obtain the image feature class to which one or more pixels in the target image belong.
  • the first feature map output by the upsampling operation can be subjected to a hole convolution operation to obtain the feature map input by the upsampling operation, so that the upsampling operation input
  • the feature map can include more global information of the target image, which improves the accuracy of obtaining the image feature class to which the pixel belongs.
  • the following uses an example to illustrate the hole convolution operation.
  • the above convolution processing includes a hole convolution operation
  • obtaining the first feature map output based on the downsampling operation to obtain the feature map input from the upsampling operation may include: the first output based on the last downsampling operation A feature map to obtain a feature map of at least one level of hole convolution operation input; perform at least one level of hole convolution operation on the feature map input from at least one level of hole convolution operation to obtain a third feature map after the hole convolution operation; where , The size of the third feature map obtained after the hole convolution operation decreases as the number of convolution processing stages increases; according to the third feature map obtained after the hole convolution operation, the feature map input by the upsampling operation is obtained.
  • the convolution processing of the aforementioned neural network may include a hole convolution operation, and the hole convolution operation may be multi-level.
  • the input of the multi-level spatial convolution operation can be the first feature map output by the last downsampling operation, or it can be the feature map obtained by the first feature map output by the last downsampling operation after at least one level of convolution processing.
  • the input of the first-level convolution operation may be a feature map obtained based on the previous-level convolution processing.
  • the third feature map obtained after the hole convolution operation of this level can be obtained, and the third feature map obtained according to the multi-level hole convolution operation .
  • the hole convolution operation can reduce the loss of information in the input feature map during the convolution process, and increase the area size of the target image mapped by the pixel points in the first feature map, so as to retain as much relevant information as possible to make the final Determine the image area more accurately.
  • multiple third feature maps obtained after at least one level of the hole convolution operation may be feature-fused , Obtain the first fusion feature map; based on the first fusion feature map, obtain the feature map input by the upsampling operation.
  • a third feature map can be obtained after a first-level hole convolution operation, and the size of multiple third feature maps obtained after a multi-level hole convolution operation can be reduced as the number of convolution processing stages increases, that is, The higher the number of convolution processing stages, the smaller the size of the third feature map obtained, so that multiple third feature maps obtained after multi-level hole convolution operations can be considered to have a pyramid structure.
  • the first fusion feature map can include more global information about the target image.
  • the feature map input by the upsampling operation can be obtained.
  • the first fusion feature map is used as the feature map input by the upsampling operation, or the first fusion feature map is convolved.
  • the feature map obtained after processing can be used as the input feature map of the upsampling operation. In this way, the feature map input by the upsampling operation can include more global information of the target image, and the accuracy of the image feature class to which each pixel point belongs can be improved.
  • obtaining the feature map of the input of the up-sampling operation based on the first feature map output by the down-sampling operation may include: In the case that the current up-sampling operation is the first-level up-sampling operation, according to the last-level down-sampling Operate the first feature map output to obtain the feature map of the current upsampling operation input; if the current upsampling operation is greater than or equal to the second level upsampling operation, the second feature map output by the previous upsampling operation is combined with The first feature map matching the same feature map size is fused to obtain a second fusion feature map; based on the second fusion feature map, the feature map input by the current upsampling operation is obtained.
  • the feature map output by the previous-level convolution processing can be used as the feature map input to the first-level up-sampling operation.
  • the first fusion feature map obtained by the first-level hole convolution operation is used as the feature map input by the first-level upsampling operation, or the first fusion feature map is subjected to convolution processing to obtain the feature map input by the first-level upsampling operation.
  • the second feature map output by the up-sampling of the previous stage of the current up-sampling operation, and the second feature map that matches the same feature map size The first feature map is fused to obtain the second fusion feature map, and the feature map of the current upsampling operation input can be obtained based on the second fusion feature map.
  • the second fusion feature map is used as the feature map input by the current upsampling operation, or at least one level of convolution processing is performed on the second fusion feature map to obtain the feature map input by the current upsampling operation. In this way, the feature map input by the current upsampling operation can be combined with the local detailed information and global information of the target image.
  • Fig. 5 shows a block diagram of an example of a neural network structure according to an embodiment of the present disclosure.
  • the network structure of the neural network can adopt the network structure of U network, V network, and full convolutional network. As shown in Figure 5, the network structure of the neural network can be symmetrical.
  • the neural network can perform multi-level convolution processing on the input target image.
  • the convolution processing can include convolution operation, up-sampling operation, down-sampling operation, and hole convolution. Operation, splicing operation, plus connection operation.
  • ASPP may represent a hollow space pyramid pooling module, and the convolution processing of the hollow space pyramid pooling module may include a hole convolution operation and an add connection operation.
  • 5 consecutive target images with a size of 512*512*1 can be used as the input of the neural network, and the relative position information can be combined at the same time, that is, the above x-map and y-map, that is, two input channels can be added, a total of 7 input channels .
  • the size of the feature map obtained from the target image can be reduced to 256*256 to 128*128, and finally to 64* A feature map of 64 pixels.
  • the number of channels is increased from 7 to 256.
  • the obtained feature map passes through the ASPP module, that is, after the connection operation of the cavity convolution and the spatial pyramid structure, as many target images can be retained as possible Related information.
  • the 64*64 size feature map is gradually increased to 512*512, which is the same size as the target image.
  • the feature image of the same size obtained in the down-sampling or pooling operation can be fused with the feature image of the same size obtained in the deconvolution or up-sampling operation, so that the result is
  • the fusion feature map can combine the local detail information and global information of the target image.
  • the image feature class to which each pixel in the target image belongs can be obtained, and the segmentation of different image regions of the target image can be realized.
  • Fig. 6 shows a flowchart of an example of the above neural network training process according to an embodiment of the present disclosure.
  • an explanation is provided for the training process of the neural network after determining the image area corresponding to each image feature class in the target image, and then using the determined classification result of each pixel of the target image to train the neural network.
  • it further includes:
  • Step S21 comparing the image feature class corresponding to the pixel in the target image with the labeled reference image feature class to obtain a comparison result
  • Step S22 Determine the first loss and the second loss in the image processing process according to the comparison result
  • Step S23 based on the first loss and the second loss, adjust the processing parameters used in the image processing process to make the image feature class corresponding to the pixel in the target image the same as the reference image feature class.
  • the target image may be a training sample used for neural network training
  • the image feature class of one or more pixels in the target image can be pre-labeled
  • the pre-labeled image feature class can be a reference image feature class.
  • the image feature class corresponding to one or more pixels of the target image can be compared with the labeled reference image feature class.
  • Use different loss functions to obtain the comparison result for example, use cross-entropy loss function, Deiss loss function, mean square error loss, etc., or you can combine multiple loss functions to obtain a joint loss function.
  • the first loss and second loss in the image processing process can be determined. Combining the determined first loss and second loss, the processing parameters used by the neural network can be adjusted to make the target image
  • the image feature class corresponding to each pixel is the same as the labeled reference image feature class, completing the training process of the neural network.
  • adjusting the processing parameters used in the image processing process based on the first loss and the second loss may include: obtaining a first weight corresponding to the first loss and the second loss The corresponding second weight; based on the first weight and the second weight, the first loss and the second loss are weighted to obtain the target loss; the target loss is used in the image processing process based on the target loss The processing parameters are adjusted.
  • the weight values for the first loss and the second loss can be set separately according to the actual application scenario, for example, the first loss is set to 0.8 for the first loss. Weight, set a second weight of 0.2 for the second loss to get the final target loss. Then, you can use back propagation to update the processing parameters of the neural network based on the target loss, and iteratively optimize the neural network to make the target loss obtained by the neural network converge or reach the maximum number of iterations to obtain the trained neural network.
  • the image processing solution provided by the embodiments of the present disclosure can be applied to segmentation of different bone regions in a CT image sequence, for example, segmentation of different bones of a pelvic structure.
  • the above-mentioned neural network can be used to first determine the upper and lower boundaries of the CT image representing the pelvic region in the CT image sequence, that is, determine the target image of the pelvic CT image Sequence interval. Then, on the basis of the obtained target image sequence interval of the pelvic CT image, segment the pelvic CT image in the target image sequence interval.
  • the target image can be segmented into the left hip region, the right hip region, and the left femur.
  • the image area of the five bones in the area, the right femur area and the spine area can accurately distinguish the five bones included in the pelvic region, which is more conducive to judgment.
  • the location of the pelvic tumor is convenient for planning surgery.
  • rapid pelvic region positioning can be realized (the image processing solution provided by the embodiments of the present disclosure generally takes 30 seconds to segment the pelvic region, and the related segmentation method requires ten minutes or even several hours).
  • the present disclosure also provides image processing devices, electronic equipment, computer-readable storage media, and programs, all of which can be used to implement any image processing method provided in the present disclosure.
  • image processing devices electronic equipment, computer-readable storage media, and programs, all of which can be used to implement any image processing method provided in the present disclosure.
  • the writing order of the steps does not mean a strict execution order but constitutes any limitation on the implementation process.
  • the specific execution order of each step should be based on its function and possibility.
  • the inner logic is determined.
  • Fig. 7 shows a block diagram of an image processing device according to an embodiment of the present disclosure. As shown in Fig. 7, the image processing device includes:
  • the obtaining module 31 is used to obtain the image sequence to be processed
  • the determining module 32 is configured to determine the image sequence interval where the target image is located in the image sequence to be processed, and obtain the target image sequence interval;
  • the segmentation module 33 is configured to segment the target image in the target image sequence interval, and determine an image area corresponding to at least one image feature class in the target image interval.
  • the determining module 32 is specifically configured to determine the sampling step size of the image sequence; according to the sampling step size, the image of the image sequence is acquired to obtain the sampled image; According to the image characteristics of the target image, the sampled image with the target image characteristic is determined; according to the arrangement position of the sampled image with the target image characteristic in the image sequence, the image sequence interval in which the target image is located is determined to obtain the target image sequence interval.
  • the segmentation module 33 is specifically configured to segment the target image in the target image sequence interval based on the target image in the target image sequence interval and preset relative position information, and determine The image area corresponding to at least one image feature class in the target image in the target image interval.
  • the segmentation module 33 is specifically configured to generate, in an image processing cycle, based on a preset number of continuous target images in the target image sequence interval and preset relative position information Input information; perform at least one level of convolution processing on the input information to determine the image feature class to which the pixel in the target image in the target image interval belongs; determine the image feature class to which the pixel in the target image belongs An image area corresponding to at least one image feature class in the target image in the target image interval.
  • the convolution processing includes an up-sampling operation and a down-sampling operation.
  • the segmentation module 33 is specifically configured to obtain a feature map of the down-sampling operation input based on the input information;
  • the input feature map is down-sampled to obtain the first feature map output by the down-sampling operation; based on the first feature map output by the down-sampling operation, the feature map input to the up-sampling operation is obtained; the feature map input by the up-sampling operation is uploaded
  • the sampling operation obtains the second feature map output by the up-sampling operation; based on the second feature map output by the last-level up-sampling operation, the image feature class to which the pixel in the target image belongs is determined.
  • the convolution processing further includes a hole convolution operation;
  • the segmentation module 33 is specifically configured to obtain at least one level of holes based on the first feature map output by the last level of downsampling operation Convolution operation input feature map; at least one level of hole convolution operation is performed on the feature map input of at least one level of hole convolution operation to obtain the third feature map after the hole convolution operation; where, the result is obtained after the hole convolution operation
  • the size of the third feature map decreases as the number of convolution processing stages increases; according to the third feature map obtained after the hole convolution operation, the feature map input by the upsampling operation is obtained.
  • the segmentation module 33 is specifically configured to perform feature fusion on multiple third feature maps obtained after at least one level of hole convolution operation to obtain a first fusion feature map; based on the first fusion Feature map, to get the feature map of the upsampling operation input.
  • the segmentation module 33 is specifically configured to obtain the current up-sampling operation according to the first feature map output by the last-stage down-sampling operation when the current up-sampling operation is the first-stage up-sampling operation.
  • the feature map of the input of the upsampling operation if the current upsampling operation is greater than or equal to the second-level upsampling operation, the second feature map output by the previous upsampling and the first feature matching the same feature map size
  • the graphs are fused to obtain a second fused feature map; based on the second fused feature map, a feature map of the current upsampling operation input is obtained.
  • the device further includes: a training module, configured to compare image feature classes corresponding to pixels in the target image in the target image interval with the labeled reference image feature classes to obtain a comparison The result; the first loss and the second loss in the image processing process are determined according to the comparison result; the processing parameters used in the image processing process are adjusted based on the first loss and the second loss, so that all The image feature class corresponding to the pixel in the target image is the same as the reference image feature class.
  • a training module configured to compare image feature classes corresponding to pixels in the target image in the target image interval with the labeled reference image feature classes to obtain a comparison The result; the first loss and the second loss in the image processing process are determined according to the comparison result; the processing parameters used in the image processing process are adjusted based on the first loss and the second loss, so that all The image feature class corresponding to the pixel in the target image is the same as the reference image feature class.
  • the training module is specifically configured to obtain a first weight corresponding to the first loss and a second weight corresponding to the second loss; based on the first weight and the first weight Two weights, weighting the first loss and the second loss to obtain a target loss; adjust the processing parameters used in the image processing process based on the target loss.
  • the device further includes: a preprocessing module for acquiring an image sequence formed by images acquired at a preset acquisition period; preprocessing the image sequence to obtain an image to be processed sequence.
  • the preprocessing module is specifically configured to perform direction correction on the images of the image sequence according to the direction identification of the images of the image sequence to obtain the image sequence to be processed.
  • the preprocessing module is specifically configured to convert the images of the image sequence into an image of a preset size; perform center cropping on the image of the preset size to obtain the image to be processed sequence.
  • the target image is a pelvic computer tomography CT image
  • the image area includes one of a left hip area, a right hip area, a left femur area, a right femur area, and a spine area Or more.
  • the functions or modules contained in the device provided in the embodiments of the present disclosure can be used to execute the methods described in the above method embodiments.
  • the functions or modules contained in the device provided in the embodiments of the present disclosure can be used to execute the methods described in the above method embodiments.
  • An embodiment of the present disclosure also provides an electronic device, including: a processor; a memory for storing executable instructions of the processor; wherein the processor is configured as the above method.
  • the electronic device can be provided as a terminal, server or other form of device.
  • Fig. 8 is a block diagram showing an electronic device 1900 according to an exemplary embodiment.
  • the electronic device 1900 may be provided as a server.
  • the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922, such as application programs.
  • the application program stored in the memory 1932 may include one or more modules each corresponding to a set of instructions.
  • the processing component 1922 is configured to execute instructions to perform the above-described methods.
  • the electronic device 1900 may also include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input output (I/O) interface 1958 .
  • the electronic device 1900 can operate based on an operating system stored in the memory 1932, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM or the like.
  • a non-volatile or volatile computer-readable storage medium is also provided, such as the memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the electronic device 1900. The above method.
  • An embodiment of the present disclosure also provides a computer program, wherein the computer program includes computer-readable code, and when the computer-readable code runs in an electronic device, the processor in the electronic device executes the above method.
  • the present disclosure may be a system, method, and/or computer program product.
  • the computer program product may include a computer-readable storage medium loaded with computer-readable program instructions for enabling a processor to implement various aspects of the present disclosure.
  • the computer-readable storage medium may be a tangible device that can hold and store instructions used by the instruction execution device.
  • the computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing.
  • Computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) Or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanical encoding device, such as a printer with instructions stored thereon
  • RAM random access memory
  • ROM read-only memory
  • EPROM erasable programmable read-only memory
  • flash memory flash memory
  • SRAM static random access memory
  • CD-ROM compact disk read-only memory
  • DVD digital versatile disk
  • memory stick floppy disk
  • mechanical encoding device such as a printer with instructions stored thereon
  • the computer-readable storage medium used here is not interpreted as a transient signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (for example, light pulses through fiber optic cables), or through wires Transmission of electrical signals.
  • the computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to various computing/processing devices, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and/or a wireless network.
  • the network may include copper transmission cables, optical fiber transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers.
  • the network adapter card or network interface in each computing/processing device receives computer-readable program instructions from the network, and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing/processing device .
  • the computer program instructions used to perform the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, status setting data, or in one or more programming languages.
  • Source code or object code written in any combination, the programming language includes object-oriented programming languages such as Smalltalk, C++, etc., and conventional procedural programming languages such as "C" language or similar programming languages.
  • Computer-readable program instructions can be executed entirely on the user's computer, partly on the user's computer, executed as a stand-alone software package, partly on the user's computer and partly executed on a remote computer, or entirely on the remote computer or server carried out.
  • the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (for example, using an Internet service provider to access the Internet connection).
  • LAN local area network
  • WAN wide area network
  • an electronic circuit such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), can be customized by using the status information of the computer-readable program instructions.
  • the computer-readable program instructions are executed to realize various aspects of the present disclosure.
  • These computer-readable program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine such that when these instructions are executed by the processor of the computer or other programmable data processing device , A device that implements the functions/actions specified in one or more blocks in the flowchart and/or block diagram is produced. It is also possible to store these computer-readable program instructions in a computer-readable storage medium. These instructions make computers, programmable data processing apparatuses, and/or other devices work in a specific manner, so that the computer-readable medium storing instructions includes An article of manufacture, which includes instructions for implementing various aspects of the functions/actions specified in one or more blocks in the flowchart and/or block diagram.
  • each block in the flowchart or block diagram may represent a module, program segment, or part of an instruction, and the module, program segment, or part of an instruction contains one or more functions for implementing the specified logical function.
  • Executable instructions may also occur in a different order from the order marked in the drawings. For example, two consecutive blocks can actually be executed in parallel, or they can sometimes be executed in the reverse order, depending on the functions involved.
  • each block in the block diagram and/or flowchart, and the combination of the blocks in the block diagram and/or flowchart can be implemented by a dedicated hardware-based system that performs the specified functions or actions Or it can be realized by a combination of dedicated hardware and computer instructions.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Multimedia (AREA)
  • Software Systems (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Radiology & Medical Imaging (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Quality & Reliability (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • Biomedical Technology (AREA)
  • Apparatus For Radiation Diagnosis (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

一种图像处理方法及装置、电子设备和存储介质，其中，所述方法包括：获取待处理的图像序列（S11）；确定所述待处理的图像序列中目标图像所在的图像序列区间，得到目标图像序列区间（S12）；对所述目标图像序列区间的目标图像进行分割，确定所述目标图像序列区间中至少一个图像特征类对应的图像区域（S13）。

Description

图像处理方法及装置、电子设备和存储介质
本公开要求在2019年7月29日提交中国专利局、申请号为201910690342.3、申请名称为“图像处理方法及装置、电子设备和存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本公开中。
技术领域
本公开涉及计算机技术领域，尤其涉及一种图像处理方法及装置、电子设备和存储介质。
背景技术
骨骼损伤在意外伤害中具有比较严重的伤害等级，例如，在交通事故等意外事件中从高处的坠落造成的高强度创伤，会导致骨折或者骨裂等骨骼损伤，会引起伤患休克，甚至死亡。医学成像技术对骨骼的诊断和治疗起着十分重要的作用。三维电子计算机断层扫描(Computed Tomography，CT)图像可以用于展示骨骼区域的解剖结构和损伤情况。基于对CT图像的分析，有助于骨骼解剖结构、手术规划以及术后恢复评价等方面。
目前，针对骨骼的CT图像的分析可以包括骨骼区域的分割，这需要在每一个CT图像中进行骨骼区域的手动定位或手动分割。
发明内容
本公开提出了一种图像处理技术方案。
根据本公开的一方面，提供了一种图像处理方法，包括：获取待处理的图像序列；确定所述待处理的图像序列中目标图像所在的图像序列区间，得到目标图像序列区间；对所述目标图像序列区间的目标图像进行分割，确定所述目标图像区间中至少一个图像特征类对应的图像区域。这样，可以实现自动对目标图像中不同图像特征类的图像区域进行分割。
在一种可能的实现方式中，所述确定所述待处理的图像序列中目标图像所在的图像序列区间，得到目标图像序列区间，包括：确定图像序列的采样步长；根据所述采样步长，获取所述图像序列的图像，得到采样图像；根据所述采样图像的图像特征，确定具有目标图像特征的采样图像；根据具有目标图像特征的采样图像在所述图像序列的排列位置，确定所述目标图像所在的图像序列区间，得到目标图像序列区间。这样，可以快速地确定目标图像所在的目标图像序列区间，减少图像处理过程中的工作量，提高图像处理的效率。
在一种可能的实现方式中，所述对所述目标图像序列区间的目标图像进行分割，确定所述目标图像区间中至少一个图像特征类对应的图像区域，包括：基于所述目标图像序列区间的目标图像以及预设的相对位置信息，对所述目标图像序列区间的目标图像进行分割，确定所述目标图像区间的目标图像中至少一个图像特征类对应的图像区域。这样，在对目标图像序列区间的目标图像进行图像区域划分的过程中，可以结合预设的相对位置信息，减少图像区域划分的错误。
在一种可能的实现方式中，所述基于所述目标图像序列区间的目标图像以及预设的相对位置信息，对所述目标图像序列区间的目标图像进行分割，确定所述目标图像区间的目标图像中至少一个图像特征类对应的图像区域，包括：在图像处理周期内，基于所述目标图像序列区间内连续的预设个数 的目标图像以及预设的相对位置信息，生成输入信息；对所述输入信息进行至少一级卷积处理，确定所述目标图像区间的目标图像中每个像素点所属的图像特征类；根据所述目标图像中像素点所属的图像特征类，确定所述目标图像区间的目标图像中至少一个图像特征类对应的图像区域。这样，输入的目标图像为连续的目标图像，不仅可以提高图像处理的效率，还可以考虑到目标图像之间的关联信息。
在一种可能的实现方式中，所述卷积处理包括上采样操作和下采样操作，所述对所述输入信息进行至少一级卷积处理，确定所述目标图像中像素点所属的图像特征类，包括：基于所述输入信息得到下采样操作输入的特征图；对所述下采样操作输入的特征图进行下采样操作，得到所述下采样操作输出的第一特征图；基于所述下采样操作输出的第一特征图，得到所述上采样操作输入的特征图；对所述上采样操作输入的特征图进行上采样操作，得到所述上采样操作输出的第二特征图；基于最后一级上采样操作输出的第二特征图，确定所述目标图像中每个像素点所属的图像特征类。这样，通过上采样操作和下采样操作，可以准确地提取目标图像的图像特征，从而可以得到像素点所属的图像特征类。
在一种可能的实现方式中，所述卷积处理还包括空洞卷积操作；所述基于所述下采样操作输出的第一特征图，得到所述上采样操作输入的特征图，包括：
基于最后一级下采样操作输出的第一特征图，得到至少一级空洞卷积操作输入的特征图；对至少一级空洞卷积操作输入的特征图进行至少一级空洞卷积操作，得到所述空洞卷积操作后的第三特征图；其中，所述空洞卷积操作后得到的第三特征图的尺寸随卷积处理级数的增加而减小；根据所述空洞卷积操作后得到的第三特征图，得到所述上采样操作输入的特征图。这样，可以结合目标图像的局部细节信息以及全局信息，使最终确定图像区域更加准确。
在一种可能的实现方式中，所述根据所述空洞卷积操作后得到的第三特征图，得到所述上采样操作输入的特征图，包括：
将所述至少一级空洞卷积操作后得到的多个第三特征图进行特征融合，得到第一融合特征图；基于所述第一融合特征图，得到所述上采样操作输入的特征图。这样，上采样操作输入的特征图可以包括更多的目标图像的全局信息，提高得到的像素点所属的图像特征类的准确性。
在一种可能的实现方式中，所述基于所述下采样操作输出的第一特征图，得到所述上采样操作输入的特征图，包括：
在当前上采样操作为第一级上采样操作的情况下，根据最后一级下采样操作输出的第一特征图，得到当前上采样操作输入的特征图；在当前上采样操作为大于或等于第二级上采样操作的情况下，将前一级上采样输出的第二特征图与匹配于相同特征图尺寸的第一特征图进行融合，得到第二融合特征图；基于所述第二融合特征图，得到当前上采样操作输入的特征图。这样，当前上采样操作输入的特征图可以结合目标图像的局部细节信息以及全局信息。
在一种可能的实现方式中，所述确定所述目标图像区间中至少一个图像特征类对应的图像区域之后，还包括：将所述目标图像区间的目标图像中像素点对应的图像特征类与标注的参照图像特征类进行比对，得到比对结果；根据所述比对结果确定图像处理过程中的第一损失和第二损失；基于所述第一损失和所述第二损失，对图像处理过程中使用的处理参数进行调整，使所述目标图像中像素点对应 的图像特征类与所述参照图像特征类相同。这样，可以通过多种损失对神经网络使用的处理参数进行调整，使神经网络的训练可以具有更好的效果。
在一种可能的实现方式中，所述基于所述第一损失和所述第二损失，对图像处理过程中使用的处理参数进行调整，包括：获取所述第一损失对应的第一权重以及所述第二损失对应的第二权重；基于所述第一权重和所述第二权重，对所述第一损失和所述第二损失进行加权处理，得到目标损失；基于所述目标损失对图像处理过程中使用的处理参数进行调整。这样，可以根据实际的应用场景为第一损失和第二损失分别设置的权重值，使神将网络的训练具有更好的效果。
在一种可能的实现方式中，所述获取待处理的图像序列之前，还包括：获取以预设的采集周期采集的图像形成的图像序列；对所述图像序列进行预处理，得到待处理的图像序列。这样，可以减少图像序列中的图像的无关信息，对图像中有用的相关信息进行增强。
在一种可能的实现方式中，所述对所述图像序列进行预处理，得到待处理的图像序列，包括：根据所述图像序列的图像的方向标识，对所述图像序列的图像进行方向校正，得到待处理的图像序列。这样，可以根据图像的采集方向对图像序列的图像进行方向校正，使图像的采集方向朝向预设方向。
在一种可能的实现方式中，所述对所述图像序列进行预处理，得到待处理的图像序列，包括：将所述图像序列的图像转换为预设尺寸的图像；对所述预设尺寸的图像进行中心剪裁，得到待处理的图像序列。这样，对预设尺寸的图像进行中心剪裁，将图像中无关的信息删除，保留图像中有用的相关信息。
在一种可能的实现方式中，所述目标图像为骨盆电子计算机断层扫描CT图像，所述图像区域包括左髋骨区域、右髋骨区域、左股骨区域、右股骨区域和脊椎区域中的一个或多个。从而可以实现对CT图像中左髋骨区域、右髋骨区域、左股骨区域、右股骨区域和脊椎区域中的一个或多个不同区域的分割。
根据本公开的一方面，提供了一种图像处理装置，包括：
获取模块，用于获取待处理的图像序列；确定模块，用于确定所述待处理的图像序列中目标图像所在的图像序列区间，得到目标图像序列区间；分割模块，用于对所述目标图像序列区间的目标图像进行分割，确定所述目标图像区间中至少一个图像特征类对应的图像区域。
在一种可能的实现方式中，所述确定模块，具体用于确定图像序列的采样步长；根据所述采样步长，获取所述图像序列的图像，得到采样图像；根据所述采样图像的图像特征，确定具有目标图像特征的采样图像；根据具有目标图像特征的采样图像在所述图像序列的排列位置，确定所述目标图像所在的图像序列区间，得到目标图像序列区间。
在一种可能的实现方式中，所述分割模块，具体用于基于所述目标图像序列区间的目标图像以及预设的相对位置信息，对所述目标图像序列区间的目标图像进行分割，确定所述目标图像区间的目标图像中至少一个图像特征类对应的图像区域。
在一种可能的实现方式中，所述分割模块，具体用于在图像处理周期内，基于所述目标图像序列区间内连续的预设个数的目标图像以及预设的相对位置信息，生成输入信息；对所述输入信息进行至 少一级卷积处理，确定所述目标图像区间的目标图像中像素点所属的图像特征类；根据所述目标图像中像素点所属的图像特征类，确定所述目标图像区间的目标图像中至少一个图像特征类对应的图像区域。
在一种可能的实现方式中，所述卷积处理包括上采样操作和下采样操作，所述分割模块，具体用于基于所述输入信息得到下采样操作输入的特征图；对所述下采样操作输入的特征图进行下采样操作，得到所述下采样操作输出的第一特征图；基于所述下采样操作输出的第一特征图，得到所述上采样操作输入的特征图；对所述上采样操作输入的特征图进行上采样操作，得到所述上采样操作输出的第二特征图；基于最后一级上采样操作输出的第二特征图，确定所述目标图像中像素点所属的图像特征类。
在一种可能的实现方式中，所述卷积处理还包括空洞卷积操作；所述分割模块，具体用于基于最后一级下采样操作输出的第一特征图，得到至少一级空洞卷积操作输入的特征图；对至少一级空洞卷积操作输入的特征图进行至少一级空洞卷积操作，得到所述空洞卷积操作后的第三特征图；其中，所述空洞卷积操作后得到的第三特征图的尺寸随卷积处理级数的增加而减小；根据所述空洞卷积操作后得到的第三特征图，得到所述上采样操作输入的特征图。
在一种可能的实现方式中，分割模块，具体用于将所述至少一级空洞卷积操作后得到的多个第三特征图进行特征融合，得到第一融合特征图；基于所述第一融合特征图，得到所述上采样操作输入的特征图。
在一种可能的实现方式中，所述分割模块，具体用于在当前上采样操作为第一级上采样操作的情况下，根据最后一级下采样操作输出的第一特征图，得到当前上采样操作输入的特征图；在当前上采样操作为大于或等于第二级上采样操作的情况下，将前一级上采样输出的第二特征图与匹配于相同特征图尺寸的第一特征图进行融合，得到第二融合特征图；基于所述第二融合特征图，得到当前上采样操作输入的特征图。
在一种可能的实现方式中，所述装置还包括：训练模块，用于将所述目标图像区间的目标图像中像素点对应的图像特征类与标注的参照图像特征类进行比对，得到比对结果；根据所述比对结果确定图像处理过程中的第一损失和第二损失；基于所述第一损失和所述第二损失，对图像处理过程中使用的处理参数进行调整，使所述目标图像中像素点对应的图像特征类与所述参照图像特征类相同。
在一种可能的实现方式中，所述训练模块，具体用于获取所述第一损失对应的第一权重以及所述第二损失对应的第二权重；基于所述第一权重和所述第二权重，对所述第一损失和所述第二损失进行加权处理，得到目标损失；基于所述目标损失对图像处理过程中使用的处理参数进行调整。
在一种可能的实现方式中，所述装置还包括：预处理模块，用于获取以预设的采集周期采集的图像形成的图像序列；对所述图像序列进行预处理，得到待处理的图像序列。
在一种可能的实现方式中，所述预处理模块，具体用于根据所述图像序列的图像的方向标识，对所述图像序列的图像进行方向校正，得到待处理的图像序列。
在一种可能的实现方式中，所述预处理模块，具体用于将所述图像序列的图像转换为预设尺寸的 图像；对所述预设尺寸的图像进行中心剪裁，得到待处理的图像序列。
在一种可能的实现方式中，所述目标图像为骨盆电子计算机断层扫描CT图像，所述图像区域包括左髋骨区域、右髋骨区域、左股骨区域、右股骨区域和脊椎区域中的一个或多个。
根据本公开的一方面，提供了一种电子设备，包括：
处理器；
用于存储处理器可执行指令的存储器；
其中，所述处理器被配置为：执行上述图像处理方法。
根据本公开的一方面，提供了一种计算机可读存储介质，其上存储有计算机程序指令，所述计算机程序指令被处理器执行时实现上述图像处理方法。
根据本公开的一方面，提供了一种计算机程序，其中，所述计算机程序包括计算机可读代码，当所述计算机可读代码在电子设备中运行时，所述电子设备中的处理器执行用于实现上述图像处理方法。
在本公开实施例中，可以获取待处理的图像序列，然后确定待处理的图像序列中目标图像所在的图像序列区间，得到目标图像序列区间，从而可以针对确定的目标图像序列区间中目标图像进行图像处理，减少图像处理的工作量。然后可以对目标图像序列区间的目标图像进行分割，确定目标图像序列区域中至少一个图像特征类对应的图像区域，这样，可以自动对目标图像中不同图像特征类的图像区域进行分割，例如，对CT图像中的骨骼区域进行分割，节省人力资源。
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，而非限制本公开。
根据下面参考附图对示例性实施例的详细说明，本公开的其它特征及方面将变得清楚。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，这些附图示出了符合本公开的实施例，并与说明书一起用于说明本公开的技术方案。
图1示出根据本公开实施例的图像处理方法的流程图。
图2示出根据本公开实施例的对图像序列进行预处理的流程图。
图3示出根据本公开实施例的确定目标图像序列区间的流程图。
图4示出根据本公开实施例的确定目标图像中每个图像特征类对应的图像区域的流程图。
图5示出根据本公开实施例的神经网络结构示例的框图。
图6示出根据本公开实施例的上述神经网络训练过程示例的流程图。
图7示出根据本公开实施例的图像处理装置的框图。
图8示出根据本公开实施例的电子设备一示例的框图。
具体实施方式
以下将参考附图详细说明本公开的各种示例性实施例、特征和方面。附图中相同的附图标记表示功能相同或相似的元件。尽管在附图中示出了实施例的各种方面，但是除非特别指出，不必按比例绘制附图。
在这里专用的词“示例性”意为“用作例子、实施例或说明性”。这里作为“示例性”所说明的任何实施例不必解释为优于或好于其它实施例。
本文中术语“和/或”，仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本文中术语“至少一种”表示多种中的任意一种或多种中的至少两种的任意组合，例如，包括A、B、C中的至少一种，可以表示包括从A、B和C构成的集合中选择的任意一个或多个元素。
另外，为了更好地说明本公开，在下文的具体实施方式中给出了众多的具体细节。本领域技术人员应当理解，没有某些具体细节，本公开同样可以实施。在一些实例中，对于本领域技术人员熟知的方法、手段、元件和电路未作详细描述，以便于凸显本公开的主旨。
本公开实施例提供的图像处理方案，可以确定获取的图像序列中具有目标图像特征的目标图像所在的目标图像序列区间，从而可以针对目标图像序列区间中的目标图像进行图像处理，而不是针对图像序列中的每个图像进行图像处理，可以减少图像处理过程中的工作量，提高图像处理的效率。然后对确定的目标图像序列区间的目标图像进行分割，确定目标图像中每个图像特征类对应的图像区域。这里，在确定至少一个图像特征类对应的图像区域的过程中，可以利用神经网络对目标图像进行处理，并且可以结合相对位置信息，使在目标图像中确定的每个图像特征类对应的图像区域更加准确，避免分割结果中明显的错误。
本公开实施例提供的图像处理方案，可以应用于对图像分类、图像分割等应用场景，也可以应用于医学领域中的医学影像，例如，对CT图像进行骨盆区域标注。在相关技术中，大多是基于人工对骨盆区域标注，标注过程十分耗时并且容易出现误差。在一些半监督的骨盆区域标注方式中，需要人工选择进行盆骨区域标注的种子点，并对错误的标注进行人工修正，该标注方式同样耗时，标注一张三维的CT图像需要十几分钟。而本公开实施例提供的图像处理方案，可以快速、准确地确定骨盆区域，为患者的诊断提供有效的参考。
下面通过实施例对本公开提供的图像处理方案进行说明。
图1示出根据本公开实施例的图像处理方法的流程图。该图像处理方法可以由终端设备、服务器或其它图像处理装置执行，其中，终端设备可以为用户设备(User Equipment，UE)、移动设备、用户终端、终端、蜂窝电话、无绳电话、个人数字处理(Personal Digital Assistant，PDA)、手持设备、计算设备、车载设备、可穿戴设备等。在一些可能的实现方式中，该图像处理方法可以通过处理器调用存储器中存储的计算机可读指令的方式来实现。下面以图像处理装置作为执行主体为例对本公开实施例的图像处理方法进行说明。
如图1所示，所述图像处理方法包括以下步骤：
步骤S11，获取待处理的图像序列。
在本公开实施例中，图像序列可以包括至少两个图像，图像序列中的每个图像可以按照预设的排列规则进行排序，形成图像序列。预设的排列规则可以包括时间排列规则和/或空间排列规则，例如，可以按照图像的采集时间的先后顺序将多个图像进行排序，形成图像序列，或者，可以按照图像的采 集位置在空间的坐标进行排序，形成图像序列。
举例来说，图像序列可以是利用CT扫描患者得到的一组CT图像，每个CT图像的采集时间不同，可以按照CT图像的采集时间的先后将得到的CT图像形成图像序列，图像图裂中每个CT图像对应的人体部位可以不同。
图2示出根据本公开实施例的对图像序列进行预处理的流程图。
在一种可能的实现方式中，待处理的图像序列可以是经过预处理的图像序列。如图2所示，上述步骤S11之前，还可以包括以下步骤：
S01，获取以预设时间间隔采集的图像形成的图像序列；
S02，对所述图像序列进行预处理，得到待处理的图像序列。
这里，以图像序列中的图像是按照时间排列规则进行排序为例，可以获取以预设的采集周期采集的图像，并按照采集时间的先后顺序由一组图像形成图像序列。针对图像序列中的每个图像，可以对每个图像进行预处理，得到预处理之后的图像序列。其中，预处理可以包括方向校正、去除异常像素值、像素归一化、中心剪裁等操作。经过预处理，可以减少图像序列中的图像的无关信息，对图像中有用的相关信息进行增强。
在一个示例中，在对所述图像序列进行预处理，得到待处理的图像序列时，可以根据所述图像序列的图像的方向标识，对所述图像序列的图像进行方向校正，得到待处理的图像序列。这里，图像序列中的图像在采集时，可以携带有采集的相关信息，例如，可以携带图像的采集时间、图像的采集方向等采集的相关信息，从而可以根据图像的采集方向对图像序列的图像进行方向校正，使图像的采集方向朝向预设方向。例如，可以将CT图像旋转至向预设方向，在CT图像的采集方向表示为坐标轴的情况下，可以使CT图像的坐标轴的x轴或者y轴与预设方向平行，使CT图像可以表征人体的横断面。
在一个示例中，在对所述图像序列进行预处理，得到待处理的图像序列时，可以将所述图像序列的图像转换为预设尺寸的图像，然后对所述预设尺寸的图像进行中心剪裁，得到待处理的图像序列。这里，可以通过图像的重采样或者边缘剪裁将图像序列中的图像的尺寸转变为统一尺寸，然后对预设尺寸的图像进行中心剪裁，将图像中无关的信息删除，保留图像中有用的相关信息。
举例来说，在图像序列是CT图像序列的情况下，在对CT图像序列进行预处理时，可以对CT图像序列中的一个或多个CT图像进行方向校正，确保CT图像表征人体的横断面结构；可以去除CT图像中的异常值，使CT图像中像素点的像素值在[-1024，1024]的区间内，并对CT图像中像素点的像素值归一化处理为[-1，1]；将CT图像重采样到统一尺度，如0.8*0.8*1mm 3；将CT图像进行中心裁剪，例如，进行中心剪裁得到512*512像素点大小的CT图像，不足512*512像素点大小的图像位置可以将像素值设置为预设值，例如，设置为-1。上述几种预处理方式可以进行任意组合。
步骤S12，确定所述待处理的图像序列中目标图像所在的图像序列区间，得到目标图像序列区间。
在本公开实施例中，可以提取图像序列中每个图像的图像特征，或者，提取至少两个图像的图像特征，然后根据提取的图像特征，确定图像序列中具有目标图像特征的目标图像，再确定目标图像在图像序列中的排列位置，由间隔最大的两个目标图像的排列位置，可以得到具有目标图像特征的目标 图像所在的目标图像序列区间。这里，可以利用神经网络对图像序列中的图像进行图像特征提取，并根据提取的图像特征确定具有目标图像特征的目标图像，进一步确定目标图像所在的目标图像序列区间。目标图像序列区间可以是待处理的图像序列的一部分，举例来说，图像序列包括100个图像，其中，目标图像的排列位置在10-20的图像序列区间，则该图像序列区间可以是目标图像序列区间。在一些实施方式中，还可以将图像序列中的图像与预设图像进行匹配，得到匹配结果，然后根据匹配结果确定目标图像所在的图像序列区间，例如，将匹配结果大于70％的图像确定为目标图像，并确定该目标图像所在的图像序列区间，得到目标图像序列区间。
图3示出根据本公开实施例的确定图像序列区间的流程图。
在一种可能的实现方式中，步骤S12可以包括以下步骤：
步骤S121，确定图像序列的采样步长；
步骤S122，根据所述采样步长，获取所述图像序列的图像，得到采样图像；
步骤S123，根据所述采样图像的图像特征，确定具有目标图像特征的采样图像；
步骤S124，根据具有目标图像特征的采样图像在所述图像序列的排列位置，确定所述目标图像所在的图像序列区间，得到目标图像序列区间。
在该种可能的实现方式中，采样步长可以根据实际应用场景进行设定，例如，以30个图像为采样步长，每隔采样步长，可以获取图像序列中的一个图像，并将获取的该图像作为采样图像。针对获取的采样图像，可以利用上述神经网络对采样图像的图像特征进行提取，确定具有目标图像特征的采样图像，然后可以确定具有目标图像特征的采样图像在图像序列中的排列位置，其中，两个具有目标图像特征的采样图像对应的排列位置可以确定一个图像序列区间，可以将得到的多个图像序列区间中最大的图像序列区间作为目标图像所在的目标图像序列区间。由于图像序列为按照预设排列规则进行排列的，从而根据具有目标图像特征的采样图像形成的目标图像序列区间中的图像，也具有目标图像特征，为目标图像。在一些实施方式中，还可以扩大图像序列区间的上下边界，使得最终确定的图像序列区间可以包括所有目标图像。
举例来说，在图像序列为CT图像序列的情况下，可以以采样步长为30对CT图像序列中的CT图像进行等间距采样，即，可以理解为每隔30个CT图像在CT图像序列中抽取一个CT图像，抽取的CT图像作为采样图像。然后可以利用神经网络对采样图像进行不同图像区域的标注，判断采样图像中是否存在具有目标图像特征的图像区域，例如，判断采样图像中是否存在表征具有髋骨结构(图像特征类)的图像区域。通过这种方式，可以快速定位具有表征髋骨结构的CT图像的起始以及截止范围，即，可以快速定位目标图像所在的图像序列区间。一些实施方式中，还可以适当增加图像序列区间的起始以及截止范围，确保可以得到完整的具有表征髋骨结构的CT图像。这里的髋骨结构可以包括相邻髋骨的左股骨头结构、右股骨头结构和椎骨结构。
通过这样方式，在确定图像序列中的目标图像时，可以通过对图像序列的图像进行采样，选取图像序列中的若干个图像进行图像特征提取，确定具有目标图像特征的目标图像所在的目标图像序列区间，减少图像处理过程中的工作量，提高图像处理的效率。
步骤S13，对所述目标图像序列区间的目标图像进行分割，确定所述目标图像区间中每个图像特征类对应的图像区域。
在本公开实施例中，可以利用神经网络对目标图像序列区域内的目标图像进行图像区域划分，确定目标图像区间的目标图像中每个图像特征类对应的图像区域。例如，可以将目标图像序列区间的一个或多个目标图像作为神经网络的输入，由神经网络输出目标图像中像素点所属的图像特征类，然后根据多个图像特征类对应的像素点，可以确定目标图像中一个或多个图像特征类对应的图像区域。这里，图像特征类可以表征目标图像的每类图像特征，目标图像所具有的目标图像特征可以包括多类图像特征，即可以理解为，目标图像特征包括多个子图像特征，每个子图像特征对应一个图像特征类。举例来说，目标图像可以是具有盆骨特征的CT图像，图像特征类可以是盆骨特征包括的左髋骨特征类、右髋骨特征类、左股骨特征类、右股骨特征类、脊椎特征类等。在对目标图像序列区间中具有盆骨特征的CT图像进行不同骨骼区域的分割过程中，可以分别确定CT图像中分别属于左髋骨特征类、右髋骨特征类、左股骨特征类、右股骨特征类、脊椎特征类的像素点，然后根据一个或多个图像特征类对应的像素点，可以将CT图像分割为左髋骨区域(左髋骨特征类的像素点形成的图像区域)、右髋骨区域(右髋骨特征类的像素点形成的图像区域)、左股骨区域(左股骨特征类的像素点形成的图像区域)、右股骨区域(右股骨特征类的像素点形成的图像区域)和脊椎区域(脊椎特征类的像素点形成的图像区域)五个骨骼的图像区域。
在一种可能的实现方式中，在对目标图像序列区间的目标图像进行分割，确定目标图像区间中每个图像特征类对应的图像区域的过程中，可以基于所述目标图像序列区间的目标图像以及预设的相对位置信息，对所述目标图像序列区间的目标图像进行分割，确定所述目标图像区间的目标图像中每个图像特征类对应的图像区域。这里，在对目标图像序列区间的目标图像进行图像区域划分的过程中，可以结合预设的相对位置信息，减少图像区域划分的错误。相对位置信息可以是表征一个图像特征类对应的图像区域位于图像的大致方位，例如，左髋骨结构位于图像的左侧区域，右髋骨结构位于图像的右侧区域，从而根据相对位置关系，如果得到的左髋骨结构的图像特征类对应的图像区域位于图像的右侧区域，可以确定该结果是错误的。在一些实施方式中，可以将图像序列区域内的目标图像与预设图像中一个或多个图像特征类对应的预设图像区间进行匹配，然后根据匹配结果确定目标图像中一个或多个图像特征类对应的图像区域，例如，在匹配结果大于75％的情况下可以认为目标图像的图像区域对应于预设图像区间的图像特征类。
图4示出根据本公开实施例的确定目标图像中每个图像特征类对应的图像区域的流程图。
在一种可能的实现方式中，如图4所示，上述步骤S13可以包括以下步骤：
步骤S131，在图像处理周期内，基于所述目标图像序列区间内连续的预设个数的目标图像以及预设的相对位置信息，生成输入信息；
步骤S132，对所述输入信息进行至少一级卷积处理，确定所述目标图像区间的目标图像中像素点所属的图像特征类；
步骤S133，根据所述目标图像中像素点所属的图像特征类，确定所述目标图像区间的目标图像中 至少一个图像特征类对应的图像区域。
在该种可能的实现方式中，可以利用上述神经网络确定目标图像中一个或多个图像特征类对应的图像区域，从而对目标图像中不同的图像区域进行划分。其中，图像序列区间包括的目标图像以及相对位置信息可以作为神经网络的输入，从而可以由目标图像以及相对位置信息生成神经网络的输入信息，然后利用神经网络对输入信息进行至少一级卷积处理，目标图像中像素点所属的图像特征类可以是神经网络的输出。这里，图像处理周期可以对应神经网络一次输入输出的处理周期，在一个图像处理周期内，神经网络输入的目标图像可以是连续的预设个数的目标图像，例如，将连续5个尺寸为512*512*1cm 3的目标图像的作为神经网络的输入。这里的连续可以理解为目标图像在图像序列中的排列位置相邻，由于输入的目标图像为连续的目标图像，相比于一个图像处理周期仅对一个目标图像进行处理，不仅可以提高图像处理的效率，还可以考虑到目标图像之间的关联信息，例如，一个图像特征类对应的图像区域在多个目标图像的位置是大致一致的，或者，一个图像特征类对应的图像区域在多个目标图像中的位置变化是连续的，从而可以提高目标图像分割的准确性。这里，相对位置信息可以包括x方向的相对位置信息以及y方向的相对位置，可以利用x图表示x方向的相对位置信息，以及，利用y图表示y方向的相对位置信息。其中，x图和y图的尺寸可以与目标图像的尺寸相同，x图中像素点的特征值可以表示该像素点在x方向的相对位置，y图中像素点的特征值可以表示该像素点在y方向的相对位置，从而可以利用相对位置信息，使得神经网络针对多个像素点分类后确定的图像特征类具有一个先验信息，例如，如果一个像素点在x图的特征值为-1，则可以表示该像素点位于目标图像的左侧，得到的分类结果应是对应于左侧图像区域的图像特征类。其中，神经网络可以是卷积神经网络，可以包括多级中间层，一级中间层可以对应一级卷积处理。利用神经网络可以确定目标图像中像素点所属的图像特征类，从而可以确定属于一个或多个图像特征类的像素点形成的图像区域，实现对目标图像的不同图像区域的分割。
在一个示例中，上述神经网络的卷积处理可以包括下采样操作和上采样操作，上述对所述输入信息进行至少一级卷积处理，确定所述目标图像中像素点所属的图像特征类，可以包括：基于所述输入信息得到下采样操作输入的特征图；对下采样操作输入的特征图进行下采样操作，得到下采样操作后的第一特征图；基于下采样操作输出的第一特征图，得到上采样操作输入的特征图；对上采样操作输入的特征图进行上采样操作，得到上采样操作输出的第二特征图；基于最后一级上采样操作输出的第二特征图，确定所述目标图像中像素点所属的图像特征类。
在该示例中，上述神经网络的卷积处理可以包括下采样操作和上采样操作，下采样操作的输入可以是基于上一级卷积处理得到的特征图。针对一级下采样操作处理输入的特征图，对该特征图进行下采样操作之后，可以得到该级下采样操作之后得到的第一特征图。不同级下采样操作后得到的第一特征图的尺寸可以不同。经过对输入信息进行多级下采样操作后，多级下采样处理中最后一级下次采样操作输出的第一特征图，可以作为上采样操作输入的特征图，或者，最后一级下次采样操作输出的第一特征图经过卷积处理后得到的特征图，可以作为上次采样操作输入的特征图。相应地，上采样操作的输入可以是基于上一级卷积处理得到的特征图。针对一级上采样操作处理输入的特征图，对该特征 图进行上采样操作之后，可以得到该级上采样操作之后得到的第二特征图。这里，下采样操作的级数与上采样操作的级数可以相同，该神经网络可以采用对称结构。然后可以根据最后一级上采样操作输出的第二特征图，可以得到目标图像中像素点所属的图像特征类，例如，对最后一级上采样操作输出的第二特征图进行卷积处理、归一化处理等其他处理，可以得到目标图像中一个或多个像素点所属的图像特征类。
为了结合目标图像的局部细节信息以及全局信息，可以在上采样操作之前，将上采样操作输出的第一特征图经过空洞卷积操作，得到上采样操作输入的特征图，从而上采样操作输入的特征图可以包括更多的目标图像的全局信息，提高得到像素点所属的图像特征类的准确性。下面通过一个示例对空洞卷积操作进行说明。
在一个示例中，上述卷积处理包括空洞卷积操作，所述基于下采样操作输出的第一特征图，得到上采样操作输入的特征图可以包括：基于最后一级下采样操作输出的第一特征图，得到至少一级空洞卷积操作输入的特征图；对至少一级空洞卷积操作输入的特征图进行至少一级空洞卷积操作，得到空洞卷积操作后的第三特征图；其中，空洞卷积操作后得到的第三特征图的尺寸随卷积处理级数的增加而减小；根据空洞卷积操作后得到的第三特征图，得到上采样操作输入的特征图。
在该示例中，上述神经网络的卷积处理可以包括空洞卷积操作，空洞卷积操作可以为多级。多级空间卷积操作的输入可以是最后一级下采样操作输出的第一特征图，或者，可以是最后一级下采样操作输出的第一特征图经过至少一级卷积处理得到的特征图。一级空洞卷积操作的输入可以是基于上一级卷积处理得到的特征图。针对一级空洞卷积操作输入的特征图，对该特征图进行空洞处理之后，可以得到该级空洞卷积操作之后得到的第三特征图，根据多级空洞卷积操作得到的第三特征图，可以得到上采样操作输入的特征图。空洞卷积操作可以减少卷积过程中输入的特征图中信息的损失，增加第一特征图中的像素点映射的目标图像的区域大小，从而可以尽可能的保留更多的相关信息，使最终确定图像区域更加准确。
在一个示例中，在根据空洞卷积操作后得到的第三特征图，得到上采样操作输入的特征图时，可以将至少一级空洞卷积操作后得到的多个第三特征图进行特征融合，得到第一融合特征图；基于所述第一融合特征图，得到上采样操作输入的特征图。这里，一级空洞卷积操作后可以得到对一个第三特征图，多级空洞卷积操作后得到的多个第三特征图的尺寸可以随卷积处理级数的增加而减小，即，卷积处理级数越高，得到的第三特征图的尺寸越小，从而经过多级空洞卷积操作后得到的多个第三特征图可以认为具有金字塔结构。由于第三特征图的尺寸不断减小，一些相关信息会存在损失，从而可以将多级空洞卷积操作得到的多个第三特征图进行融合，得到第一融合特征图，第一融合特征图可以包括更多的目标图像的全局信息。然后根据第一融合特征图，可以得到上采样操作输入的特征图，例如，将第一融合特征图作为上采样操作输入的特征图，或者，对第一融合特征图进行卷积处理，卷积处理后得到的特征图可以作为上采样操作输入的特征图。这样，上采样操作输入的特征图可以包括更多的目标图像的全局信息，提高得到的每个像素点所属的图像特征类的准确性。
在一个示例中，上述基于下采样操作输出的第一特征图，得到上采样操作输入的特征图可以包括： 在当前上采样操作为第一级上采样操作的情况下，根据最后一级下采样操作输出的第一特征图，得到当前上采样操作输入的特征图；在当前上采样操作为大于或等于第二级上采样操作的情况下，将前一级上采样输出的第二特征图与匹配于相同特征图尺寸的第一特征图进行融合，得到第二融合特征图；基于所述第二融合特征图，得到当前上采样操作输入的特征图。
在该示例中，在当前上采样操作是第一级上采样操作时，可以将上一级卷积处理输出的特征图作为第一级上采样操作输入的特征图，例如，可以将经过上述多级空洞卷积操作得到的第一融合特征图作为第一级上采样操作输入的特征图，或者，将第一融合特征图经过卷积处理，得到第一级上采样操作输入的特征图。在当前上采样操作是大于或等于第二级上采样操作时，可以将当前上采样操作的前一级上采样输出的第二特征图，以及与该第二特征图匹配于相同特征图尺寸的第一特征图，进行融合，得到第二融合特征图，基于第二融合特征图可以得到当前上采样操作输入的特征图。例如，将第二融合特征图作为当前上采样操作输入的特征图，或者，对第二融合特征图进行至少一级卷积处理，得到当前上采样操作输入的特征图。这样，当前上采样操作输入的特征图可以结合目标图像的局部细节信息以及全局信息。
图5示出根据本公开实施例的神经网络结构示例的框图。
下面结合一示例对上述神经网络的网络结果进行说明。神经网络的网络结构可以采用U网络、V网络、全卷积网络的网络结构。如图5所示，神经网络的网络结构可以对称，该神经网络可以对输入的目标图像进行多级卷积处理，卷积处理可以包括卷积操作、上采样操作、下采样操作、空洞卷积操作、拼接操作、加连接操作。其中，ASPP可以表示空洞空间金字塔池化模块，空洞空间金字塔池化模块的卷积处理可以包括空洞卷积操作以及加连接操作。可以将大小为512*512*1的连续5个目标图像作为神经网络的输入，同时结合相对位置信息，即，上述x图和y图，即，可以增加两个输入通道，一共7个输入通道。通过不同级的卷积操作，3次下采样或池化操作、归一化操作、以及激活操作，可以将由目标图像得到的特征图的尺寸降为256*256至128*128，最后到64*64个像素点的特征图，同时，通道数从7增加到256，得到的特征图经过ASPP模块，即，经过空洞卷积和空间金字塔结构的连接操作，可以尽可能的保留更多的目标图像的相关信息。之后再经过3次解卷积操作或上采样操作，将64*64大小的特征图逐渐升为512*512，与目标图像的尺寸相同。在解卷积操作或上采样操作过程中，可以将下采样或池化操作中得到的相同尺寸的特征图与解卷积操作或上采样操作得到的相同尺寸的特征图像进行融合，这样得到的融合特征图可以结合目标图像的局部细节信息和全局信息。然后再经过三个不同的卷积操作，可以得到目标图像中每个像素点所属的图像特征类，实现目标图像不同图像区域的分割。
图6示出根据本公开实施例的上述神经网络训练过程示例的流程图。
在一个示例中，提供了上述神经网络在确定目标图像中每个图像特征类对应的图像区域之后，利用确定的目标图像的每个像素点的分类结果对神经网络进行训练的训练过程的说明。如图6所示，上述步骤S13之后，还包括：
步骤S21，将所述目标图像中像素点对应的图像特征类与标注的参照图像特征类进行比对，得到 比对结果；
步骤S22，根据所述比对结果确定图像处理过程中的第一损失和第二损失；
步骤S23，基于所述第一损失和所述第二损失，对图像处理过程中使用的处理参数进行调整，使所述目标图像中像素点对应的图像特征类与所述参照图像特征类相同。
这里，目标图像可以是用于神经网络训练的训练样本，可以对目标图像中一个或多个像素点所述的图像特征类预先进行标注，预先标注的图像特征类可以是参照图像特征类。在利用神经网络确定目标图像中至少一个图像特征类对应的图像区域之后，可以将目标图像的一个或多个像素点对应的图像特征类与标注的参照图像特征类进行比对，在对比时可以利用不同的损失函数得到比对结果，例如，使用交叉熵损失函数、戴斯损失函数、均方误差损失等，或者，可以将多个损失函数相结合，得到联合的损失函数。根据不同损失函数得到的比对结果可以确定图像处理过程中的第一损失和第二损失，结合确定的第一损失和第二损失，可以对神经网络使用的处理参数进行调整，使目标图像的每个像素点对应的图像特征类与标注的参照图像特征类相同，完成神经网络的训练过程。
在一个示例中，基于所述第一损失和所述第二损失，对图像处理过程中使用的处理参数进行调整，可以包括：获取所述第一损失对应的第一权重以及所述第二损失对应的第二权重；基于所述第一权重和所述第二权重，对所述第一损失和所述第二损失进行加权处理，得到目标损失；基于所述目标损失对图像处理过程中使用的处理参数进行调整。
这里，在利用第一损失和第二损失调整神经网络的处理参数时，可以根据实际的应用场景为第一损失和第二损失分别设置的权重值，例如，为第一损失设置0.8的第一权重，为第二损失设置0.2的第二权重，得到最终的目标损失。然后可以使用反向传播的方式基于目标损失更新神经网络的处理参数，迭代优化神经网络，使神经网络得到的目标损失收敛或者达到最大的迭代次数，得到训练之后的神经网络。
本公开实施例提供的图像处理方案，可以应用在对CT图像序列中不同骨骼区域的分割中，例如，对骨盆结构的不同骨骼进行分割。可以通过对CT图像序列中CT图像的采样，再结合横断面的相对位置信息，利用上述神经网络先确定表征骨盆区域的CT图像在CT图像序列中的上下边界，即确定骨盆CT图像的目标图像序列区间。然后在得到的骨盆CT图像的目标图像序列区间的基础上，对该目标图像序列区间中的骨盆CT图像进行分割，例如，可以将目标图像分割为左髋骨区域、右髋骨区域、左股骨区域、右股骨区域和脊椎区域五个骨骼的图像区域。相比于目前粗略地分割盆骨区域的方法，即，不区分五个骨骼的分割方法，本公开实施例提供的图像处理方案可以精确地区分盆骨区域包括的五个骨骼，更加有利于判断骨盆肿瘤的位置，方便制定手术规划。同时，可以实现快速的骨盆区域定位(本公开实施例提供的图像处理方案对盆骨区域分割一般需30秒，相关的分割方法需要十几分钟，甚至几个小时)。
可以理解，本公开提及的上述各个方法实施例，在不违背原理逻辑的情况下，均可以彼此相互结合形成结合后的实施例，限于篇幅，本公开不再赘述。
此外，本公开还提供了图像处理装置、电子设备、计算机可读存储介质、程序，上述均可用来实 现本公开提供的任一种图像处理方法，相应技术方案和描述和参见方法部分的相应记载，不再赘述。
本领域技术人员可以理解，在具体实施方式的上述方法中，各步骤的撰写顺序并不意味着严格的执行顺序而对实施过程构成任何限定，各步骤的具体执行顺序应当以其功能和可能的内在逻辑确定。
图7示出根据本公开实施例的图像处理装置的框图，如图7所示，所述图像处理装置包括：
获取模块31，用于获取待处理的图像序列；
确定模块32，用于确定所述待处理的图像序列中目标图像所在的图像序列区间，得到目标图像序列区间；
分割模块33，用于对所述目标图像序列区间的目标图像进行分割，确定所述目标图像区间中至少一个图像特征类对应的图像区域。
在一种可能的实现方式中，所述确定模块32，具体用于确定图像序列的采样步长；根据所述采样步长，获取所述图像序列的图像，得到采样图像；根据所述采样图像的图像特征，确定具有目标图像特征的采样图像；根据具有目标图像特征的采样图像在所述图像序列的排列位置，确定所述目标图像所在的图像序列区间，得到目标图像序列区间。
在一种可能的实现方式中，所述分割模块33，具体用于基于所述目标图像序列区间的目标图像以及预设的相对位置信息，对所述目标图像序列区间的目标图像进行分割，确定所述目标图像区间的目标图像中至少一个图像特征类对应的图像区域。
在一种可能的实现方式中，所述分割模块33，具体用于在图像处理周期内，基于所述目标图像序列区间内连续的预设个数的目标图像以及预设的相对位置信息，生成输入信息；对所述输入信息进行至少一级卷积处理，确定所述目标图像区间的目标图像中像素点所属的图像特征类；根据所述目标图像中像素点所属的图像特征类，确定所述目标图像区间的目标图像中至少一个图像特征类对应的图像区域。
在一种可能的实现方式中，所述卷积处理包括上采样操作和下采样操作，所述分割模块33，具体用于基于所述输入信息得到下采样操作输入的特征图；对下采样操作输入的特征图进行下采样操作，得到下采样操作输出的第一特征图；基于下采样操作输出的第一特征图，得到上采样操作输入的特征图；对上采样操作输入的特征图进行上采样操作，得到上采样操作输出的第二特征图；基于最后一级上采样操作输出的第二特征图，确定所述目标图像中像素点所属的图像特征类。
在一种可能的实现方式中，所述卷积处理还包括空洞卷积操作；所述分割模块33，具体用于，基于最后一级下采样操作输出的第一特征图，得到至少一级空洞卷积操作输入的特征图；对至少一级空洞卷积操作输入的特征图进行至少一级空洞卷积操作，得到空洞卷积操作后的第三特征图；其中，空洞卷积操作后得到的第三特征图的尺寸随卷积处理级数的增加而减小；根据空洞卷积操作后得到的第三特征图，得到上采样操作输入的特征图。
在一种可能的实现方式中，分割模块33，具体用于将至少一级空洞卷积操作后得到的多个第三特征图进行特征融合，得到第一融合特征图；基于所述第一融合特征图，得到上采样操作输入的特征图。
在一种可能的实现方式中，所述分割模块33，具体用于在当前上采样操作为第一级上采样操作的情况下，根据最后一级下采样操作输出的第一特征图，得到当前上采样操作输入的特征图；在当前上采样操作为大于或等于第二级上采样操作的情况下，将前一级上采样输出的第二特征图与匹配于相同特征图尺寸的第一特征图进行融合，得到第二融合特征图；基于所述第二融合特征图，得到当前上采样操作输入的特征图。
在一种可能的实现方式中，所述装置还包括：训练模块，用于将所述目标图像区间的目标图像中像素点对应的图像特征类与标注的参照图像特征类进行比对，得到比对结果；根据所述比对结果确定图像处理过程中的第一损失和第二损失；基于所述第一损失和所述第二损失，对图像处理过程中使用的处理参数进行调整，使所述目标图像中像素点对应的图像特征类与所述参照图像特征类相同。
在一种可能的实现方式中，所述训练模块，具体用于获取所述第一损失对应的第一权重以及所述第二损失对应的第二权重；基于所述第一权重和所述第二权重，对所述第一损失和所述第二损失进行加权处理，得到目标损失；基于所述目标损失对图像处理过程中使用的处理参数进行调整。
在一种可能的实现方式中，所述装置还包括：预处理模块，用于获取以预设的采集周期采集的图像形成的图像序列；对所述图像序列进行预处理，得到待处理的图像序列。
在一种可能的实现方式中，所述预处理模块，具体用于根据所述图像序列的图像的方向标识，对所述图像序列的图像进行方向校正，得到待处理的图像序列。
在一种可能的实现方式中，所述预处理模块，具体用于将所述图像序列的图像转换为预设尺寸的图像；对所述预设尺寸的图像进行中心剪裁，得到待处理的图像序列。
在一种可能的实现方式中，所述目标图像为骨盆电子计算机断层扫描CT图像，所述图像区域包括左髋骨区域、右髋骨区域、左股骨区域、右股骨区域和脊椎区域中的一个或多个。
在一些实施例中，本公开实施例提供的装置具有的功能或包含的模块可以用于执行上文方法实施例描述的方法，其具体实现可以参照上文方法实施例的描述，为了简洁，这里不再赘述。
本公开实施例还提出一种电子设备，包括：处理器；用于存储处理器可执行指令的存储器；其中，所述处理器被配置为上述方法。
电子设备可以被提供为终端、服务器或其它形态的设备。
图8是根据一示例性实施例示出的一种电子设备1900的框图。例如，电子设备1900可以被提供为一服务器。参照图8，电子设备1900包括处理组件1922，其进一步包括一个或多个处理器，以及由存储器1932所代表的存储器资源，用于存储可由处理组件1922的执行的指令，例如应用程序。存储器1932中存储的应用程序可以包括一个或一个以上的每一个对应于一组指令的模块。此外，处理组件1922被配置为执行指令，以执行上述方法。
电子设备1900还可以包括一个电源组件1926被配置为执行电子设备1900的电源管理，一个有线或无线网络接口1950被配置为将电子设备1900连接到网络，和一个输入输出(I/O)接口1958。电子设备1900可以操作基于存储在存储器1932的操作系统，例如Windows ServerTM，Mac OS XTM，UnixTM,LinuxTM，FreeBSDTM或类似。
在示例性实施例中，还提供了一种非易失性或易失性计算机可读存储介质，例如包括计算机程序指令的存储器1932，上述计算机程序指令可由电子设备1900的处理组件1922执行以完成上述方法。
本公开实施例还提出一种计算机程序，其中，所述计算机程序包括计算机可读代码，当所述计算机可读代码在电子设备中运行时，所述电子设备中的处理器执行用于实现上述方法。
本公开可以是系统、方法和/或计算机程序产品。计算机程序产品可以包括计算机可读存储介质，其上载有用于使处理器实现本公开的各个方面的计算机可读程序指令。
计算机可读存储介质可以是可以保持和存储由指令执行设备使用的指令的有形设备。计算机可读存储介质例如可以是――但不限于――电存储设备、磁存储设备、光存储设备、电磁存储设备、半导体存储设备或者上述的任意合适的组合。计算机可读存储介质的更具体的例子(非穷举的列表)包括：便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、静态随机存取存储器(SRAM)、便携式压缩盘只读存储器(CD-ROM)、数字多功能盘(DVD)、记忆棒、软盘、机械编码设备、例如其上存储有指令的打孔卡或凹槽内凸起结构、以及上述的任意合适的组合。这里所使用的计算机可读存储介质不被解释为瞬时信号本身，诸如无线电波或者其他自由传播的电磁波、通过波导或其他传输媒介传播的电磁波(例如，通过光纤电缆的光脉冲)、或者通过电线传输的电信号。
这里所描述的计算机可读程序指令可以从计算机可读存储介质下载到各个计算/处理设备，或者通过网络、例如因特网、局域网、广域网和/或无线网下载到外部计算机或外部存储设备。网络可以包括铜传输电缆、光纤传输、无线传输、路由器、防火墙、交换机、网关计算机和/或边缘服务器。每个计算/处理设备中的网络适配卡或者网络接口从网络接收计算机可读程序指令，并转发该计算机可读程序指令，以供存储在各个计算/处理设备中的计算机可读存储介质中。
用于执行本公开操作的计算机程序指令可以是汇编指令、指令集架构(ISA)指令、机器指令、机器相关指令、微代码、固件指令、状态设置数据、或者以一种或多种编程语言的任意组合编写的源代码或目标代码，所述编程语言包括面向对象的编程语言—诸如Smalltalk、C++等，以及常规的过程式编程语言—诸如“C”语言或类似的编程语言。计算机可读程序指令可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络—包括局域网(LAN)或广域网(WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。在一些实施例中，通过利用计算机可读程序指令的状态信息来个性化定制电子电路，例如可编程逻辑电路、现场可编程门阵列(FPGA)或可编程逻辑阵列(PLA)，该电子电路可以执行计算机可读程序指令，从而实现本公开的各个方面。
这里参照根据本公开实施例的方法、装置(系统)和计算机程序产品的流程图和/或框图描述了本公开的各个方面。应当理解，流程图和/或框图的每个方框以及流程图和/或框图中各方框的组合，都可以由计算机可读程序指令实现。
这些计算机可读程序指令可以提供给通用计算机、专用计算机或其它可编程数据处理装置的处理 器，从而生产出一种机器，使得这些指令在通过计算机或其它可编程数据处理装置的处理器执行时，产生了实现流程图和/或框图中的一个或多个方框中规定的功能/动作的装置。也可以把这些计算机可读程序指令存储在计算机可读存储介质中，这些指令使得计算机、可编程数据处理装置和/或其他设备以特定方式工作，从而，存储有指令的计算机可读介质则包括一个制造品，其包括实现流程图和/或框图中的一个或多个方框中规定的功能/动作的各个方面的指令。
也可以把计算机可读程序指令加载到计算机、其它可编程数据处理装置、或其它设备上，使得在计算机、其它可编程数据处理装置或其它设备上执行一系列操作步骤，以产生计算机实现的过程，从而使得在计算机、其它可编程数据处理装置、或其它设备上执行的指令实现流程图和/或框图中的一个或多个方框中规定的功能/动作。
附图中的流程图和框图显示了根据本公开的多个实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段或指令的一部分，所述模块、程序段或指令的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个连续的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或动作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
以上已经描述了本公开的各实施例，上述说明是示例性的，并非穷尽性的，并且也不限于所披露的各实施例。在不偏离所说明的各实施例的范围和精神的情况下，对于本技术领域的普通技术人员来说许多修改和变更都是显而易见的。本文中所用术语的选择，旨在最好地解释各实施例的原理、实际应用或对市场中技术的技术改进，或者使本技术领域的其它普通技术人员能理解本文披露的各实施例。

Claims (31)

  1. 一种图像处理方法，其特征在于，包括：
    获取待处理的图像序列；
    确定所述待处理的图像序列中目标图像所在的图像序列区间，得到目标图像序列区间；
    对所述目标图像序列区间的目标图像进行分割，确定所述目标图像区间中至少一个图像特征类对应的图像区域。
  2. 根据权利要求1所述的方法，其特征在于，所述确定所述待处理的图像序列中目标图像所在的图像序列区间，得到目标图像序列区间，包括：
    确定图像序列的采样步长；
    根据所述采样步长，获取所述图像序列的图像，得到采样图像；
    根据所述采样图像的图像特征，确定具有目标图像特征的采样图像；
    根据具有目标图像特征的采样图像在所述图像序列的排列位置，确定所述目标图像所在的图像序列区间，得到目标图像序列区间。
  3. 根据权利要求1或2所述的方法，其特征在于，所述对所述目标图像序列区间的目标图像进行分割，确定所述目标图像区间中至少一个图像特征类对应的图像区域，包括：
    基于所述目标图像序列区间的目标图像以及预设的相对位置信息，对所述目标图像序列区间的目标图像进行分割，确定所述目标图像区间的目标图像中至少一个图像特征类对应的图像区域。
  4. 根据权利要求3所述的方法，其特征在于，所述基于所述目标图像序列区间的目标图像以及预设的相对位置信息，对所述目标图像序列区间的目标图像进行分割，确定所述目标图像区间的目标图像中至少一个图像特征类对应的图像区域，包括：
    在图像处理周期内，基于所述目标图像序列区间内连续的预设个数的目标图像以及预设的相对位置信息，生成输入信息；
    对所述输入信息进行至少一级卷积处理，确定所述目标图像区间的目标图像中像素点所属的图像特征类；
    根据所述目标图像中像素点所属的图像特征类，确定所述目标图像区间的目标图像中至少一个图像特征类对应的图像区域。
  5. 根据权利要求4所述的方法，其特征在于，所述卷积处理包括上采样操作和下采样操作，所述对所述输入信息进行至少一级卷积处理，确定所述目标图像中像素点所属的图像特征类，包括：
    基于所述输入信息得到下采样操作输入的特征图；
    对所述下采样操作输入的特征图进行下采样操作，得到所述下采样操作输出的第一特征图；
    基于所述下采样操作输出的第一特征图，得到所述上采样操作输入的特征图；
    对所述上采样操作输入的特征图进行上采样操作，得到所述上采样操作输出的第二特征图；
    基于最后一级所述上采样操作输出的第二特征图，确定所述目标图像中像素点所属的图像特征类。
  6. 根据权利要求5所述的方法，其特征在于，所述卷积处理还包括空洞卷积操作；所述基于所述 下采样操作输出的第一特征图，得到所述上采样操作输入的特征图，包括：
    基于最后一级下采样操作输出的第一特征图，得到至少一级空洞卷积操作输入的特征图；
    对至少一级空洞卷积操作输入的特征图进行至少一级空洞卷积操作，得到所述空洞卷积操作后的第三特征图；其中，所述空洞卷积操作后得到的第三特征图的尺寸随卷积处理级数的增加而减小；
    根据所述空洞卷积操作后得到的第三特征图，得到所述上采样操作输入的特征图。
  7. 根据权利要求6所述的方法，其特征在于，所述根据所述空洞卷积操作后得到的第三特征图，得到所述上采样操作输入的特征图，包括：
    将所述至少一级空洞卷积操作后得到的多个第三特征图进行特征融合，得到第一融合特征图；
    基于所述第一融合特征图，得到所述上采样操作输入的特征图。
  8. 根据权利要求5所述的方法，其特征在于，所述基于所述下采样操作输出的第一特征图，得到所述上采样操作输入的特征图，包括：
    在当前上采样操作为第一级上采样操作的情况下，根据最后一级下采样操作输出的第一特征图，得到当前上采样操作输入的特征图；
    在当前上采样操作为大于或等于第二级上采样操作的情况下，将前一级上采样输出的第二特征图与匹配于相同特征图尺寸的第一特征图进行融合，得到第二融合特征图；
    基于所述第二融合特征图，得到当前上采样操作输入的特征图。
  9. 根据权利要求1至8中任意一项所述的方法，其特征在于，所述确定所述目标图像区间中至少一个图像特征类对应的图像区域之后，还包括：
    将所述目标图像区间的目标图像中像素点对应的图像特征类与标注的参照图像特征类进行比对，得到比对结果；
    根据所述比对结果确定图像处理过程中的第一损失和第二损失；
    基于所述第一损失和所述第二损失，对图像处理过程中使用的处理参数进行调整，使所述目标图像中像素点对应的图像特征类与所述参照图像特征类相同。
  10. 根据权利要求9所述的方法，其特征在于，所述基于所述第一损失和所述第二损失，对图像处理过程中使用的处理参数进行调整，包括：
    获取所述第一损失对应的第一权重以及所述第二损失对应的第二权重；
    基于所述第一权重和所述第二权重，对所述第一损失和所述第二损失进行加权处理，得到目标损失；
    基于所述目标损失对图像处理过程中使用的处理参数进行调整。
  11. 根据权利要求1至10中任意一项所述的方法，其特征在于，所述获取待处理的图像序列之前，还包括：
    获取以预设的采集周期采集的图像形成的图像序列；
    对所述图像序列进行预处理，得到待处理的图像序列。
  12. 根据权利要求11所述的方法，其特征在于，所述对所述图像序列进行预处理，得到待处理的 图像序列，包括：
    根据所述图像序列的图像的方向标识，对所述图像序列的图像进行方向校正，得到待处理的图像序列。
  13. 根据权利要求12所述的方法，其特征在于，所述对所述图像序列进行预处理，得到待处理的图像序列，包括：
    将所述图像序列的图像转换为预设尺寸的图像；
    对所述预设尺寸的图像进行中心剪裁，得到待处理的图像序列。
  14. 根据权利要求1至13中任意一项所述的方法，其特征在于，所述目标图像为骨盆电子计算机断层扫描CT图像，所述图像区域包括左髋骨区域、右髋骨区域、左股骨区域、右股骨区域和脊椎区域中的一个或多个。
  15. 一种图像处理装置，其特征在于，包括：
    获取模块，用于获取待处理的图像序列；
    确定模块，用于确定所述待处理的图像序列中目标图像所在的图像序列区间，得到目标图像序列区间；
    分割模块，用于对所述目标图像序列区间的目标图像进行分割，确定所述目标图像区间中至少一个图像特征类对应的图像区域。
  16. 根据权利要求15所述的装置，其特征在于，所述确定模块，具体用于，
    确定图像序列的采样步长；
    根据所述采样步长，获取所述图像序列的图像，得到采样图像；
    根据所述采样图像的图像特征，确定具有目标图像特征的采样图像；
    根据具有目标图像特征的采样图像在所述图像序列的排列位置，确定所述目标图像所在的图像序列区间，得到目标图像序列区间。
  17. 根据权利要求15或16所述的装置，其特征在于，所述分割模块，具体用于，
    基于所述目标图像序列区间的目标图像以及预设的相对位置信息，对所述目标图像序列区间的目标图像进行分割，确定所述目标图像区间的目标图像中至少一个图像特征类对应的图像区域。
  18. 根据权利要求17所述的装置，其特征在于，所述分割模块，具体用于，
    在图像处理周期内，基于所述目标图像序列区间内连续的预设个数的目标图像以及预设的相对位置信息，生成输入信息；
    对所述输入信息进行至少一级卷积处理，确定所述目标图像区间的目标图像中像素点所属的图像特征类；
    根据所述目标图像中每个像素点所属的图像特征类，确定所述目标图像区间的目标图像中至少一个图像特征类对应的图像区域。
  19. 根据权利要求18所述的装置，其特征在于，所述卷积处理包括上采样操作和下采样操作，所述分割模块，具体用于，
    基于所述输入信息得到下采样操作输入的特征图；
    对所述下采样操作输入的特征图进行下采样操作，得到所述下采样操作输出的第一特征图；
    基于所述下采样操作输出的第一特征图，得到所述上采样操作输入的特征图；
    对所述上采样操作输入的特征图进行上采样操作，得到所述上采样操作输出的第二特征图；
    基于最后一级所述上采样操作输出的第二特征图，确定所述目标图像中像素点所属的图像特征类。
  20. 根据权利要求19所述的装置，其特征在于，所述卷积处理还包括空洞卷积操作；所述分割模块，具体用于，
    基于最后一级下采样操作输出的第一特征图，得到至少一级空洞卷积操作输入的特征图；
    对至少一级空洞卷积操作输入的特征图进行至少一级空洞卷积操作，得到所述空洞卷积操作后的第三特征图；其中，所述空洞卷积操作后得到的第三特征图的尺寸随卷积处理级数的增加而减小；
    根据所述至少一空洞卷积操作后得到的第三特征图，得到所述上采样操作输入的特征图。
  21. 根据权利要求20所述的装置，其特征在于，分割模块，具体用于，
    将各级空洞卷积操作后得到的多个第三特征图进行特征融合，得到第一融合特征图；
    基于所述第一融合特征图，得到所述上采样操作输入的特征图。
  22. 根据权利要求19所述的装置，其特征在于，所述分割模块，具体用于，
    在当前上采样操作为第一级上采样操作的情况下，根据最后一级下采样操作输出的第一特征图，得到当前上采样操作输入的特征图；
    在当前上采样操作为大于或等于第二级上采样操作的情况下，将前一级上采样输出的第二特征图与匹配于相同特征图尺寸的第一特征图进行融合，得到第二融合特征图；
    基于所述第二融合特征图，得到当前上采样操作输入的特征图。
  23. 根据权利要求15至22中任意一项所述的装置，其特征在于，所述装置还包括：
    训练模块，用于将所述目标图像区间的目标图像中像素点对应的图像特征类与标注的参照图像特征类进行比对，得到比对结果；根据所述比对结果确定图像处理过程中的第一损失和第二损失；基于所述第一损失和所述第二损失，对图像处理过程中使用的处理参数进行调整，使所述目标图像中像素点对应的图像特征类与所述参照图像特征类相同。
  24. 根据权利要求23所述的装置，其特征在于，所述训练模块，具体用于，
    获取所述第一损失对应的第一权重以及所述第二损失对应的第二权重；
    基于所述第一权重和所述第二权重，对所述第一损失和所述第二损失进行加权处理，得到目标损失；
    基于所述目标损失对图像处理过程中使用的处理参数进行调整。
  25. 根据权利要求15至24中任意一项所述的装置，其特征在于，所述装置还包括：
    预处理模块，用于获取以预设的采集周期采集的图像形成的图像序列；对所述图像序列进行预处理，得到待处理的图像序列。
  26. 根据权利要求25所述的装置，其特征在于，所述预处理模块，具体用于根据所述图像序列的图像的方向标识，对所述图像序列的图像进行方向校正，得到待处理的图像序列。
  27. 根据权利要求25所述的装置，其特征在于，所述预处理模块，具体用于将所述图像序列的图像转换为预设尺寸的图像；对所述预设尺寸的图像进行中心剪裁，得到待处理的图像序列。
  28. 根据权利要求15至27中任意一项所述的装置，其特征在于，所述目标图像为骨盆电子计算机断层扫描CT图像，所述图像区域包括左髋骨区域、右髋骨区域、左股骨区域、右股骨区域和脊椎区域中的一个或多个。
  29. 一种电子设备，其特征在于，包括：
    处理器；
    用于存储处理器可执行指令的存储器；
    其中，所述处理器被配置为调用所述存储器存储的指令，以执行权利要求1至14中任意一项所述的方法。
  30. 一种计算机可读存储介质，其上存储有计算机程序指令，其特征在于，所述计算机程序指令被处理器执行时实现权利要求1至14中任意一项所述的方法。
  31. 一种计算机程序，其特征在于，所述计算机程序包括计算机可读代码，当所述计算机可读代码在电子设备中运行时，所述电子设备中的处理器执行用于实现权利要求1至14中任意一项所述的方法。
PCT/CN2020/079544 2019-07-29 2020-03-16 图像处理方法及装置、电子设备和存储介质 Ceased WO2021017481A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
JP2021562337A JP2022529493A (ja) 2019-07-29 2020-03-16 画像処理方法及び装置、電子機器並びに記憶媒体
KR1020217031481A KR20210134945A (ko) 2019-07-29 2020-03-16 이미지 처리 방법과 장치, 전자 기기 및 저장 매체
US17/553,997 US20220108452A1 (en) 2019-07-29 2021-12-17 Method and device for image processing, electronic device and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910690342.3 2019-07-29
CN201910690342.3A CN110490878A (zh) 2019-07-29 2019-07-29 图像处理方法及装置、电子设备和存储介质

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/553,997 Continuation US20220108452A1 (en) 2019-07-29 2021-12-17 Method and device for image processing, electronic device and storage medium

Publications (1)

Publication Number Publication Date
WO2021017481A1 true WO2021017481A1 (zh) 2021-02-04

Family

ID=68548433

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/079544 Ceased WO2021017481A1 (zh) 2019-07-29 2020-03-16 图像处理方法及装置、电子设备和存储介质

Country Status (6)

Country Link
US (1) US20220108452A1 (zh)
JP (1) JP2022529493A (zh)
KR (1) KR20210134945A (zh)
CN (1) CN110490878A (zh)
TW (1) TWI755717B (zh)
WO (1) WO2021017481A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113781636A (zh) * 2021-09-14 2021-12-10 杭州柳叶刀机器人有限公司 盆骨建模方法与系统、存储介质、计算机程序产品

Families Citing this family (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110490878A (zh) * 2019-07-29 2019-11-22 上海商汤智能科技有限公司 图像处理方法及装置、电子设备和存储介质
CN111028246A (zh) * 2019-12-09 2020-04-17 北京推想科技有限公司 一种医学图像分割方法、装置、存储介质及电子设备
SG10201913754XA (en) * 2019-12-30 2020-12-30 Sensetime Int Pte Ltd Image processing method and apparatus, electronic device, and storage medium
CN111260666B (zh) * 2020-01-19 2022-05-24 上海商汤临港智能科技有限公司 图像处理方法及装置、电子设备、计算机可读存储介质
CN111294512A (zh) * 2020-02-10 2020-06-16 深圳市铂岩科技有限公司 图像处理方法、装置、存储介质及摄像装置
CN111291817B (zh) * 2020-02-17 2024-01-23 北京迈格威科技有限公司 图像识别方法、装置、电子设备和计算机可读介质
CN113628123B (zh) * 2020-05-09 2026-01-13 深圳市中兴微电子技术有限公司 图像恢复模型的训练方法及装置、电子设备、可读介质
CN111340711B (zh) * 2020-05-21 2020-09-08 腾讯科技（深圳）有限公司 一种超分辨率重建方法、装置、设备和存储介质
CN111639607A (zh) * 2020-06-01 2020-09-08 广州虎牙科技有限公司 模型训练、图像识别方法和装置、电子设备及存储介质
US11488371B2 (en) * 2020-12-17 2022-11-01 Concat Systems, Inc. Machine learning artificial intelligence system for producing 360 virtual representation of an object
JP7062747B1 (ja) * 2020-12-25 2022-05-06 楽天グループ株式会社 情報処理装置、情報処理方法およびプログラム
JP7636278B2 (ja) 2021-06-29 2025-02-26 株式会社日立ハイテク 検査装置及び計測装置
KR20230105233A (ko) 2022-01-03 2023-07-11 삼성전자주식회사 이미지 기반의 이미지 효과를 제공하는 전자 장치 및 그 제어 방법
KR20240065993A (ko) * 2022-11-07 2024-05-14 한국전기연구원 저대조도 영상 융합방법 및 장치
CN115797881B (zh) * 2022-12-26 2026-03-03 江苏大学 一种用于交通道路路面信息的多任务联合感知网络模型及检测方法
CN116543147B (zh) * 2023-03-10 2024-10-11 武汉库柏特科技有限公司 一种颈动脉超声图像分割方法、装置、设备及存储介质
CN118279248B (zh) * 2024-03-18 2024-10-11 中国人民解放军总医院第六医学中心 一种基于图像处理技术的股骨近端中pct的提取方法
CN120356077B (zh) * 2025-03-11 2025-11-25 中国石油大学(北京) 地层裂缝识别方法、装置、设备、介质及程序产品
CN120070458B (zh) * 2025-04-30 2025-08-12 汇耀品尚能源科技(嘉兴)有限公司 一种索结构的振动测量方法、装置、设备及介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110280461A1 (en) * 2009-02-11 2011-11-17 Koninklijke Philips Electronics N.V. Group-wise image registration based on motion model
CN109064447A (zh) * 2018-06-29 2018-12-21 沈阳东软医疗系统有限公司 骨密度展示方法、装置及设备
CN109658401A (zh) * 2018-12-14 2019-04-19 上海商汤智能科技有限公司 图像处理方法及装置、电子设备和存储介质
CN110490878A (zh) * 2019-07-29 2019-11-22 上海商汤智能科技有限公司 图像处理方法及装置、电子设备和存储介质

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10176408B2 (en) * 2015-08-14 2019-01-08 Elucid Bioimaging Inc. Systems and methods for analyzing pathologies utilizing quantitative imaging
CN108603922A (zh) * 2015-11-29 2018-09-28 阿特瑞斯公司 自动心脏体积分割
CN105590324A (zh) * 2016-02-03 2016-05-18 上海联影医疗科技有限公司 医学图像的分割方法及其装置
US9947102B2 (en) * 2016-08-26 2018-04-17 Elekta, Inc. Image segmentation using neural network method
JP6886887B2 (ja) * 2017-08-02 2021-06-16 日本放送協会 誤差計算器およびそのプログラム
CN107665491B (zh) * 2017-10-10 2021-04-09 清华大学 病理图像的识别方法及系统
CN107871119B (zh) * 2017-11-01 2021-07-06 西安电子科技大学 一种基于目标空间知识和两阶段预测学习的目标检测方法
CN107767384B (zh) * 2017-11-03 2021-12-03 电子科技大学 一种基于对抗训练的图像语义分割方法
CN108446729A (zh) * 2018-03-13 2018-08-24 天津工业大学 基于卷积神经网络的鸡蛋胚胎分类方法
CN108647684A (zh) * 2018-05-02 2018-10-12 深圳市唯特视科技有限公司 一种基于引导注意力推理网络的弱监督语义分割方法
US10304193B1 (en) * 2018-08-17 2019-05-28 12 Sigma Technologies Image segmentation and object detection using fully convolutional neural network
CN109934235B (zh) * 2019-03-20 2021-04-20 中南大学 一种无监督的腹部ct序列图像多器官同时自动分割方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110280461A1 (en) * 2009-02-11 2011-11-17 Koninklijke Philips Electronics N.V. Group-wise image registration based on motion model
CN109064447A (zh) * 2018-06-29 2018-12-21 沈阳东软医疗系统有限公司 骨密度展示方法、装置及设备
CN109658401A (zh) * 2018-12-14 2019-04-19 上海商汤智能科技有限公司 图像处理方法及装置、电子设备和存储介质
CN110490878A (zh) * 2019-07-29 2019-11-22 上海商汤智能科技有限公司 图像处理方法及装置、电子设备和存储介质

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113781636A (zh) * 2021-09-14 2021-12-10 杭州柳叶刀机器人有限公司 盆骨建模方法与系统、存储介质、计算机程序产品
CN113781636B (zh) * 2021-09-14 2023-06-20 杭州柳叶刀机器人有限公司 盆骨建模方法与系统、存储介质、计算机程序产品

Also Published As

Publication number Publication date
TWI755717B (zh) 2022-02-21
TW202105322A (zh) 2021-02-01
KR20210134945A (ko) 2021-11-11
US20220108452A1 (en) 2022-04-07
JP2022529493A (ja) 2022-06-22
CN110490878A (zh) 2019-11-22

Similar Documents

Publication Publication Date Title
TWI755717B (zh) 圖像處理方法及裝置、電子設備和計算機可讀儲存介質
KR20210082234A (ko) 이미지 처리 방법 및 장치, 전자 기기 및 기억 매체
CN110956635A (zh) 一种肺段分割方法、装置、设备及存储介质
CN106340021B (zh) 血管提取方法
CN108778134B (zh) 用于根据3d解剖图像表征骨骼的中心轴的系统和方法
CN108074270B (zh) 一种pet衰减校正方法和装置
CN112382359B (zh) 患者注册方法、装置、电子设备及计算机可读介质
WO2018000564A1 (zh) 血管中心线提取方法及系统
KR20220019278A (ko) 이미지 처리를 위한 딥 러닝 모델들
US9317926B2 (en) Automatic spinal canal segmentation using cascaded random walks
CN112308846A (zh) 血管分割方法、装置及电子设备
CN107851312A (zh) 用于在三维解剖图像中自动分割个别骨骼骨的系统及方法
KR20200137768A (ko) 두부경부 컴퓨터단층촬영영상에서의 안와뼈 분할 방법 및 장치
CN114037665B (zh) 下颌神经管的分割方法、装置、电子设备及存储介质
CN110211200B (zh) 一种基于神经网络技术的牙弓线生成方法及其系统
US8131075B2 (en) Fast 4D segmentation of large datasets using graph cuts
CN114418989A (zh) 口腔医学影像的牙齿分割方法、装置、设备及存储介质
CN115439453B (zh) 一种脊椎椎体定位方法、装置、电子设备及存储介质
WO2024011943A1 (zh) 基于深度学习的膝关节髌骨置换三维术前规划方法及系统
CN113962957A (zh) 医学图像处理方法、骨骼图像处理方法、装置、设备
KR20240025489A (ko) 인공지능을 이용한 혈관 구조 결정 방법 및 장치
CN117237322B (zh) 一种基于医学图像的器官分割建模方法及终端
CN107464243B (zh) 一种主动脉瓣的定位方法、装置和设备
CN115546089A (zh) 医学图像分割方法、病理图像处理方法、装置、设备
CN111161240B (zh) 血管分类方法、装置、计算机设备和可读存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20848286

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 20217031481

Country of ref document: KR

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 2021562337

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20848286

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 20848286

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 21.10.2022)

122 Ep: pct application non-entry in european phase

Ref document number: 20848286

Country of ref document: EP

Kind code of ref document: A1