WO2023050690A1 - 图像处理方法、装置、电子设备、存储介质和程序 - Google Patents

图像处理方法、装置、电子设备、存储介质和程序 Download PDF

Info

Publication number
WO2023050690A1
WO2023050690A1 PCT/CN2022/077184 CN2022077184W WO2023050690A1 WO 2023050690 A1 WO2023050690 A1 WO 2023050690A1 CN 2022077184 W CN2022077184 W CN 2022077184W WO 2023050690 A1 WO2023050690 A1 WO 2023050690A1
Authority
WO
WIPO (PCT)
Prior art keywords
rib
image
segmented
segmentation
segmentation result
Prior art date
Application number
PCT/CN2022/077184
Other languages
English (en)
French (fr)
Inventor
吴宇
袁璟
赵亮
Original Assignee
上海商汤智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海商汤智能科技有限公司 filed Critical 上海商汤智能科技有限公司
Publication of WO2023050690A1 publication Critical patent/WO2023050690A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30008—Bone

Definitions

  • the present disclosure relates to the field of computer technology, and in particular to an image processing method, device, electronic equipment, storage medium and program.
  • Ribs are an important part of human bones, which are located in the upper half of the human torso and can effectively protect the organs in the human chest. Chest trauma (for example, rib fracture) caused by accidents such as car accidents often occurs. In the diagnosis of rib fracture, doctors need to give accurate rib identification of the rib where the fracture is located (rib fracture) based on medical images of the chest (for example, chest CT images). Anatomical labels, eg, rib number 3 on the left). In related technologies, a neural network is used to segment ribs to obtain rib segmentation and labeling results. Simultaneously completing the two tasks of segmentation and labeling through a neural network can easily lead to misidentification between adjacent ribs, resulting in low accuracy in rib segmentation and naming.
  • the disclosure proposes a technical solution of an image processing method, device, electronic equipment, storage medium and program.
  • the present disclosure provides an image processing method, including: performing semantic category segmentation on a rib image to be segmented to obtain a semantic category segmentation result corresponding to the rib image to be segmented; performing instance segmentation on the rib image to be segmented, Obtain the instance segmentation result corresponding to the rib image to be segmented; determine the target labeling result corresponding to the rib image to be segmented according to the semantic category segmentation result and the instance segmentation result, wherein the target labeling result includes the A plurality of ribs in the rib image to be segmented and a rib identification corresponding to each rib are described.
  • the present disclosure provides an image processing device, including: a semantic category segmentation module configured to perform semantic category segmentation on a rib image to be segmented to obtain a semantic category segmentation result corresponding to the rib image to be segmented; an instance segmentation module, It is configured to perform instance segmentation on the rib image to be segmented to obtain an instance segmentation result corresponding to the rib image to be segmented; the marking module is configured to determine the segment to be segmented according to the semantic category segmentation result and the instance segmentation result A target marking result corresponding to the rib image, wherein the target marking result includes multiple ribs in the rib image to be segmented and a rib identification corresponding to each rib.
  • the present disclosure provides an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to call the instructions stored in the memory to perform the above method .
  • the present disclosure provides a computer-readable storage medium on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the above method is implemented.
  • the present disclosure provides a computer program, including computer readable code, when the computer readable code is run in an electronic device, a processor in the electronic device executes any one of the above images Approach.
  • the rib image to be segmented is subjected to semantic category segmentation, and the semantic category segmentation result corresponding to the rib image to be segmented is obtained; based on the local geometric information of the image, the rib image to be segmented is instance-segmented, and the rib image to be segmented is obtained.
  • Segment the instance segmentation result corresponding to the rib image comprehensively consider the global semantic information of the image and the local geometric information of the image, therefore, based on the semantic category segmentation result and the instance segmentation result, the determined target labeling result has a high accuracy rate, and the target labeling result includes A plurality of ribs in the rib image to be segmented and a rib identification corresponding to each rib, thereby effectively improving the accuracy of rib segmentation and marking.
  • FIG. 1 shows a flowchart of an image processing method provided by an embodiment of the present disclosure
  • FIG. 2 shows a schematic diagram of z-axis classification of an initial rib image provided by an embodiment of the present disclosure
  • FIG. 3 shows a schematic diagram of a preset z-axis classification network provided by an embodiment of the present disclosure
  • Fig. 4 shows a schematic diagram of the convex hull region of ribs provided by an embodiment of the present disclosure
  • Fig. 5 shows a schematic diagram of a preset rib convex hull segmentation network provided by an embodiment of the present disclosure
  • FIG. 6 shows a schematic diagram of a semantic category segmentation result corresponding to a rib image to be segmented provided by an embodiment of the present disclosure
  • FIG. 7 shows a schematic diagram of a preset semantic category segmentation network provided by an embodiment of the present disclosure
  • FIG. 8 shows a schematic diagram of an example segmentation result corresponding to a rib image to be segmented provided by an embodiment of the present disclosure
  • Fig. 9 shows a schematic diagram of a preset rib instance segmentation network provided by an embodiment of the present disclosure.
  • Fig. 10 shows a schematic diagram of the target marking results corresponding to the rib image to be segmented provided by the embodiment of the present disclosure
  • FIG. 11 shows a schematic diagram of fine binary segmentation results corresponding to rib images to be segmented provided by an embodiment of the present disclosure
  • Fig. 12 shows a schematic diagram of a preset rib binary segmentation network provided by an embodiment of the present disclosure
  • Fig. 13 shows a schematic diagram of the existence of cohesive ribs in the target marking results provided by the embodiments of the present disclosure
  • Fig. 14 shows a schematic diagram of the bonded ribs shown in Fig. 13 provided by an embodiment of the present disclosure after debonding;
  • Fig. 15 shows a block diagram of an image processing device provided by an embodiment of the present disclosure
  • Fig. 16 shows a block diagram of an electronic device provided by an embodiment of the present disclosure
  • Fig. 17 shows a block diagram of an electronic device provided by an embodiment of the present disclosure.
  • the image processing method provided by the embodiments of the present disclosure may be executed by electronic devices such as terminal devices or servers, and the terminal devices may be user equipment (User Equipment, UE), mobile devices, user terminals, cellular phones, cordless phones, personal digital assistants ( Personal Digital Assistant, PDA), handheld devices, computing devices, vehicle-mounted devices, wearable devices, etc.
  • the image processing method can be realized by calling the computer-readable instructions stored in the memory by the processor.
  • the image processing method may be performed by a server.
  • the image processing method provided by the embodiments of the present disclosure can accurately locate the anatomical label of the rib where the fracture is located and the approximate location of the fracture, and provide a visualization result of 3D reconstruction of the rib. It can provide technical support for rib fracture detection, quantitative analysis of rib parameters, rib disease screening, lesion detection and other projects, and realize automatic analysis of rib related parameters, thereby reducing the workload and error rate of doctors, and shortening the diagnosis cycle.
  • FIG. 1 shows a flow chart of an image processing method provided by an embodiment of the present disclosure.
  • the image processing method provided by the embodiment of the present application may include the following steps when executed by an electronic device:
  • Step S11 Semantic category segmentation is performed on the rib image to be segmented to obtain a semantic category segmentation result corresponding to the rib image to be segmented.
  • Step S12 Carry out instance segmentation on the rib image to be segmented, and obtain an instance segmentation result corresponding to the rib image to be segmented.
  • Step S13 According to the semantic category segmentation result and the instance segmentation result, determine the target labeling result corresponding to the rib image to be segmented, wherein the target labeling result includes multiple ribs in the rib image to be segmented and the rib identification corresponding to each rib.
  • the rib image to be segmented is subjected to semantic category segmentation, and the semantic category segmentation result corresponding to the rib image to be segmented is obtained; based on the local geometric information of the image, the rib image to be segmented is instance-segmented, and the rib image to be segmented is obtained.
  • the target labeling result determined based on the semantic category segmentation result and the instance segmentation result has a high accuracy rate
  • the target labeling result includes to-be Segment multiple ribs in the rib image and the rib identification corresponding to each rib, thereby effectively improving the segmentation accuracy and marking accuracy of the ribs.
  • the image processing method provided in the embodiment of the present application further includes the following steps: acquiring the original chest scan image; performing image preprocessing on the original chest scan image to obtain an initial rib image; Packet segmentation to obtain the rib convex hull area in the initial rib image; according to the rib convex hull area, the initial rib image is cropped to obtain the rib image to be segmented.
  • the rib convex hull area in the initial rib image can be determined by performing convex hull segmentation on the initial rib image, so that the rib convex hull area can be determined according to the rib convex hull
  • the bag area is cut from the initial rib image to obtain the rib image to be segmented, so that the subsequent rib marking based on the cropped rib image to be segmented can effectively reduce the consumption of computing resources and improve the efficiency of rib marking.
  • the original chest scan image may be a chest computed tomography (CT) image I.
  • CT images have good bone-soft tissue contrast
  • chest CT images I are usually used as medical images for rib fracture diagnosis.
  • image preprocessing is performed on the chest CT image I to obtain an initial rib image I n .
  • Image preprocessing may include: one or more of reorientation, cropping, normalization, etc., and the present disclosure does not limit the specific manner of image preprocessing.
  • the chest CT image I is pre-processed according to the preset identity matrix to obtain the initial rib image I n , so that the rib direction in the initial rib image I n is consistent with the preset coordinate axis (x/y/z axis) to improve the efficiency of subsequent processing.
  • the preset unit matrix can be set according to the actual situation, and the present disclosure does not limit the specific form of the preset unit matrix.
  • the chest CT image I in addition to the ribs, also includes other large-area background parts.
  • the chest CT image I can be cropped using a preset grayscale threshold .
  • the cropping process of the chest CT image I by using the preset gray threshold may include the following steps: performing binarization on the chest CT image I based on the preset gray threshold to obtain a binary image. According to the bounding box formed by the pixels with a pixel value of 1 in the binary image, the chest CT image I is cropped to obtain the initial rib image I n to reduce the image size, thereby effectively reducing the subsequent computing resource consumption and improving processing efficiency .
  • the specific value of the preset grayscale threshold may be set according to actual conditions, which is not specifically limited in this embodiment of the present disclosure.
  • the grayscale value of the pixel in the chest CT image I is greater than or equal to the preset grayscale threshold, and the corresponding pixel value in the binary image is 1; the grayscale value in the chest CT image I is less than the preset grayscale threshold The corresponding pixel value in the binary image is 0.
  • the preset gray value normalization window is used to perform normalization processing, so that the gray value of the initial rib image I n is within a reasonable gray value range Inside, improve rib marking accuracy.
  • the preset grayscale value normalization window can be set according to the actual situation.
  • the value range of the preset grayscale value normalization window is [-1000, 2000].
  • the actual value range of the value normalization window is not specifically limited.
  • performing convex hull segmentation on the initial rib image to obtain the rib convex hull area in the initial rib image includes: determining whether the rib area in the initial rib image meets the preset integrity requirements; When the rib area in the image meets the preset integrity requirements, the convex hull segmentation is performed on the initial rib image to obtain the rib convex hull area.
  • the quality inspection of the initial rib image I can be performed first, and the rib area in the initial rib image I n can be detected Whether the preset integrity requirements are met.
  • the rib area in the initial rib image I n meets the preset integrity requirements, which may mean that the rib area in the initial rib image is relatively complete and there is no serious missing situation, which is not specifically limited in the embodiments of the present disclosure.
  • the initial rib image includes a plurality of slice images; in the above image processing method, determining whether the rib area in the initial rib image meets the preset integrity requirements may include the following steps: The rib category of each slice image in the image, wherein different rib categories are used to indicate different rib regions; according to the rib category of each slice image, the rib category sequence is determined; when the rib category sequence includes the preset rib category, Determine that the rib region in the initial rib image meets preset integrity requirements.
  • the rib region can be divided into multiple rib regions, and each rib region can correspond to a rib category. Therefore, by determining the rib corresponding to each slice image in the initial rib image I n category, and then determine the rib category sequence corresponding to the initial rib image I n .
  • the rib category sequence includes the preset rib category, it can be determined that most of the rib regions are included in the initial rib image I n , that is, the rib regions in the initial rib image I n are relatively complete and there is no serious missing situation , the rib area in the initial rib image I n meets the preset integrity requirements.
  • the initial rib image I can be resampled to obtain the second rib image, wherein the resolution of the second rib image is the third resolution.
  • a specific value of the third resolution may be determined according to an actual situation, which is not specifically limited in this embodiment of the present disclosure.
  • the third resolution is 1mm ⁇ 1mm ⁇ 3mm, that is, the actual physical size corresponding to each pixel in the second rib image is 1mm ⁇ 1mm ⁇ 3mm.
  • the initial rib image I n is obtained through image preprocessing of the chest CT image I, and the chest CT image I includes multiple slice images, therefore, the initial rib image I n and the initial rib image I n are obtained after resampling
  • the second rib image also includes a plurality of slice images.
  • the rib category of each slice image may be determined based on a preset z-axis classification network.
  • the multiple slice images included in the second rib image are traversed, and each slice image is respectively input into the preset z-axis classification network, so that the rib category of each slice image can be obtained, and different rib categories are used to indicate different rib regions.
  • Fig. 2 shows a schematic diagram of performing z-axis classification on an initial rib image provided by an embodiment of the present disclosure.
  • the complete rib region may include 5 rib categories (category 0, category 1, category 2, category 3 and category 4).
  • the rib category of each slice image can be determined; according to the rib category of the slice image, it can be determined which rib region the slice image is located in.
  • rib category q i of slice image i is category 1
  • rib category qj of slice image j is category 3.
  • the rib category sequence Q (q 1 ,q 2 ,...,q d ), q i ⁇ ⁇ category 0, category 1, category 2, category 3, category 4 ⁇ , i ⁇ [0,d], where d is the total number of sliced images.
  • the rib category sequence Q is processed into a monotonous and ordered sequence of rib categories
  • the preset rib category In the case where the preset rib category is included, it can be determined that the rib region in the initial rib image In meets the preset integrity requirements.
  • the preset rib categories include category 1, category 2 and category 3.
  • the rib categories include category 1, category 2 and category 3, it can be determined that the rib region in the initial rib image I n is relatively complete, and there is no serious missing situation.
  • the preset z-axis classification network may include several convolutional layers, downsampling layers, global average pooling layers, and fully connected layers.
  • Fig. 3 shows a schematic diagram of a preset z-axis classification network provided by an embodiment of the present disclosure.
  • the learning rate setting strategy of warmup and cosine annealing can be used, and the cross-entropy loss function can be used to train the preset z-axis classification network for several training rounds.
  • the training of the preset z-axis classification network can be realized by minimizing the loss function L 1 .
  • the loss function L1 looks like this:
  • y i in the loss function L 1 is the classification label corresponding to the training sample image i
  • the specific network structure and training process of the preset z-axis classification network may adopt other network structures and training methods in related technologies, which are not specifically limited in this embodiment of the present disclosure.
  • the scanning area of the chest CT image I is very large, and the chest CT image I also includes other bones such as vertebrae, hip bones, and femurs, and the rib area only accounts for a small part of it. Therefore, the initial rib image I is determined by the above method When the rib region in meets the preset integrity requirements, the initial rib image In can be segmented by convex hull to obtain the rib convex hull region in the initial rib image In .
  • the initial rib image I n is cropped based on the rib convex hull area, which may include the rib image I v of the rib area to be segmented. Subsequent rib marking based on the rib image I v to be segmented can reduce the waste of computing resources and improve the efficiency of rib marking.
  • the initial rib image I n can be segmented based on a preset rib convex hull segmentation network to obtain the rib convex hull area in the initial rib image I n .
  • the initial rib image I n may be resampled to obtain the first rib image I sp3 , where the resolution of the first rib image I sp3 is the first resolution.
  • the specific value of the first resolution may be determined according to actual conditions, which is not specifically limited in this embodiment of the present disclosure.
  • the first resolution is 3mm ⁇ 3mm ⁇ 3mm, that is, the actual physical size corresponding to each pixel in the first rib image I sp3 is 3mm ⁇ 3mm ⁇ 3mm.
  • FIG. 4 shows a schematic diagram of a rib convex region provided by an embodiment of the present disclosure.
  • the rib convex region includes the rib convex region H l corresponding to the left rib, and the rib convex region H r corresponding to the right rib.
  • a detection frame including the entire area of the rib may be determined.
  • the detection frame may be the smallest rectangular frame including the rib convex hull region H l and the rib convex hull region H r .
  • the initial rib image I n is cropped to obtain the rib image I v to be segmented including the whole area of the rib. Based on the rib image I v to be segmented, subsequent rib marking is performed.
  • the preset rib convex hull segmentation network can be a 3D-U network, including an encoder consisting of several convolutional layers and downsampling layers, and a number of convolutional layers and upsampling layers A decoder composed of layers, a non-local module is embedded between the encoder and the decoder, and a skip connection is introduced in the corresponding stages of the encoder and the decoder.
  • Fig. 5 shows a schematic diagram of a preset rib convex hull segmentation network provided by an embodiment of the present disclosure.
  • the learning rate setting strategy of warmup and cosine annealing can be used, and cross entropy (cross entropy loss) and dice loss can be used as loss functions to train the preset rib convex hull
  • the split network is trained for several training epochs.
  • the training of the preset rib convex hull segmentation network can be realized by minimizing the loss function L 2 .
  • the loss function L2 looks like this:
  • yi in formula (2) is the segmentation label corresponding to the training sample image i, is the segmentation prediction probability corresponding to the training sample image i determined according to the preset rib convex hull segmentation network, Y is the real segmentation result corresponding to the training sample image i, is the prediction segmentation result corresponding to the training sample image i determined according to the preset rib convex hull segmentation network.
  • the specific network structure and training process of the preset rib convex hull segmentation network may adopt other network structures and training methods in related technologies, which are not specifically limited in this embodiment of the present disclosure.
  • a convex hull algorithm may be used to perform convex hull segmentation on the initial rib image I n to obtain rib convex hull regions in the initial rib image I n .
  • the specific algorithm form of the convex hull algorithm can be flexibly set according to the actual situation, which is not specifically limited in the embodiments of the present disclosure.
  • performing semantic category segmentation on the rib image I v to be segmented to obtain the semantic category segmentation result A l corresponding to the rib image I v to be segmented may include the following steps: The rib image I v is resampled to obtain the first rib image I sp3 , wherein the resolution of the first rib image I sp3 is the first resolution; using the preset semantic category segmentation network, the first rib image I sp3 is semantically category segmentation to obtain the semantic category segmentation result A l corresponding to the rib image I v to be segmented.
  • FIG. 6 shows a schematic diagram of a semantic category segmentation result corresponding to a rib image to be segmented provided by an embodiment of the present disclosure.
  • the semantic category segmentation result A l includes multiple ribs in the rib image I v to be segmented and the first predicted rib identification corresponding to each rib.
  • each rib and the first predicted rib identifier corresponding to each rib may be indicated by different colors. For example, red is used to indicate rib 1 on the right, purple is used to indicate rib 2 on the left, and so on.
  • semantic category segmentation result may indicate each rib and the first predicted rib identifier corresponding to each rib, which is not specifically limited in this embodiment of the present disclosure.
  • the network structure and training method of the preset semantic category segmentation network may be the same as the network structure and training method of the preset rib convex hull segmentation network.
  • FIG. 7 shows a schematic diagram of a preset semantic category segmentation network provided by an embodiment of the present disclosure.
  • the specific network structure and training process of the preset semantic category segmentation network may adopt other network structures and training methods in related technologies, which are not specifically limited in this embodiment of the present disclosure.
  • the instance segmentation of the rib image to be segmented to obtain the instance segmentation result corresponding to the rib image to be segmented includes: resampling the rib image to be segmented to obtain the first rib image, wherein the first rib image The resolution of is the first resolution; the pixel points in the first rib image are position-encoded to obtain a position-encoded image; based on the position-encoded image, instance segmentation is performed on the first rib image to obtain an instance segmentation result.
  • segmenting the rib image I v to be segmented at the first resolution can effectively improve the efficiency of instance segmentation, and perform position encoding on the first rib image I sp3 at the first resolution, based on the position encoding image obtained after position encoding , performing instance segmentation on the first rib image I sp3 can effectively improve instance segmentation accuracy.
  • the following formula (3) can be used to perform position encoding on the first rib image I sp3 to obtain the position encoding image I c ,
  • (i, j, k) are the corresponding pixels in the first rib image I sp3 and the position coding image I c
  • ( ⁇ x , ⁇ y , ⁇ z ) are the image center pixels of the first rib image I sp3
  • w x , w y and w z are preset hyperparameters. Specific values of the preset hyperparameters w x , w y and w z may be determined according to actual conditions, and are not specifically limited in this embodiment of the present disclosure.
  • instance segmentation is performed on the first rib image to obtain an instance segmentation result, including: performing binary segmentation on the first rib image to obtain the target rough binary image corresponding to the first rib image value segmentation results; based on the position coded image, determine the pixel embedding vector corresponding to the first rib image; determine the corresponding rib area in the first rib image according to the target rough binary segmentation result and the pixel embedding vector corresponding to the first rib image The pixel embedding vector of the first rib image; the pixel embedding vector corresponding to the rib region in the first rib image is clustered to obtain the instance segmentation result.
  • performing binary segmentation on the first rib image to obtain a target rough binary segmentation result corresponding to the first rib image includes: performing binary segmentation on the first rib image to obtain the first rib image The corresponding initial rough binary segmentation result; according to the rib convex hull area, filter the non-rib area in the initial rough binary segmentation result to obtain the target rough binary segmentation result.
  • instance segmentation may be performed on the first rib image I sp3 based on a preset rib instance segmentation network. Input the first rib image Isp3 and the position-coded image Ic into the preset rib instance segmentation network at the same time.
  • the preset rib instance segmentation network includes two branches, and one branch is used to perform binary segmentation on the first rib image I sp3 to obtain an initial rough binary segmentation result A bc corresponding to the first rib image I sp3 .
  • the initial rough binary segmentation result A bc is obtained by filter the initial rough binary segmentation result A bc to eliminate the false positive part that was mistakenly segmented into the rib area in the initial rough binary segmentation result A bc , and obtain a target with higher precision Coarse binary segmentation result A bc .
  • the target rough binary segmentation result A bc A bc ⁇ (H l ⁇ H r ).
  • another branch of the preset rib instance segmentation network is used to determine the pixel embedding vector A e corresponding to the first rib image I sp3 based on the position-coded image I c .
  • the pixel embedding vector A e includes an embedding vector corresponding to each pixel in the first rib image I sp3 .
  • the dimension of the embedding vector may be 8 dimensions, or may be set to other dimensions according to actual conditions, which is not specifically limited in this embodiment of the present disclosure.
  • clustering algorithm besides the mean-shift clustering algorithm, other clustering algorithms in related technologies may also be used as the clustering algorithm, which is not specifically limited in this embodiment of the present disclosure.
  • Fig. 8 shows a schematic diagram of an example segmentation result corresponding to a rib image to be segmented provided by an embodiment of the present disclosure.
  • the instance segmentation result A ins includes multiple ribs in the rib image I v to be segmented.
  • the instance segmentation result A ins cannot determine the rib identity of each rib.
  • the preset rib instance segmentation network may be a 3D-U network, two encoder branches share a decoder, and a non-local module is embedded between the encoder and decoder.
  • one encoder branch is used to perform binary segmentation on the first rib image I sp3 to obtain the initial rough binary segmentation result A bc corresponding to the first rib image I sp3
  • the other encoder branch is used to encode the image I based on the position c , determining the pixel embedding vector A e corresponding to the first rib image I sp3 .
  • FIG. 9 shows a schematic diagram of a preset rib instance segmentation network provided by an embodiment of the present disclosure.
  • the encoder branch for binary segmentation can use cross entropy (cross entropy loss) and dice loss as the loss function, and the loss function for binary segmentation
  • the split encoder branch is trained for several training epochs.
  • the specific training formula can refer to the above formula (2).
  • the encoder branch for embedding vector prediction when training the preset rib instance segmentation network, can use discriminative loss as the loss function, and the encoder branch for embedding vector prediction using the loss function is trained for several training rounds .
  • the training of the encoder branch for embedding vector prediction can be achieved by minimizing the loss function L d .
  • the loss function L d looks like this:
  • C in the loss function L d is the total number of categories in the training sample image
  • N c is the number of pixels belonging to the same category in the training sample image
  • ⁇ c is the mean vector of the category
  • ⁇ c is the pixel in the training sample image Embedding vector for i.
  • the specific network structure and training process of the preset rib instance segmentation network may adopt other network structures and training methods in related technologies, which are not specifically limited in this embodiment of the present disclosure.
  • the semantic category segmentation result corresponding to the rib image to be segmented is obtained according to the global information of the image, and multiple ribs in the rib image to be segmented and the first predicted rib identifier corresponding to each rib can be obtained.
  • the accuracy of the first predicted rib identification is low due to possible mis-segmentation between adjacent ribs in semantic category segmentation.
  • the instance segmentation result corresponding to the rib image to be segmented is obtained according to the local information of the image. Although the rib identification of each rib cannot be obtained in the instance segmentation result, the instance segmentation result can well separate adjacent ribs. Therefore, considering the results of semantic category segmentation and instance segmentation comprehensively, the rib identification of each rib can be obtained more accurately.
  • determining the target labeling result corresponding to the rib image to be segmented includes: determining the unilateral rib sequence according to the instance segmentation result and the rib convex hull area, wherein, The unilateral rib sequence includes multiple unilateral ribs and their ordering.
  • the unilateral rib sequence includes the left rib sequence and the right rib sequence; according to the semantic category segmentation results and the unilateral rib sequence, determine the multiple ribs in the rib image to be segmented. Ribs and corresponding rib identification for each rib.
  • determining the unilateral rib sequence according to the instance segmentation result and the rib convex area includes: determining the first unilateral rib set according to the instance segmentation result and the rib convex area, wherein the first single The side rib sequence includes N unilateral ribs, N is the number of unilateral ribs; sort the N unilateral ribs in the z-axis direction, and take the K ribs before sorting to form the second unilateral rib set, where K is an integer greater than 1 and less than N; perform plane fitting on each rib in the second unilateral rib set to obtain K fitting planes; according to the positional relationship between the K fitting planes, perform Sort to get the sequence of one-sided ribs.
  • K is a hyperparameter
  • K is an integer greater than 1 and less than N
  • specific value of K may be determined according to actual conditions, which is not specifically limited in this embodiment of the present disclosure.
  • the way of sorting the N unilateral ribs in the first unilateral rib set F1 in the z-axis direction may be based on the 95th percentile of the z-axis, the 90th percentile of the z-axis, and the 85th percentile of the z-axis.
  • the number of digits and the like are not specifically limited in the embodiments of the present disclosure.
  • Plane fitting is performed on each rib in the second unilateral rib set, and a fitting plane corresponding to each rib is determined.
  • the image processing method further includes: determining the plane normal vector and rib centroid corresponding to each rib according to the fitting plane corresponding to each rib in the K ribs; Two ribs R i and R j , according to the plane normal vector and rib centroid corresponding to rib R i , and the plane normal vector and rib centroid corresponding to rib R j , determine the inner product between rib R i and rib R j , where , the size of the inner product between rib R i and rib R j is used to indicate the positional relationship between the fitting plane corresponding to rib R i and the fitting plane corresponding to rib R j .
  • any rib R i ⁇ S l taking the plane where rib R i is located as a reference, determine the inner product of rib R i and the remaining K-1 ribs, and according to the rib R i and the remaining K-1 ribs The inner product of K ribs is sorted to obtain the rib sequence S i corresponding to rib R i . Traverse the K ribs in the second unilateral rib set S1 to obtain K rib sequences. According to the ordering of each rib in the K rib sequences, the final unilateral rib sequence S is obtained by voting.
  • the rank with the most occurrences in the K rib sequences is counted as its final rank in the final unilateral rib sequence S.
  • the corresponding unilateral rib sequence S can also be obtained in the above manner.
  • the semantic category segmentation result and the unilateral rib sequence determine multiple ribs in the rib image to be segmented and the rib identification corresponding to each rib, including: according to the semantic category segmentation result, determine the unilateral The rib identification corresponding to the target rib in the rib sequence, wherein the target rib is any rib in the unilateral rib sequence; according to the rib identification corresponding to the target rib and the unilateral rib sequence, determine the rib image to be segmented.
  • the unilateral rib sequence includes a sequence of ribs. Therefore, based on the semantic category segmentation results, after determining the rib identification corresponding to any target rib in the unilateral rib sequence, the sequence based on the unilateral rib sequence , the rib identification corresponding to each rib in the rib image Iv to be segmented can be obtained.
  • the unilateral rib sequence S (R 1 ,R 2 ,...,R K ), then its corresponding rib identification is (q * ,q * +1,...,q * +k- 1), where, Perform the above processing on the left and right unilateral rib sequences respectively, and obtain the target marking result A c corresponding to the rib image I v to be segmented.
  • FIG. 10 shows a schematic diagram of an object marking result corresponding to a rib image to be segmented provided by an embodiment of the present disclosure.
  • the target marking result Ac includes multiple ribs in the rib image Iv to be segmented and the rib identification corresponding to each rib.
  • different colors may be used to indicate each rib and the rib identification corresponding to each rib. For example, red is used to indicate rib 1 on the right, purple is used to indicate rib 2 on the left, and so on.
  • target marking result Ac may indicate each rib and the rib identification corresponding to each rib, which is not specifically limited in this embodiment of the present disclosure.
  • the target labeling result A c in FIG. 10 is more accurate than the semantic category segmentation result A l in FIG. 6 , and the rib identification of each rib is more accurate.
  • the target labeling result obtained by combining the image global information and the image local information can improve the rib labeling accuracy in scenarios such as rib fracture, severe fracture, and rib misalignment.
  • the image processing method further includes: in the case that the semantic category segmentation and the instance segmentation are performed at the first resolution, performing binary segmentation on the rib image to be segmented at the second resolution, Obtain a fine binary segmentation result corresponding to the rib image to be segmented, wherein the second resolution is greater than the first resolution; update the target marking result according to the fine binary segmentation result.
  • the segmentation result of each rib in the target labeling result is relatively rough, so based on the finer resolution Binary segmentation results, update target labeling results, and obtain target labeling results with higher resolution and higher segmentation accuracy.
  • binary segmentation is performed on the rib image I v to be segmented at the second resolution to obtain a fine binary segmentation result A bf corresponding to the rib image I v to be segmented, including: the rib image I to be segmented I v performs resampling to obtain the third rib image I sp1.5 , wherein the resolution of the third rib image I sp1.5 is the second resolution; using the preset rib binary segmentation network, the third rib image I sp1 .5 Perform binary segmentation to obtain the fine binary segmentation result A bf corresponding to the rib image I v to be segmented.
  • the second resolution is 1.5 mm ⁇ 1.5 mm ⁇ 1.5 mm, that is, the actual physical size corresponding to each pixel in the third rib image I sp1.5 is 1.5 mm ⁇ 1.5 mm ⁇ 1.5 mm.
  • the third rib image I sp1.5 includes a plurality of slice images
  • the preset rib binary segmentation network is used to traverse each slice image in the third rib image I sp1.5 to determine the corresponding
  • the predicted binary segmentation results of all slice images are stacked to obtain the fine binary segmentation result A bf corresponding to the rib image I v to be segmented.
  • FIG. 11 shows a schematic diagram of a fine binary segmentation result corresponding to a rib image to be segmented provided by an embodiment of the present disclosure.
  • the fine binary segmentation result A bf includes multiple ribs in the rib image I v to be segmented.
  • the fine binary segmentation result A bf cannot determine the rib identity of each rib.
  • the preset rib binary segmentation network can be a 2.5D Unet network, including an encoder consisting of several convolutional layers and downsampling layers, and a number of convolutional layers and upsampling layers
  • the decoder is composed of a non-local module embedded between the encoder and the decoder, and a skip connection is introduced in the corresponding stages of the encoder and the decoder.
  • Fig. 12 shows a schematic diagram of a preset rib binary segmentation network provided by an embodiment of the present disclosure.
  • the specific network structure and training process of the preset rib binary segmentation network may adopt other network structures and training methods in related technologies, which are not specifically limited in this embodiment of the present disclosure.
  • updating the target marking result according to the fine binary segmentation result includes: determining multiple connected domains in the fine binary segmentation result; and updating the target marking result according to the multiple connected domains.
  • FIG. 13 shows a schematic diagram of the existence of adhered ribs in the target labeling results provided by an embodiment of the present disclosure. As shown in Figure 13, there may be adhesions between adjacent ribs in the target marking results, resulting in the adhesion ribs including at least two adjacent ribs being indicated by the same color in the target marking results, that is, including at least two adjacent ribs Adhesive ribs of ribs were determined to be the same rib identification, resulting in inaccurate rib labeling.
  • the image processing method further includes: performing adhesion detection on the target marking result based on the preset rib adhesion detection condition, and determining whether there is an adhesion rib in the target marking result, where the adhesion rib is used to indicate the corresponding At least two adjacent ribs identified by the same rib; when it is determined that there are cohesive ribs in the target marking result, determine the pixel embedding vector corresponding to the cohesive rib; cluster the pixel embedding vector corresponding to the cohesive rib to obtain the cohesion Rib identification corresponding to each rib after rib disassembly.
  • the preset rib adhesion detection conditions may include at least the following four conditions:
  • Condition 1 relative volume prior condition; for any rib R i in the target labeling result, if Then it can be determined that the rib R i is a suspected cohesive rib. in, is the average relative volume of the rib R i , V r (R i ) can be obtained through big data statistics,
  • Condition 2 The rib sequence is discontinuous Condition 1; for any two adjacent ribs in the target mark result and If S i +1 ⁇ S i+1 , it can be determined and Suspected adhesion of ribs.
  • T 2 is a hyperparameter, and the specific value of T 2 can be determined according to the actual situation, which is not specifically limited in the embodiments of the present disclosure.
  • Condition 4 category condition; for any rib R i in the target labeling result, if the rib R i corresponds to multiple rib labels in the target labeling result before being updated with the fine binary segmentation result, and the volume of a certain rib label occupies If the ratio is less than 0.8, it can be determined that the rib R i is an impure rib.
  • any rib R i in the target marking result if it satisfies the condition pair d) and any one of the conditions a), b), and c), it can be determined that the rib R i is an adherent rib. Traverse each rib in the target mark result to get the set M of cohesive ribs.
  • the adhesion detection conditions may include other forms of adhesion detection conditions in addition to the above four types, which are not specifically limited in this embodiment of the present disclosure.
  • FIG. 14 shows a schematic diagram of the bonded ribs shown in FIG. 13 provided by an embodiment of the present disclosure after debonding.
  • the bonded ribs indicated by the same color in FIG. 14 are indicated by different colors in FIG. 13 , that is, the debonding of the bonded ribs is realized.
  • Embodiments of the present disclosure also provide image processing devices, electronic equipment, computer-readable storage media, and programs, all of which can be used to implement any image processing method provided by the present disclosure.
  • image processing devices electronic equipment, computer-readable storage media, and programs, all of which can be used to implement any image processing method provided by the present disclosure.
  • Fig. 15 shows a block diagram of an image processing apparatus provided by an embodiment of the present disclosure.
  • the image processing apparatus 1500 provided by the embodiment of the present disclosure may include the following modules:
  • the semantic category segmentation module 1501 is configured to perform semantic category segmentation on the rib image to be segmented, and obtain the semantic category segmentation result corresponding to the rib image to be segmented;
  • the instance segmentation module 1502 is configured to perform instance segmentation on the rib image to be segmented, and obtain an instance segmentation result corresponding to the rib image to be segmented;
  • the marking module 1503 is configured to determine the target marking result corresponding to the rib image to be segmented according to the semantic category segmentation result and the instance segmentation result, wherein the target marking result includes multiple ribs in the rib image to be segmented and the rib corresponding to each rib logo.
  • the image processing device 1500 provided by the embodiment of the present disclosure further includes: a fine binary segmentation module configured to perform semantic category segmentation and instance segmentation at the first resolution, at Binary segmentation is performed on the rib image to be segmented at the second resolution to obtain a fine binary segmentation result corresponding to the rib image to be segmented, wherein the second resolution is greater than the first resolution; the update module is configured to be based on the fine binary value Segmentation results, update target labeling results.
  • a fine binary segmentation module configured to perform semantic category segmentation and instance segmentation at the first resolution, at Binary segmentation is performed on the rib image to be segmented at the second resolution to obtain a fine binary segmentation result corresponding to the rib image to be segmented, wherein the second resolution is greater than the first resolution
  • the update module is configured to be based on the fine binary value Segmentation results, update target labeling results.
  • the image processing device 1500 further includes: an acquisition module configured to acquire an original chest scan image; an image preprocessing module configured to perform image preprocessing on the original chest scan image, Obtain the initial rib image; the convex hull segmentation module is configured to perform convex hull segmentation on the initial rib image to obtain the rib convex hull area in the initial rib image; the cropping module is configured to crop the initial rib image according to the rib convex hull area, Obtain the rib image to be segmented.
  • the convex hull segmentation module includes: a first determination submodule configured to determine whether the rib region in the initial rib image meets the preset integrity requirements; the convex hull segmentation submodule is configured to When the rib area in the rib image meets the preset integrity requirements, the initial rib image is segmented by convex hull to obtain the rib convex hull area.
  • the initial rib image includes multiple slice images;
  • the first determining submodule includes: a first determining unit configured to determine the rib category of each slice image in the multiple slice images, wherein different The rib category is configured to indicate different rib regions;
  • the second determining unit is configured to determine a rib category sequence according to the rib category of each slice image;
  • the third determining unit is configured to include a preset rib category in the rib category sequence , to determine that the rib region in the initial rib image meets the preset integrity requirements.
  • the instance segmentation module 1502 provided by the embodiment of the present disclosure includes: a resampling submodule configured to resample the rib image to be segmented to obtain the first rib image, wherein the first rib image The resolution is the first resolution; the position encoding submodule is configured to perform position encoding on the pixels in the first rib image to obtain a position encoding image; the instance segmentation submodule is configured to encode the image based on the position and encode the first rib image Perform instance segmentation to obtain instance segmentation results.
  • the instance segmentation submodule includes: a binary segmentation unit configured to perform binary segmentation on the first rib image to obtain the target rough binary value corresponding to the first rib image Segmentation result; the fourth determination unit is configured to determine the pixel point embedding vector corresponding to the first rib image based on the position coded image; the fifth determination unit is configured to determine the pixel point corresponding to the first rib image according to the target rough binary segmentation result
  • the embedding vector is used to determine the pixel point embedding vector corresponding to the rib region in the first rib image;
  • the clustering unit is configured to perform clustering on the pixel point embedding vector corresponding to the rib region in the first rib image to obtain an instance segmentation result.
  • the binary segmentation unit provided by the embodiment of the present disclosure is specifically configured to: perform binary segmentation on the first rib image to obtain an initial rough binary segmentation result corresponding to the first rib image;
  • the convex hull area filters the non-rib area in the initial rough binary segmentation result to obtain the target rough binary segmentation result.
  • the labeling module 1503 provided by the embodiment of the present disclosure includes: a second determination submodule configured to determine a unilateral rib sequence according to the instance segmentation result and the rib convex area, wherein the unilateral rib The sequence includes multiple unilateral ribs and their ordering.
  • the unilateral rib sequence includes the left rib sequence and the right rib sequence;
  • the third determining submodule is configured to determine the rib to be segmented according to the semantic category segmentation result and the unilateral rib sequence The multiple ribs in the image and the corresponding rib ID for each rib.
  • the second determining submodule provided by the embodiment of the present disclosure includes: a sixth determining unit configured to determine the first set of unilateral ribs according to the instance segmentation result and the rib convex region, wherein, The first unilateral rib sequence includes N unilateral ribs, N is the number of unilateral ribs; the first sorting unit is configured to sort N unilateral ribs in the z-axis direction, and takes the K ribs before sorting to form The second unilateral rib set, wherein K is an integer greater than 1 and less than N; the plane fitting unit is configured to perform plane fitting on each rib in the second unilateral rib set to obtain K fitting planes; The second sorting unit is configured to sort the K ribs according to the positional relationship between the K fitting planes to obtain a single rib sequence.
  • the image processing device 1500 provided by the embodiment of the present disclosure further includes: a first determination module configured to determine the plane corresponding to each rib according to the fitting plane corresponding to each rib in the K ribs Normal vector and rib centroid; the second determination module is configured for any two ribs R i and R j in the K ribs, according to the plane normal vector and rib centroid corresponding to rib R i , and the plane method corresponding to rib R j vector and rib centroid to determine the inner product between rib R i and rib R j , where the size of the inner product between rib R i and said rib R j is configured to indicate the fitting plane corresponding to rib R i and The positional relationship between the fitting planes corresponding to rib Rj .
  • a first determination module configured to determine the plane corresponding to each rib according to the fitting plane corresponding to each rib in the K ribs Normal vector and rib centroid
  • the second determination module is configured for any two ribs R i
  • the third determination submodule provided by the embodiment of the present disclosure is specifically configured to: determine the rib identification corresponding to the target rib in the unilateral rib sequence according to the semantic category segmentation result, where the target rib is Any rib in the unilateral rib sequence; according to the rib identification corresponding to the target rib and the unilateral rib sequence, multiple ribs in the rib image to be segmented and the rib identification corresponding to each rib are determined.
  • the updating module provided by the embodiment of the present disclosure is specifically configured to: determine multiple connected domains in the fine binary segmentation result; and update the target labeling result according to the multiple connected domains.
  • the image processing device 1500 provided in the embodiment of the present disclosure further includes: a third determining module, configured to perform adhesion detection on the target marking result based on preset rib adhesion detection conditions, and determine Whether there are cohesive ribs, wherein the cohesive ribs are configured to indicate at least two adjacent ribs corresponding to the same rib identification; the fourth determination module is configured to determine that the cohesive ribs correspond to The pixel point embedding vector; the clustering module configured to cluster the pixel point embedding vectors corresponding to the bonded ribs to obtain the rib identification corresponding to each rib after the bonded ribs are debonded.
  • a third determining module configured to perform adhesion detection on the target marking result based on preset rib adhesion detection conditions, and determine Whether there are cohesive ribs, wherein the cohesive ribs are configured to indicate at least two adjacent ribs corresponding to the same rib identification
  • the fourth determination module is configured to determine that the cohesive ribs correspond to The pixel point embedding
  • the functions or modules included in the apparatus provided by the embodiments of the present disclosure may be used to execute the methods described in the above method embodiments, and for specific implementation, refer to the descriptions of the above method embodiments.
  • Embodiments of the present disclosure also provide a computer-readable storage medium, on which computer program instructions are stored, and the above-mentioned method is implemented when the computer program instructions are executed by a processor.
  • the computer readable storage medium may be a non-transitory computer readable storage medium.
  • An embodiment of the present disclosure also proposes an electronic device, including: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to invoke the instructions stored in the memory to execute the above method.
  • An embodiment of the present disclosure also provides a computer program product, including computer-readable codes.
  • the processor in the device executes the method for implementing the image processing method provided in any one of the above embodiments. instruction.
  • the embodiments of the present disclosure also provide another computer program product, which is used for storing computer-readable instructions, and when the instructions are executed, the computer executes the operations of the image processing method provided by any of the above-mentioned embodiments.
  • Electronic devices may be provided as terminals, servers, or other forms of devices.
  • Fig. 16 shows a block diagram of an electronic device provided by an embodiment of the present disclosure.
  • the electronic device 1600 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant and other terminals.
  • electronic device 1600 may include one or more of the following components: processing component 1602, memory 1604, power supply component 1606, multimedia component 1608, audio component 1610, input/output (I/O) interface 1612, sensor component 1614 , and the communication component 1616.
  • the processing component 1602 generally controls the overall operations of the electronic device 1600, such as those associated with display, telephone calls, data communications, camera operations, and recording operations.
  • the processing component 1602 may include one or more processors 1620 to execute instructions to complete all or part of the steps of the above method. Additionally, processing component 1602 may include one or more modules that facilitate interaction between processing component 1602 and other components. For example, processing component 1602 may include a multimedia module to facilitate interaction between multimedia component 1608 and processing component 1602 .
  • the memory 1604 is configured to store various types of data to support operations at the electronic device 1600 . Examples of such data include instructions for any application or method operating on the electronic device 1600, contact data, phonebook data, messages, pictures, videos, and the like.
  • the memory 1604 can be realized by any type of volatile or non-volatile memory device or their combination, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), Magnetic Memory, Flash Memory, Magnetic or Optical Disk.
  • SRAM static random access memory
  • EEPROM electrically erasable programmable read-only memory
  • EPROM erasable Programmable Read Only Memory
  • PROM Programmable Read Only Memory
  • ROM Read Only Memory
  • Magnetic Memory Flash Memory
  • Magnetic or Optical Disk Magnetic Disk
  • the power supply component 1606 provides power to various components of the electronic device 1600 .
  • Power components 1606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for electronic device 1600 .
  • the multimedia component 1608 includes a screen providing an output interface between the electronic device 1600 and the user.
  • the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user.
  • the touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may not only sense a boundary of a touch or swipe action, but also detect a duration and pressure associated with the touch or swipe operation.
  • the multimedia component 1608 includes a front camera and/or a rear camera. When the electronic device 1600 is in an operation mode, such as a shooting mode or a video mode, the front camera and/or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have focal length and optical zoom capability.
  • the audio component 1610 is configured to output and/or input audio signals.
  • the audio component 1610 includes a microphone (MIC), which is configured to receive an external audio signal when the electronic device 1600 is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode. Received audio signals may be further stored in memory 1604 or sent via communication component 1616 .
  • the audio component 1610 also includes a speaker for outputting audio signals.
  • the I/O interface 1612 provides an interface between the processing component 1602 and a peripheral interface module.
  • the peripheral interface module may be a keyboard, a click wheel, a button, and the like. These buttons may include, but are not limited to: a home button, volume buttons, start button, and lock button.
  • Sensor assembly 1614 includes one or more sensors for providing various aspects of status assessment for electronic device 1600 .
  • the sensor component 1614 can detect the open/closed state of the electronic device 1600, the relative positioning of components, such as the display and the keypad of the electronic device 1600, and the sensor component 1614 can also detect the electronic device 1600 or one of the electronic devices 1600 Changes in the position of components, presence or absence of user contact with electronic device 1600 , electronic device 1600 orientation or acceleration/deceleration and temperature changes in electronic device 1600 .
  • Sensor assembly 1614 may include a proximity sensor configured to detect the presence of nearby objects in the absence of any physical contact.
  • the sensor assembly 1614 may also include an optical sensor, such as a complementary metal-oxide-semiconductor (CMOS) or charge-coupled device (CCD) image sensor, for use in imaging applications.
  • CMOS complementary metal-oxide-semiconductor
  • CCD charge-coupled device
  • the sensor component 1614 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.
  • the communication component 1616 is configured to facilitate wired or wireless communication between the electronic device 1600 and other devices.
  • the electronic device 1600 can access a wireless network based on a communication standard, such as a wireless network (WiFi), a second generation mobile communication technology (2G) or a third generation mobile communication technology (3G), or a combination thereof.
  • the communication component 1616 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel.
  • the communication component 1616 also includes a near field communication (NFC) module to facilitate short-range communication.
  • the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) technology and other technologies.
  • RFID Radio Frequency Identification
  • IrDA Infrared Data Association
  • UWB Ultra Wide Band
  • Bluetooth Bluetooth
  • the electronic device 1600 can be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gates Arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic implementations for performing the methods described above.
  • ASICs Application Specific Integrated Circuits
  • DSPs Digital Signal Processors
  • DSPDs Digital Signal Processing Devices
  • PLDs Programmable Logic Devices
  • FPGAs Field Programmable Gates Arrays
  • controllers microcontrollers, microprocessors or other electronic implementations for performing the methods described above.
  • An embodiment of the present disclosure provides a non-volatile computer-readable storage medium, such as a memory 1604 including computer program instructions, which can be executed by the processor 1620 of the electronic device 1600 to complete the above method.
  • a non-volatile computer-readable storage medium such as a memory 1604 including computer program instructions, which can be executed by the processor 1620 of the electronic device 1600 to complete the above method.
  • An embodiment of the present disclosure provides a computer program product, including computer-readable codes, or a non-volatile computer-readable storage medium bearing computer-readable codes, when the computer-readable codes are stored in a processor of an electronic device During operation, the processor in the electronic device executes the above method.
  • Fig. 17 shows a block diagram of an electronic device provided by an embodiment of the present disclosure.
  • an electronic device 1700 may be provided as a server.
  • electronic device 1700 includes processing component 1722 , which further includes one or more processors, and memory resources represented by memory 1732 for storing instructions executable by processing component 1722 , such as application programs.
  • the application programs stored in memory 1732 may include one or more modules each corresponding to a set of instructions.
  • the processing component 1722 is configured to execute instructions to perform the above method.
  • Electronic device 1700 may also include a power supply component 1726 configured to perform power management of electronic device 1700, a wired or wireless network interface 1750 configured to connect electronic device 1700 to a network, and an input-output (I/O) interface 1758 .
  • the electronic device 1700 can operate based on the operating system stored in the memory 1732, such as the Microsoft server operating system (Windows ServerTM), the graphical user interface-based operating system (Mac OS XTM) introduced by Apple Inc., and the multi-user and multi-process computer operating system (UnixTM). ), a free and open source Unix-like operating system (LinuxTM), an open source Unix-like operating system (FreeBSDTM), or similar.
  • a non-transitory computer-readable storage medium such as a memory 1732 including computer program instructions, which can be executed by the processing component 1722 of the electronic device 1700 to implement the above method.
  • the present disclosure can be a system, method and/or computer program product.
  • a computer program product may include a computer readable storage medium having computer readable program instructions thereon for causing a processor to implement various aspects of the present disclosure.
  • a computer readable storage medium may be a tangible device that can retain and store instructions for use by an instruction execution device.
  • a computer readable storage medium may be, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing.
  • Computer-readable storage media include: portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or flash memory), static random access memory (SRAM), compact disc read only memory (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanically encoded device, such as a printer with instructions stored thereon A hole card or a raised structure in a groove, and any suitable combination of the above.
  • RAM random access memory
  • ROM read-only memory
  • EPROM erasable programmable read-only memory
  • flash memory static random access memory
  • SRAM static random access memory
  • CD-ROM compact disc read only memory
  • DVD digital versatile disc
  • memory stick floppy disk
  • mechanically encoded device such as a printer with instructions stored thereon
  • a hole card or a raised structure in a groove and any suitable combination of the above.
  • computer-readable storage media are not to be construed as transient signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., pulses of light through fiber optic cables), or transmitted electrical signals.
  • Computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a respective computing/processing device, or downloaded to an external computer or external storage device over a network, such as the Internet, a local area network, a wide area network, and/or a wireless network.
  • the network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers.
  • a network adapter card or a network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing/processing device .
  • Computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or Source or object code written in any combination, including object-oriented programming languages—such as Smalltalk, C++, etc., and conventional procedural programming languages—such as the “C” language or similar programming languages.
  • Computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server implement.
  • the remote computer can be connected to the user computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (such as via the Internet using an Internet service provider). connect).
  • LAN local area network
  • WAN wide area network
  • an electronic circuit such as a programmable logic circuit, field programmable gate array (FPGA), or programmable logic array (PLA)
  • FPGA field programmable gate array
  • PDA programmable logic array
  • These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine such that when executed by the processor of the computer or other programmable data processing apparatus , producing an apparatus for realizing the functions/actions specified in one or more blocks in the flowchart and/or block diagram.
  • These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause computers, programmable data processing devices and/or other devices to work in a specific way, so that the computer-readable medium storing instructions includes An article of manufacture comprising instructions for implementing various aspects of the functions/acts specified in one or more blocks in flowcharts and/or block diagrams.
  • each block in a flowchart or block diagram may represent a module, a portion of a program segment, or an instruction that includes one or more Executable instructions.
  • the functions noted in the block may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved.
  • each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations can be implemented by a dedicated hardware-based system that performs the specified function or action , or may be implemented by a combination of dedicated hardware and computer instructions.
  • the computer program product can be specifically realized by means of hardware, software or a combination thereof.
  • the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK) etc. wait.
  • a software development kit Software Development Kit, SDK
  • Embodiments of the present disclosure relate to an image processing method and device, electronic equipment, and a storage medium.
  • the method includes: performing semantic category segmentation on a rib image to be segmented to obtain a semantic category segmentation result corresponding to the rib image to be segmented; Instance segmentation to obtain the instance segmentation result corresponding to the rib image to be segmented; according to the semantic category segmentation result and the instance segmentation result, determine the target labeling result corresponding to the rib image to be segmented, wherein the target labeling result includes multiple roots in the rib image to be segmented Ribs and corresponding rib identification for each rib.
  • the embodiment of the present disclosure can comprehensively consider the global semantic information of the image and the local geometric information of the image to improve the segmentation and labeling accuracy of ribs.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

本公开涉及一种图像处理方法及装置、电子设备和存储介质，所述方法包括：对待分割肋骨图像进行语义类别分割，得到待分割肋骨图像对应的语义类别分割结果；对待分割肋骨图像进行实例分割，得到待分割肋骨图像对应的实例分割结果；根据语义类别分割结果和实例分割结果，确定待分割肋骨图像对应的目标标记结果，其中，目标标记结果中包括待分割肋骨图像中的多根肋骨以及每根肋骨对应的肋骨标识。本公开实施例可以综合考虑图像全局语义信息和图像局部几何信息，提高肋骨的分割和标记精度。

Description

图像处理方法、装置、电子设备、存储介质和程序
相关申请的交叉引用
本专利申请要求2021年09月28日提交的中国专利申请号为202111145059.6、申请人为上海商汤智能科技有限公司，申请名称为“图像处理方法及装置、电子设备和存储介质”的优先权，该申请的全文以引用的方式并入本申请中。
技术领域
本公开涉及计算机技术领域，尤其涉及一种图像处理方法、装置、电子设备、存储介质和程序。
背景技术
肋骨是人体骨骼的重要组成部分，其位于人体躯干的上半部分，能够有效地保护人体胸腔内的脏器。由于车祸等事故导致的胸部创伤(例如，肋骨骨折)时常发生，在肋骨骨折诊断中，医生需要根据胸部的医疗影像(例如，胸部CT图像)，给出骨折所在肋骨的准确肋骨标识(肋骨的解剖学标签，例如，左侧3号肋骨)。相关技术中，使用一个神经网络对肋骨进行分割，得到肋骨的分割和标记结果。通过一个神经网络同时完成分割和标记两个任务，很容易导致相邻肋骨间的误识别，导致肋骨的分割和命名精度均较低。
发明内容
本公开提出了一种图像处理方法、装置、电子设备、存储介质和程序的技术方案。
第一方面，本公开提供了一种图像处理方法，包括：对待分割肋骨图像进行语义类别分割，得到所述待分割肋骨图像对应的语义类别分割结果；对所述待分割肋骨图像进行实例分割，得到所述待分割肋骨图像对应的实例分割结果；根据所述语义类别分割结果和所述实例分割结果，确定所述待分割肋骨图像对应的目标标记结果，其中，所述目标标记结果中包括所述待分割肋骨图像中的多根肋骨以及每根肋骨对应的肋骨标识。
第二方面，本公开提供了一种图像处理装置，包括：语义类别分割模块，配置为对待分割肋骨图像进行语义类别分割，得到所述待分割肋骨图像对应的语义类别分割结果；实例分割模块，配置为对所述待分割肋骨图像进行实例分割，得到所述待分割肋骨图像对应的实例分割结果；标记模块，配置为根据所述语义类别分割结果和所述实例分割结果，确定所述待分割肋骨图像对应的目标标记结果，其中，所述目标标记结果中包括所述待分割肋骨图像中的多根肋骨以及每根肋骨对应的肋骨标识。
第三方面，本公开提供了一种电子设备，包括：处理器；用于存储处理器可执行指令的存储器；其中，所述处理器被配置为调用所述存储器存储的指令，以执行上述方法。
第四方面，本公开提供了一种计算机可读存储介质，其上存储有计算机程序指令，所述计算机程序指令被处理器执行时实现上述方法。
第五方面，本公开提供了一种计算机程序，包括计算机可读代码，当所述计算机可读代码在电子设备中运行时，所述电子设备中的处理器执行用于实现上述任意一种图像处理方法。
在本公开实施例中，基于图像全局语义信息，对待分割肋骨图像进行语义类别分割，得到待分割肋骨图像对应的语义类别分割结果；基于图像局部几何信息，对待分割肋骨图像进行实例分割，得到待分割肋骨图像对应的实例分割结果；综合考虑了图像全局语义信息和图像局部几何信息，因此，基于语义类别分割结果和实例分割结果，确定的目标标记结果具有较高的准确率，目标标记结果包括待分割肋骨图像中的多根肋骨以及每根肋骨对应的肋骨标识，从而有效提高了肋骨的分割和标记精度。
应理解，以上的一般描述和后文的细节描述仅是示例性和解释性的，而非限制本公开。根据下面参考附图对示例性实施例的详细说明，本公开的其它特征及方面将变得清楚。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，这些附图示出了符合本公开的实施例，并与说明书一起用于说明本公开的技术方案。
图1示出本公开实施例提供的一种图像处理方法的流程图；
图2示出本公开实施例提供的对初始肋骨图像进行z轴分类的示意图；
图3示出本公开实施例提供的预设z轴分类网络的示意图；
图4示出本公开实施例提供的肋骨凸包区域的示意图；
图5示出本公开实施例提供的预设肋骨凸包分割网络的示意图；
图6示出本公开实施例提供的待分割肋骨图像对应的语义类别分割结果的示意图；
图7示出本公开实施例提供的预设语义类别分割网络的示意图；
图8示出本公开实施例提供的待分割肋骨图像对应的实例分割结果的示意图；
图9示出本公开实施例提供的预设肋骨实例分割网络的示意图；
图10示出本公开实施例提供的待分割肋骨图像对应的目标标记结果的示意图
图11示出本公开实施例提供的待分割肋骨图像对应的细二值分割结果的示意图；
图12示出本公开实施例提供的预设肋骨二值分割网络的示意图；
图13示出本公开实施例提供的目标标记结果中存在粘连肋骨的示意图；
图14示出本公开实施例提供的对图13所示的粘连肋骨进行解粘连后的示意图；
图15示出本公开实施例提供的一种图像处理装置的框图；
图16示出本公开实施例提供的一种电子设备的框图；
图17示出本公开实施例提供的一种电子设备的框图。
具体实施方式
以下将参考附图详细说明本公开的各种示例性实施例、特征和方面。附图中相同的附图标记表示功能相同或相似的元件。尽管在附图中示出了实施例的各种方面，但是除非特别指出，不必按比例绘制附图。
为了更好地说明本公开，在下文的具体实施方式中给出了众多的具体细节。本领域技术人员应当理解，没有某些具体细节，本公开同样可以实施。在一些示例中，对于本领域技术人员熟知的方法、手段、元件和电路未作详细描述，以便于凸显本公开的主旨。
本公开实施例提供的图像处理方法，可以由终端设备或服务器等电子设备执行，终端设备可以为用户设备(User Equipment，UE)、移动设备、用户终端、蜂窝电话、无绳电话、个人数字助理(Personal Digital Assistant，PDA)、手持设备、计算设备、车载设备、可穿戴设备等，该图像处理方法可以通过处理器调用存储器中存储的计算机可读指令的方式来实现。或者，可通过服务器执行该图像处理方法。
本公开实施例提供的图像处理方法，可以准确定位骨折所在肋骨的解剖学标签和骨折的大致位置，并给出肋骨3D重建可视化结果。能够为肋骨骨折检测、肋骨参数定量化分析、肋骨疾病筛查、病灶检测等项目提供技术支持，实现肋骨相关参数的自动化分析，从而，降低医生的工作量和错误率，缩短了诊断周期。
图1示出本公开实施例提供的一种图像处理方法的流程图。如图1所示，本申请实施例提供的图像处理方法，在由电子设备执行时，可以包括以下步骤：
步骤S11：对待分割肋骨图像进行语义类别分割，得到待分割肋骨图像对应的语义类别分割结果。
步骤S12：对待分割肋骨图像进行实例分割，得到待分割肋骨图像对应的实例分割结果。
步骤S13：根据语义类别分割结果和实例分割结果，确定待分割肋骨图像对应的目标标记结果，其中，目标标记结果中包括待分割肋骨图像中的多根肋骨以及每根肋骨对应的肋骨标识。
在本公开实施例中，基于图像全局语义信息，对待分割肋骨图像进行语义类别分割，得到待分割肋骨图像对应的语义类别分割结果；基于图像局部几何信息，对待分割肋骨图像进行实例分割，得到待分割肋骨图像对应的实例分割结果；综合考虑了图像全局语义信息和图像局部几何信息，因此，基于语义类别分割结果和实例分割结果确定的目标标记结果具有较高的准确率，目标标记结果包括待分割肋骨图像中的多根肋骨以及每根肋骨对应的肋骨标识，从而有效提高了肋骨的分割精度和标记精度。
在一种可能的实现方式中，本申请实施例提供的图像处理方法还包括以下步骤：获取原始胸部扫描图像；对原始胸部扫描图像进行图像预处理，得到初始肋骨图像；对初始肋骨图像进行凸包分割，得到初始肋骨图像中的肋骨凸包区域；根据肋骨凸包区域，对初始肋骨图像进行裁剪，得到待分割肋骨图像。
在本公开实施例中，在对原始胸部扫描图像进行图像预处理得到初始肋骨图像后，可以通过对初始肋骨图像进行凸包分割，确定初始肋骨图像中的肋骨凸包区域，从而可以根据肋骨凸包区域，从初始肋骨图像中裁剪得到待分割肋骨图像，使得后续基于裁剪后的待分割肋骨图像进行肋骨标记，可以有效降低对计算资源的消耗，提高肋骨标记效率。
在一种可能的实现方式中，原始胸部扫描图像可以是胸部计算机X线断层扫描(CT)图像I。由于CT图像具有良好的骨-软组织对比度，因此，胸部CT图像I通常用于作为进行肋骨骨折诊断的医疗影像。
在示例中，对胸部CT图像I进行图像预处理，得到初始肋骨图像I n。图像预处理可以包括：重定向、裁剪、归一化等中的一种或多种，本公开对图像预处理的具体方式不做限定。
应理解，由于拍摄角度不同，不同胸部CT图像中的肋骨方向可能不同。
在示例中，根据预设单位矩阵对胸部CT图像I进行重定向的预处理，得到初始肋骨图像I n，使 得初始肋骨图像I n中的肋骨方向与预设的坐标轴(x/y/z轴)一致，以提高后续处理效率。预设单位矩阵可以根据实际情况进行设置，本公开对预设单位矩阵的具体形式不做限定。
在示例中，胸部CT图像I中除了包括肋骨以外，还包括大面积的其它背景部分，为了降低后续的计算资源消耗，提高处理效率，可以利用预设灰度阈值对胸部CT图像I进行裁剪处理。
在示例中，利用预设灰度阈值对胸部CT图像I进行裁剪处理可以包括以下步骤：基于预设灰度阈值对胸部CT图像I进行二值化处理，得到二值图像。根据二值图像中像素值为1的像素点构成的包围框，对胸部CT图像I进行裁剪，得到初始肋骨图像I n，以减小图像尺寸，从而有效降低后续的计算资源消耗，提高处理效率。
应理解，预设灰度阈值的具体取值可以根据实际情况进行设置，本公开实施例对此不做具体限定。
在示例中，胸部CT图像I中灰度值大于或等于预设灰度阈值的像素点，在二值图像中对应的像素值为1；胸部CT图像I中灰度值小于预设灰度阈值的像素点，在二值图像中对应的像素值为0。
在实际应用中，若拍摄胸部CT图像I的患者佩带金属等物品时，会使得拍摄得到的胸部CT图像I中存在灰度值过高的像素点，影响后续处理精度。因此，针对上述胸部CT图像I裁剪后得到的初始肋骨图像I n，利用预设灰度值归一化窗口进行归一化处理，使得初始肋骨图像I n的灰度值在合理灰度值范围内，提高肋骨标记精度。
在示例中，预设灰度值归一化窗口可以根据实际情况进行设置，例如，预设灰度值归一化窗口的取值范围是[-1000，2000]，本公开对预设灰度值归一化窗口的实际取值范围不作具体限定。
在一种可能的实现方式中，对初始肋骨图像进行凸包分割，得到初始肋骨图像中的肋骨凸包区域，包括：确定初始肋骨图像中的肋骨区域是否符合预设完整性要求；在初始肋骨图像中的肋骨区域符合预设完整性要求的情况下，对初始肋骨图像进行凸包分割，得到肋骨凸包区域。
在示例中，对胸部CT图像I进行图像预处理得到初始肋骨图像I n之后，为了提高后续肋骨标记精度，可以首先对初始肋骨图像I n进行质量检测，检测初始肋骨图像I n中的肋骨区域是否符合预设完整性要求。
应理解，由于在初始肋骨图像I n中的肋骨区域不符合预设完整性要求的情况下，无法完成后续的肋骨标记过程，因此，仅在初始肋骨图像I n中的肋骨区域符合预设完整性要求的情况下，对初始肋骨图像I n进行凸包分割，得到肋骨凸包区域。
其中，初始肋骨图像I n中的肋骨区域符合预设完整性要求，可以指的是初始肋骨图像中的肋骨区域比较完整，不存在严重缺失的情况，本公开实施例对此不作具体限定。
在一种可能的实现方式中，初始肋骨图像包括多个切片图像；在上述图像处理方法中，确定初始肋骨图像中的肋骨区域是否符合预设完整性要求，可以包括以下步骤：确定多个切片图像中每个切片图像的肋骨类别，其中，不同肋骨类别用于指示不同肋骨区域；根据每个切片图像的肋骨类别，确定肋骨类别序列；在肋骨类别序列中包括预设肋骨类别的情况下，确定初始肋骨图像中的肋骨区域符合预设完整性要求。
在示例中，根据完整肋骨区域的分布情况，可以将肋骨区域划分为多个肋骨区域，每个肋骨区域可以对应一个肋骨类别，因此，通过确定初始肋骨图像I n中每个切片图像对应的肋骨类别，进而确定初始肋骨图像I n对应的肋骨类别序列。
应理解，在肋骨类别序列中包括预设肋骨类别的情况下，可以确定初始肋骨图像I n中包括大部分肋骨区域，即初始肋骨图像I n中的肋骨区域比较完整，不存在严重缺失的情况，初始肋骨图像I n中的肋骨区域符合预设完整性要求。
在示例中，由于不同初始肋骨图像的分辨率可能不同，为了提高后续处理效率，可以对初始肋骨图像I n进行重采样，得到第二肋骨图像，其中，第二肋骨图像的分辨率是第三分辨率。第三分辨率的具体取值可以根据实际情况确定，本公开实施例对此不作具体限定。例如，第三分辨率是1mm×1mm×3mm，即，第二肋骨图像中的每个像素点对应的实际物理尺寸是1mm×1mm×3mm。
应理解，初始肋骨图像I n是胸部CT图像I经过图像预处理得到的，胸部CT图像I包括多个切片图像，因此，初始肋骨图像I n以及对初始肋骨图像I n进行重采样后得到的第二肋骨图像，也包括多个切片图像。
在示例中，可以基于预设z轴分类网络，确定每个切片图像的肋骨类别。遍历第二肋骨图像中包括的多个切片图像，将每个切片图像分别输入预设z轴分类网络，从而可以得到每个切片图像的肋骨类别，不同肋骨类别用于指示不同肋骨区域。
图2示出本公开实施例提供的对初始肋骨图像进行z轴分类的示意图。如图2所示，完整的肋骨区域可以包括5个肋骨类别(类别0、类别1、类别2、类别3和类别4)。基于预设z轴分类网络，可以确定每个切片图像的肋骨类别；根据切片图像的肋骨类别，可以确定切片图像位于哪个肋骨区域。如图2所示，切片图像i的肋骨类别q i是类别1，切片图像j的肋骨类别q j是类别3。
在示例中，根据第二肋骨图像中每个切片图像的肋骨类别，确定肋骨类别序列Q＝(q 1,q 2,...,q d)，q i∈{类别0，类别1，类别2，类别3，类别4}，i∈[0,d]，d是切片图像的总数目。根据每个切片图像的编号，将肋骨类别序列Q处理为单调有序的肋骨类别序列
Figure PCTCN2022077184-appb-000001
应理解，在肋骨类别序列
Figure PCTCN2022077184-appb-000002
中包括预设肋骨类别的情况下，可以确定初始肋骨图像I n中的肋骨区域符合预设完整性要求。例如，预设肋骨类别包括类别1、类别2和类别3。如图2所示，当肋骨类别包括类别1、类别2和类别3的情况下，可以确定初始肋骨图像I n中的肋骨区域比较完整，不存在严重缺失的情况。
在一种可能的实现方式中，预设z轴分类网络可以包括若干个卷积层、下采样层、全局平均池化层、全连接层。图3示出本公开实施例提供的预设z轴分类网络的示意图。
在示例中，对预设z轴分类网络进行训练时，可以采用warmup和cosine annealing的学习率设置策略，以及采用交叉熵损失函数，对预设z轴分类网络进行若干训练回合的训练。
在示例中，可以通过最小化损失函数L 1，实现对预设z轴分类网络的训练。损失函数L 1如下所示：
Figure PCTCN2022077184-appb-000003
其中，损失函数L 1中的y i是训练样本图像i对应的分类标签，
Figure PCTCN2022077184-appb-000004
是根据预设z轴分类网络确定的训练样本图像i对应的分类预测概率。
在示例中，预设z轴分类网络的具体网络结构和训练过程可以采用相关技术中的其它网络结构和训练方式，本公开实施例对此不作具体限定。
应理解，胸部CT图像I的扫描区域很大，胸部CT图像I中还包括椎骨、髋骨、股骨等其它骨头，肋骨区域只占其中很小一部分，因此，采用上述方式确定初始肋骨图像I n中的肋骨区域符合预设完整性要求的情况下，可以对初始肋骨图像I n进行凸包分割，以得到初始肋骨图像I n中的肋骨凸包区域。
在示例中，基于肋骨凸包区域对初始肋骨图像I n进行裁剪，可以包括肋骨区域的待分割肋骨图像I v。后续基于待分割肋骨图像I v进行肋骨标记，可以降低对计算资源的浪费，提高肋骨标记效率。
在示例中，可以基于预设肋骨凸包分割网络，对初始肋骨图像I n进行分割，得到初始肋骨图像I n中的肋骨凸包区域。为了提高分割效率，可以对初始肋骨图像I n进行重采样，得到第一肋骨图像I sp3，其中，第一肋骨图像I sp3的分辨率是第一分辨率。
应理解，第一分辨率的具体取值可以根据实际情况确定，本公开实施例对此不作具体限定。例如，第一分辨率是3mm×3mm×3mm，即，第一肋骨图像I sp3中的每个像素点对应的实际物理尺寸是3mm ×3mm×3mm。
在示例中，将第一肋骨图像I sp3输入预设肋骨凸包分割网络，预设肋骨凸包分割网络对第一肋骨图像I sp3进行分割后，得到肋骨凸包区域。图4示出本公开实施例提供的肋骨凸包区域的示意图。如图4所示，肋骨凸包区域包括左侧肋骨对应的肋骨凸包区域H l，以及右侧肋骨对应的肋骨凸包区域H r。
在示例中，根据肋骨凸包区域，可以确定包括肋骨整体区域的检测框。例如，该检测框可以是包括肋骨凸包区域H l和肋骨凸包区域H r的最小矩形框。根据该检测框，对初始肋骨图像I n进行裁剪，得到包括肋骨整体区域的待分割肋骨图像I v。基于待分割肋骨图像I v，执行后续的肋骨标记。
在一种可能的实现方式中，预设肋骨凸包分割网络可以是3D-U型网络，包括由若干个卷积层和下采样层构成的编码器，以及由若干个卷积层和上采样层构成的解码器，编码器和解码器之间嵌入non-local模块，编码器和解码器对应阶段引入跳跃连接。图5示出本公开实施例提供的预设肋骨凸包分割网络的示意图。
在示例中，对预设肋骨凸包分割网络进行训练时，可以采用warmup和cosine annealing的学习率设置策略，以及采用cross entropy(交叉熵损失)和dice loss作为损失函数，对预设肋骨凸包分割网络进行若干训练回合的训练。
在示例中，可以通过最小化损失函数L 2，实现对预设肋骨凸包分割网络的训练。损失函数L 2如下所示：
Figure PCTCN2022077184-appb-000005
其中，公式(2)中的yi是训练样本图像i对应的分割标签，
Figure PCTCN2022077184-appb-000006
是根据预设肋骨凸包分割网络确定的训练样本图像i对应的分割预测概率，Y是训练样本图像i对应的真实分割结果，
Figure PCTCN2022077184-appb-000007
是根据预设肋骨凸包分割网络确定的训练样本图像i对应的预测分割结果。
在示例中，预设肋骨凸包分割网络的具体网络结构和训练过程可以采用相关技术中的其它网络结构和训练方式，本公开实施例对此不作具体限定。
在示例中，可以利用凸包算法，对初始肋骨图像I n进行凸包分割，得到初始肋骨图像I n中的肋骨凸包区域。其中，凸包算法的具体算法形式，可以根据实际情况灵活设置，本公开实施例对此不作具体限定。
在一种可能的实现方式中，在上述图像处理方法中，对待分割肋骨图像I v进行语义类别分割，得到待分割肋骨图像I v对应的语义类别分割结果A l，可以包括以下步骤：对待分割肋骨图像I v进行重采样，得到第一肋骨图像I sp3，其中，第一肋骨图像I sp3的分辨率是第一分辨率；利用预设语义类别分割网络，对第一肋骨图像I sp3进行语义类别分割，得到待分割肋骨图像I v对应的语义类别分割结果A l。
应理解，在第一分辨率下对待分割肋骨图像I v进行语义类别分割，可以有效提高语义类别分割效率。
在示例中，图6示出本公开实施例提供的待分割肋骨图像对应的语义类别分割结果的示意图。如图6所示，语义类别分割结果A l中包括待分割肋骨图像I v中的多根肋骨以及每根肋骨对应的第一预测肋骨标识。在图6中，可以通过不同的颜色来指示每根肋骨以及每根肋骨对应的第一预测肋骨标识。例如，红色用于指示右侧1号肋骨，紫色用于指示左侧第2根肋骨，以此类推。
在示例中，语义类别分割结果中可以采用相关技术中的其它形式来指示每根肋骨以及每根肋骨对应的第一预测肋骨标识，本公开实施例对此不作具体限定。
在一种可能的实现方式中，预设语义类别分割网络的网络结构和训练方式，可以和预设肋骨凸包分割网络的网络结构和训练方式相同。图7示出本公开实施例提供的预设语义类别分割网络的示意图。
在示例中，预设语义类别分割网络的具体网络结构和训练过程可以采用相关技术中的其它网络结构和训练方式，本公开实施例对此不作具体限定。
在一种可能的实现方式中，对待分割肋骨图像进行实例分割，得到待分割肋骨图像对应的实例分割结果，包括：对待分割肋骨图像进行重采样，得到第一肋骨图像，其中，第一肋骨图像的分辨率是第一分辨率；对第一肋骨图像中的像素点进行位置编码，得到位置编码图像；基于位置编码图像，对第一肋骨图像进行实例分割，得到实例分割结果。
应理解，在第一分辨率下对待分割肋骨图像I v进行分割，可以有效提高实例分割效率，对第一分辨率的第一肋骨图像I sp3进行位置编码，基于位置编码后得到的位置编码图像，对第一肋骨图像I sp3进行实例分割，可以有效提高实例分割精度。
在示例中，可以利用下述公式(3)对第一肋骨图像I sp3进行位置编码，得到位置编码图像I c，
Figure PCTCN2022077184-appb-000008
其中，(i,j,k)是第一肋骨图像I sp3和位置编码图像I c中的对应像素点，(μ x,μ y,μ z)是第一肋骨图像I sp3的图像中心像素点，w x、w y和w z为预设超参数。预设超参数w x、w y和w z的具体取值可以根据实际情况确定，本公开实施例对此不作具体限定。
在一种可能的实现方式中，基于位置编码图像，对第一肋骨图像进行实例分割，得到实例分割结果，包括：对第一肋骨图像进行二值分割，得到第一肋骨图像对应的目标粗二值分割结果；基于位置编码图像，确定第一肋骨图像对应的像素点嵌入向量；根据目标粗二值分割结果和第一肋骨图像对应的像素点嵌入向量，确定第一肋骨图像中的肋骨区域对应的像素点嵌入向量；对第一肋骨图像中的肋骨区域对应的像素点嵌入向量进行聚类，得到实例分割结果。
在一种可能的实现方式中，对第一肋骨图像进行二值分割，得到第一肋骨图像对应的目标粗二值分割结果，包括：对第一肋骨图像进行二值分割，得到第一肋骨图像对应的初始粗二值分割结果；根据肋骨凸包区域，对初始粗二值分割结果中的非肋骨区域进行过滤，得到目标粗二值分割结果。
在示例中，可以基于预设肋骨实例分割网络，对第一肋骨图像I sp3进行实例分割。将第一肋骨图像I sp3和位置编码图像I c同时输入预设肋骨实例分割网络。
在示例中，预设肋骨实例分割网络包括两个分支，一个分支用于对第一肋骨图像I sp3进行二值分割，得到第一肋骨图像I sp3对应的初始粗二值分割结果A bc。利用肋骨凸包区域H l和H r，对初始粗二值分割结果A bc进行过滤，消除初始粗二值分割结果A bc中被误分割为肋骨区域的假阳部分，得到精度较高的目标粗二值分割结果A bc。例如，目标粗二值分割结果A bc＝A bc∩(H l∪H r)。
在示例中，预设肋骨实例分割网络的另一个分支用于基于位置编码图像I c，确定第一肋骨图像I sp3对应的像素点嵌入向量A e。像素点嵌入向量A e中包括第一肋骨图像I sp3中每个像素点对应的一个嵌入向量。嵌入向量的维度可以是8维，也可以根据实际情况设置为其它维度，本公开实施例对此不作具体限定。
在示例中，利用目标粗二值分割结果A bc和像素点嵌入向量A e，可以得到第一肋骨图像I sp3中的肋骨区域对应的像素点嵌入向量A re＝A bc∩A e。利用mean-shift聚类算法对肋骨区域对应的像素点嵌入向量A re进行聚类，得到待分割肋骨图像I v对应的实例分割结果A ins。
在示例中，聚类算法除了可以采用mean-shift聚类算法，还可以采用相关技术中的其它聚类算法，本公开实施例对此不作具体限定。
图8示出本公开实施例提供的待分割肋骨图像对应的实例分割结果的示意图。如图8所示，实例分割结果A ins中包括待分割肋骨图像I v中的多根肋骨。但是，实例分割结果A ins无法确定每根肋骨的肋骨标识。
在一种可能的实现方式中，预设肋骨实例分割网络可以是3D-U型网络，两个编码器分支共享一个解码器，编码器和解码器之间嵌入non-local模块。其中，一个编码器分支用于对第一肋骨图像I sp3进行二值分割，得到第一肋骨图像I sp3对应的初始粗二值分割结果A bc，另一个编码器分支用于基于位置编码图像I c，确定第一肋骨图像I sp3对应的像素点嵌入向量A e。图9示出本公开实施例提供的预设肋骨实例分割网络的示意图。
在示例中，对预设肋骨实例分割网络进行训练时，用于进行二值分割的编码器分支可以采用cross entropy(交叉熵损失)和dice loss作为损失函数，采用损失函数对用于进行二值分割的编码器分支进行若干训练回合的训练。具体训练公式可以参考上述公式(2)。
在示例中，对预设肋骨实例分割网络进行训练时，用于嵌入向量预测的编码器分支可以采用discriminative loss作为损失函数，采用损失函数用于嵌入向量预测的编码器分支进行若干训练回合的训练。
在示例中，可以通过最小化损失函数L d，实现对用于嵌入向量预测的编码器分支的训练。损失函数L d如下所示：
Figure PCTCN2022077184-appb-000009
其中，损失函数L d中的C是训练样本图像中的类别总数，N c是训练样本图像中属于同一类别的像素点数量，μ c是类别的均值向量，μ c是训练样本图像中像素点i的嵌入向量。
在示例中，预设肋骨实例分割网络的具体网络结构和训练过程可以采用相关技术中的其它网络结构和训练方式，本公开实施例对此不作具体限定。
应理解，待分割肋骨图像对应的语义类别分割结果是根据图像全局信息得到的，能够得到待分割肋骨图像中的多根肋骨以及每根肋骨对应的第一预测肋骨标识。但是由于语义类别分割可能在相邻肋骨之间出现误分割，导致第一预测肋骨标识的准确率较低。待分割肋骨图像对应的实例分割结果是根据图像局部信息得到的，虽然实例分割结果中无法得到每根肋骨的肋骨标识，但是实例分割结果能够很好地将相邻肋骨分割开来。因此，综合考虑语义类别分割结果和实例分割结果，可以更加准确地得到每根肋骨的肋骨标识。
在一种可能的实现方式中，根据语义类别分割结果和实例分割结果，确定待分割肋骨图像对应的目标标记结果，包括：根据实例分割结果和肋骨凸包区域，确定单侧肋骨序列，其中，单侧肋骨序列中包括多根单侧肋骨及其排序，单侧肋骨序列包括左侧肋骨序列和右侧肋骨序列；根据语义类别分割结果和单侧肋骨序列，确定待分割肋骨图像中的多根肋骨以及每根肋骨对应的肋骨标识。
在一种可能的实现方式中，根据实例分割结果和肋骨凸包区域，确定单侧肋骨序列，包括：根据实例分割结果和肋骨凸包区域，确定第一单侧肋骨集合，其中，第一单侧肋骨序列中包括N根单侧肋骨，N是单侧肋骨的数目；对N根单侧肋骨在z轴方向进行排序，并取排序前K根肋骨构成第二单侧肋骨集合，其中，K是大于1且小于N的整数；对第二单侧肋骨集合中的每根肋骨进行平面拟合，得到K个拟合平面；根据K个拟合平面之间的位置关系，对K根肋骨进行排序，得到单侧肋骨序列。
在示例中，根据肋骨凸包区域H l和H r，将实例分割结果A ins中的肋骨实例分成两个第一单侧肋 骨集合：左侧对应的第一单侧肋骨集合F l＝A ins∩H，以及右侧对应的第一单侧肋骨集合F r＝A ins∩H。
在示例中，以左侧的第一单侧肋骨集合F l为例，第一单侧肋骨集合F l中包括N根单侧肋骨，N是单侧肋骨的数目，例如，N＝12。对第一单侧肋骨集合F l中的N根单侧肋骨在z轴方向进行排序，并取排序前K根肋骨构成第二单侧肋骨集合S l＝(R 1,R 2,...,R K)。
应理解，K是超参数，K是大于1且小于N的整数，K的具体取值可以根据实际情况确定，本公开实施例对此不作具体限定。
在示例中，对第一单侧肋骨集合F l中的N根单侧肋骨在z轴方向进行排序的方式，可以是基于z轴95分位数、z轴90分位数、z轴85分位数等方式，本公开实施例对此不作具体限定。对于第二单侧肋骨集合中的每根肋骨进行平面拟合，确定每根肋骨对应的拟合平面。
在一种可能的实现方式中，该图像处理方法还包括：根据K根肋骨中每根肋骨对应的拟合平面，确定每根肋骨对应的平面法向量和肋骨质心；针对K根肋骨中的任意两根肋骨R i和R j，根据肋骨R i对应的平面法向量和肋骨质心，以及肋骨R j对应的平面法向量和肋骨质心，确定肋骨R i和肋骨R j之间的内积，其中，肋骨R i和肋骨R j之间的内积的大小，用于指示肋骨R i对应的拟合平面和肋骨R j对应的拟合平面之间的位置关系。
在示例中，针对任意两根肋骨R i、R j∈S l，对肋骨R i进行平面拟合，得到肋骨R i对应的平面法向量n i和肋骨质心m i；对肋骨R j进行平面拟合，得到肋骨R j对应的平面法向量n j和肋骨质心m j。以肋骨R i所在的平面为参考，即肋骨R i的平面法向量的方向指向人体头部所在的方向，在确定肋骨R i和肋骨R j的内积p j,i时，可以采用以下关于内积的计算公式：
p j,i＝(m j-m i)·n i   (5)。
应理解，若内积p j,i为正，则肋骨R j的拟合平面在肋骨R i的拟合平面的上方，即肋骨R j在肋骨R i上方。若内积p j,i为负，则肋骨R j的拟合平面在肋骨R i的拟合平面的下方，即肋骨R j在肋骨R i的上方。
在示例中，以肋骨R i所在的平面为参考，若肋骨R i和肋骨R x的内积p x,i为正，肋骨R i和肋骨R y的内积p y,i也为正，且内积p x,i大于内积p y,i，则肋骨R i、肋骨R x和肋骨R y从上到下的排序为R x>R y>R i。
在示例中，针对任意肋骨R i∈S l，以肋骨R i所在的平面为参考，确定肋骨R i与其余K-1根肋骨的内积，并根据肋骨R i与其余K-1根肋骨的内积，对K根肋骨进行排序，得到肋骨R i对应的肋骨序列S i。遍历第二单侧肋骨集合S l中的K根肋骨，得到K个肋骨序列。根据每根肋骨在K个肋骨序列中的排序，投票得到最终的单侧肋骨序列S。
在示例中，针对肋骨R i，统计其在K个肋骨序列中出现次数最多的排序，作为其在最终的单侧肋骨序列S中的最终排序。以此类推，针对右侧的第一单侧肋骨集合F r，也可以采用上述方式得到其对应的单侧肋骨序列S。
在一种可能的实现方式中，根据语义类别分割结果和单侧肋骨序列，确定待分割肋骨图像中的多根肋骨以及每根肋骨对应的肋骨标识，包括：根据语义类别分割结果，确定单侧肋骨序列中的目标肋骨对应的肋骨标识，其中，目标肋骨是所述单侧肋骨序列中的任意一根肋骨；根据目标肋骨对应的肋 骨标识以及单侧肋骨序列，确定待分割肋骨图像中的多根肋骨以及每根肋骨对应的肋骨标识。
在示例中，单侧肋骨序列中包括的是一个顺序的肋骨序列，因此，基于语义类别分割结果，确定单侧肋骨序列中的任意一个目标肋骨对应的肋骨标识之后，基于单侧肋骨序列的排序，就可以得到待分割肋骨图像I v中的每根肋骨对应的肋骨标识。
在示例中，单侧肋骨序列S＝(R 1,R 2,...,R K)，则其对应的肋骨标识为(q *,q *+1,...,q *+k-1)，其中，
Figure PCTCN2022077184-appb-000010
分别对左侧和右侧的单侧肋骨序列执行上述处理，得到待分割肋骨图像I v对应的目标标记结果A c。
在示例中，图10示出本公开实施例提供的待分割肋骨图像对应的目标标记结果的示意图。如图10所示，目标标记结果A c中包括待分割肋骨图像I v中的多根肋骨以及每根肋骨对应的肋骨标识。在图10中，可以通过不同的颜色来指示每根肋骨以及每根肋骨对应的肋骨标识。例如，红色用于指示右侧1号肋骨，紫色用于指示左侧第2根肋骨，以此类推。
在示例中，目标标记结果A c中可以采用相关技术中的其它形式来指示每根肋骨以及每根肋骨对应的肋骨标识，本公开实施例对此不作具体限定。图10中的目标标记结果A c相比于图6中的语义类别分割结果A l，每根肋骨的肋骨标识更加准确。
应理解，结合图像全局信息和图像局部信息得到的目标标记结果，能够提高肋骨断裂、严重骨折、肋骨错位等场景下的肋骨标记准确率。
在一种可能的实现方式中，该图像处理方法还包括：在语义类别分割和实例分割是在第一分辨率下进行的情况下，在第二分辨率下对待分割肋骨图像进行二值分割，得到待分割肋骨图像对应的细二值分割结果，其中，第二分辨率大于第一分辨率；根据细二值分割结果，更新目标标记结果。
在示例中，对待分割肋骨图像分别进行语义类别分割和实例分割是在第一分辨率下进行的情况下，目标标记结果中每根肋骨的分割结果比较粗糙，因此，基于分辨率更高的细二值分割结果，更新目标标记结果，得到分辨率更高以及分割精度更高的目标标记结果。
在一种可能的实现方式中，在第二分辨率下对待分割肋骨图像I v进行二值分割，得到待分割肋骨图像I v对应的细二值分割结果A bf，包括：对待分割肋骨图像I v进行重采样，得到第三肋骨图像I sp1.5，其中，第三肋骨图像I sp1.5的分辨率是第二分辨率；利用预设肋骨二值分割网络，对第三肋骨图像I sp1.5进行二值分割，得到待分割肋骨图像I v对应的细二值分割结果A bf。例如，第二分辨率是1.5mm×1.5mm×1.5mm，即，第三肋骨图像I sp1.5中的每个像素点对应的实际物理尺寸是1.5mm×1.5mm×1.5mm。
在示例中，第三肋骨图像I sp1.5中包括多个切片图像，利用预设肋骨二值分割网络，遍历第三肋骨图像I sp1.5中的每个切片图像，确定每个切片图像对应的预测二值分割结果，进而将所有切片图像的预测二值分割结果堆叠起来，得到待分割肋骨图像I v对应的细二值分割结果A bf。
在示例中，图11示出本公开实施例提供的待分割肋骨图像对应的细二值分割结果的示意图。如图11所示，细二值分割结果A bf中包括待分割肋骨图像I v中的多根肋骨。但是，细二值分割结果A bf无法确定每根肋骨的肋骨标识。
在一种可能的实现方式中，预设肋骨二值分割网络可以是2.5D Unet网络，包括由若干个卷积层和下采样层构成的编码器，以及由若干个卷积层和上采样层构成的解码器，编码器和解码器之间嵌入non-local模块，编码器和解码器对应阶段引入跳跃连接。图12示出本公开实施例提供的预设肋骨二值分割网络的示意图。
在示例中，对预设肋骨二值分割网络进行训练时，可以采用warmup和cosine annealing的学习率设置策略，以及采用cross entropy和dice loss作为损失函数，对预设肋骨二值分割网络进行若干训练回合 的训练。具体训练公式可以参考上述公式(2)。
在示例中，预设肋骨二值分割网络的具体网络结构和训练过程可以采用相关技术中的其它网络结构和训练方式，本公开实施例对此不作具体限定。
在一种可能的实现方式中，根据细二值分割结果，更新目标标记结果，包括：确定细二值分割结果中的多个连通域；根据多个连通域，更新目标标记结果。
在示例中，确定细二值分割结果A bf中的多个连通域{C i}，针对任意一个连通域C i，更新其肋骨标识为
Figure PCTCN2022077184-appb-000011
遍历细二值分割结果A bf中的每个连通域，得到更新后的目标标记结果A f。
图13示出本公开实施例提供的目标标记结果中存在粘连肋骨的示意图。如图13所示，目标标记结果中可能会出现相邻肋骨之间存在粘连，导致包括至少两根相邻肋骨的粘连肋骨在目标标记结果中用相同的颜色指示，即包括至少两根相邻肋骨的粘连肋骨被确定为相同的肋骨标识，导致肋骨标记不准确。
在一种可能的实现方式中，该图像处理方法还包括：基于预设肋骨粘连检测条件，对目标标记结果进行粘连检测，确定目标标记结果中是否存在粘连肋骨，其中，粘连肋骨用于指示对应相同的肋骨标识的至少两根相邻肋骨；在确定目标标记结果中存在粘连肋骨的情况下，确定粘连肋骨对应的像素点嵌入向量；对粘连肋骨对应的像素点嵌入向量进行聚类，得到粘连肋骨解粘连后的每根肋骨对应的肋骨标识。
在示例中，预设肋骨粘连检测条件可以至少包括下述四个条件：
条件1：相对体积先验条件；针对目标标记结果中的任意肋骨R i，若
Figure PCTCN2022077184-appb-000012
则可以确定肋骨R i为可疑粘连肋骨。其中，
Figure PCTCN2022077184-appb-000013
是肋骨R i的平均相对体积，V r(R i)可以通过大数据统计得到，|·|用于指示体积，N是大数据统计过程对应的数据统计量，K是待分割肋骨图像中的肋骨数量，T 1是超参数，N和T 1的具体取值可以根据实际情况确定，本公开实施例对此不作具体限定。
条件2：肋骨序列不连续条件1；对于目标标记结果中的任意相邻两根肋骨
Figure PCTCN2022077184-appb-000014
和
Figure PCTCN2022077184-appb-000015
若S i+1＜S i+1，则可以确定
Figure PCTCN2022077184-appb-000016
和
Figure PCTCN2022077184-appb-000017
为可疑粘连肋骨。
条件3：肋骨序列不连续条件2；对于目标标记结果中的任意相邻三根肋骨
Figure PCTCN2022077184-appb-000018
若
Figure PCTCN2022077184-appb-000019
则可以确定肋骨
Figure PCTCN2022077184-appb-000020
和
Figure PCTCN2022077184-appb-000021
为可疑粘连肋骨。其中，T 2是超参数，T 2的具体取值可以根据实际情况确定，本公开实施例对此不作具体限定。
条件4：类别条件；针对目标标记结果中的任意肋骨R i，若肋骨R i在利用细二值分割结果进行更新前的目标标记结果中对应多个肋骨标识，且某个肋骨标识的体积占比小于0.8，则可以确定肋骨R i是类别不纯肋骨。
应理解，针对目标标记结果中的任意肋骨R i，若其满足条件对d)且满足条件a)、b)、c)中的任意一个，则可以确定肋骨R i是粘连肋骨。遍历目标标记结果中的每根肋骨，得到粘连肋骨集合M。
在示例中，粘连检测条件除了可以包括上述四种之外，还可以包括其它形式的粘连检测条件，本公开实施例对此不作具体限定。
在本公开实施例中，针对任意肋骨R i∈M，确定肋骨R i对应的像素点嵌入向量A i＝A e∩R i，利用mean-shift聚类算法对肋骨R i对应的像素点嵌入向量A i，得到聚类实例集合{O j}。针对任意聚类实例O j，确定其对应的肋骨标识为
Figure PCTCN2022077184-appb-000022
从而实现对粘连肋骨的解粘连，提高了肋骨标记的准确率。
图14示出本公开实施例提供的对图13所示的粘连肋骨进行解粘连后的示意图。图14中的用相同的颜色指示的粘连肋骨在图13中用不同的颜色指示，即实现了粘连肋骨的解粘连。
本公开提及的上述各个方法实施例，在不违背原理逻辑的情况下，均可以彼此相互结合形成结合后的实施例，限于篇幅。本领域技术人员可以理解，在具体实施方式的上述方法中，各步骤的具体执行顺序应当以其功能和可能的内在逻辑确定。
本公开实施例还提供了图像处理装置、电子设备、计算机可读存储介质、程序，上述均可用来实现本公开提供的任一种图像处理方法，相应技术方案和描述和参见方法部分的相应记载。
图15示出本公开实施例提供的一种图像处理装置的框图。如图15所示，本公开实施例提供的图像处理装置1500可以包括以下模块：
语义类别分割模块1501，配置为对待分割肋骨图像进行语义类别分割，得到待分割肋骨图像对应的语义类别分割结果；
实例分割模块1502，配置为对待分割肋骨图像进行实例分割，得到待分割肋骨图像对应的实例分割结果；
标记模块1503，配置为根据语义类别分割结果和实例分割结果，确定待分割肋骨图像对应的目标标记结果，其中，目标标记结果中包括待分割肋骨图像中的多根肋骨以及每根肋骨对应的肋骨标识。
在一种可能的实现方式中，本公开实施例提供的图像处理装置1500还包括：细二值分割模块，配置为在语义类别分割和实例分割是在第一分辨率下进行的情况下，在第二分辨率下对待分割肋骨图像进行二值分割，得到待分割肋骨图像对应的细二值分割结果，其中，第二分辨率大于所述第一分辨率；更新模块，配置为根据细二值分割结果，更新目标标记结果。
在一种可能的实现方式中，本公开实施例提供的图像处理装置1500还包括：获取模块，配置为获取原始胸部扫描图像；图像预处理模块，配置为对原始胸部扫描图像进行图像预处理，得到初始肋骨图像；凸包分割模块，配置为对初始肋骨图像进行凸包分割，得到初始肋骨图像中的肋骨凸包区域；裁剪模块，配置为根据肋骨凸包区域，对初始肋骨图像进行裁剪，得到待分割肋骨图像。
在一种可能的实现方式中，凸包分割模块，包括：第一确定子模块，配置为确定初始肋骨图像中的肋骨区域是否符合预设完整性要求；凸包分割子模块，配置为在初始肋骨图像中的肋骨区域符合预设完整性要求的情况下，对初始肋骨图像进行凸包分割，得到肋骨凸包区域。
在一种可能的实现方式中，初始肋骨图像包括多个切片图像；第一确定子模块，包括：第一确定单元，配置为确定多个切片图像中每个切片图像的肋骨类别，其中，不同肋骨类别配置为指示不同肋骨区域；第二确定单元，配置为根据每个切片图像的肋骨类别，确定肋骨类别序列；第三确定单元，配置为在肋骨类别序列中包括预设肋骨类别的情况下，确定初始肋骨图像中的肋骨区域符合预设完整性要求。
在一种可能的实现方式中，本公开实施例提供的实例分割模块1502，包括：重采样子模块，配置为对待分割肋骨图像进行重采样，得到第一肋骨图像，其中，第一肋骨图像的分辨率是第一分辨率；位置编码子模块，配置为对第一肋骨图像中的像素点进行位置编码，得到位置编码图像；实例分割子模块，配置为基于位置编码图像，对第一肋骨图像进行实例分割，得到实例分割结果。
在一种可能的实现方式中，本公开实施例提供的实例分割子模块，包括：二值分割单元，配置为对第一肋骨图像进行二值分割，得到第一肋骨图像对应的目标粗二值分割结果；第四确定单元，配置为基于位置编码图像，确定第一肋骨图像对应的像素点嵌入向量；第五确定单元，配置为根据目标粗二值分割结果和第一肋骨图像对应的像素点嵌入向量，确定第一肋骨图像中的肋骨区域对应的像素点嵌入向量；聚类单元，配置为对第一肋骨图像中的肋骨区域对应的像素点嵌入向量进行聚类，得到实例分割结果。
在一种可能的实现方式中，本公开实施例提供的二值分割单元，具体配置为：对第一肋骨图像进行二值分割，得到第一肋骨图像对应的初始粗二值分割结果；根据肋骨凸包区域，对初始粗二值分割 结果中的非肋骨区域进行过滤，得到目标粗二值分割结果。
在一种可能的实现方式中，本公开实施例提供的标记模块1503，包括：第二确定子模块，配置为根据实例分割结果和肋骨凸包区域，确定单侧肋骨序列，其中，单侧肋骨序列中包括多根单侧肋骨及其排序，单侧肋骨序列包括左侧肋骨序列和右侧肋骨序列；第三确定子模块，配置为根据语义类别分割结果和单侧肋骨序列，确定待分割肋骨图像中的多根肋骨以及每根肋骨对应的肋骨标识。
在一种可能的实现方式中，本公开实施例提供的第二确定子模块，包括：第六确定单元，配置为根据实例分割结果和肋骨凸包区域，确定第一单侧肋骨集合，其中，第一单侧肋骨序列中包括N根单侧肋骨，N是单侧肋骨的数目；第一排序单元，配置为对N根单侧肋骨在z轴方向进行排序，并取排序前K根肋骨构成第二单侧肋骨集合，其中，K是大于1且小于N的整数；平面拟合单元，配置为对第二单侧肋骨集合中的每根肋骨进行平面拟合，得到K个拟合平面；第二排序单元，配置为根据K个拟合平面之间的位置关系，对K根肋骨进行排序，得到单侧肋骨序列。
在一种可能的实现方式中，本公开实施例提供的图像处理装置1500还包括：第一确定模块，配置为根据K根肋骨中每根肋骨对应的拟合平面，确定每根肋骨对应的平面法向量和肋骨质心；第二确定模块，配置为针对K根肋骨中的任意两根肋骨R i和R j，根据肋骨R i对应的平面法向量和肋骨质心，以及肋骨R j对应的平面法向量和肋骨质心，确定肋骨R i和肋骨R j之间的内积，其中，肋骨R i和所述肋骨R j之间的内积的大小，配置为指示肋骨R i对应的拟合平面和肋骨R j对应的拟合平面之间的位置关系。
在一种可能的实现方式中，本公开实施例提供的第三确定子模块，具体配置为：根据语义类别分割结果，确定单侧肋骨序列中的目标肋骨对应的肋骨标识，其中，目标肋骨是单侧肋骨序列中的任意一根肋骨；根据目标肋骨对应的肋骨标识以及单侧肋骨序列，确定待分割肋骨图像中的多根肋骨以及每根肋骨对应的肋骨标识。
在一种可能的实现方式中，本公开实施例提供的更新模块，具体配置为：确定细二值分割结果中的多个连通域；根据多个连通域，更新目标标记结果。
在一种可能的实现方式中，本公开实施例提供的图像处理装置1500还包括：第三确定模块，配置为基于预设肋骨粘连检测条件，对目标标记结果进行粘连检测，确定目标标记结果中是否存在粘连肋骨，其中，粘连肋骨配置为指示对应相同的肋骨标识的至少两根相邻肋骨；第四确定模块，配置为在确定目标标记结果中存在粘连肋骨的情况下，确定粘连肋骨对应的像素点嵌入向量；聚类模块，配置为对粘连肋骨对应的像素点嵌入向量进行聚类，得到粘连肋骨解粘连后的每根肋骨对应的肋骨标识。
在一些实施例中，本公开实施例提供的装置具有的功能或包含的模块可以用于执行上文方法实施例描述的方法，其具体实现可以参照上文方法实施例的描述。
本公开实施例还提出一种计算机可读存储介质，其上存储有计算机程序指令，所述计算机程序指令被处理器执行时实现上述方法。计算机可读存储介质可以是非易失性计算机可读存储介质。
本公开实施例还提出一种电子设备，包括：处理器；用于存储处理器可执行指令的存储器；其中，所述处理器被配置为调用所述存储器存储的指令，以执行上述方法。
本公开实施例还提供了一种计算机程序产品，包括计算机可读代码，当计算机可读代码在设备上运行时，设备中的处理器执行用于实现如上任一实施例提供的图像处理方法的指令。
本公开实施例还提供了另一种计算机程序产品，用于存储计算机可读指令，指令被执行时使得计算机执行上述任一实施例提供的图像处理方法的操作。
电子设备可以被提供为终端、服务器或其它形态的设备。
图16示出本公开实施例提供的一种电子设备的框图。如图16所示，电子设备1600可以是移动电话，计算机，数字广播终端，消息收发设备，游戏控制台，平板设备，医疗设备，健身设备，个人数字助理等终端。
参照图16，电子设备1600可以包括以下一个或多个组件：处理组件1602，存储器1604，电源组件1606，多媒体组件1608，音频组件1610，输入/输出(I/O)的接口1612，传感器组件1614，以及通信组件1616。
处理组件1602通常控制电子设备1600的整体操作，诸如与显示，电话呼叫，数据通信，相机操作和记录操作相关联的操作。处理组件1602可以包括一个或多个处理器1620来执行指令，以完成上述的方法的全部或部分步骤。此外，处理组件1602可以包括一个或多个模块，便于处理组件1602和其他组件之间的交互。例如，处理组件1602可以包括多媒体模块，以方便多媒体组件1608和处理组件1602 之间的交互。
存储器1604被配置为存储各种类型的数据以支持在电子设备1600的操作。这些数据的示例包括用于在电子设备1600上操作的任何应用程序或方法的指令，联系人数据，电话簿数据，消息，图片，视频等。存储器1604可以由任何类型的易失性或非易失性存储设备或者它们的组合实现，如静态随机存取存储器(SRAM)，电可擦除可编程只读存储器(EEPROM)，可擦除可编程只读存储器(EPROM)，可编程只读存储器(PROM)，只读存储器(ROM)，磁存储器，快闪存储器，磁盘或光盘。
电源组件1606为电子设备1600的各种组件提供电力。电源组件1606可以包括电源管理系统，一个或多个电源，及其他与为电子设备1600生成、管理和分配电力相关联的组件。
多媒体组件1608包括在所述电子设备1600和用户之间的提供一个输出接口的屏幕。在一些实施例中，屏幕可以包括液晶显示器(LCD)和触摸面板(TP)。如果屏幕包括触摸面板，屏幕可以被实现为触摸屏，以接收来自用户的输入信号。触摸面板包括一个或多个触摸传感器以感测触摸、滑动和触摸面板上的手势。所述触摸传感器可以不仅感测触摸或滑动动作的边界，而且还检测与所述触摸或滑动操作相关的持续时间和压力。在一些实施例中，多媒体组件1608包括一个前置摄像头和/或后置摄像头。当电子设备1600处于操作模式，如拍摄模式或视频模式时，前置摄像头和/或后置摄像头可以接收外部的多媒体数据。每个前置摄像头和后置摄像头可以是一个固定的光学透镜系统或具有焦距和光学变焦能力。
音频组件1610被配置为输出和/或输入音频信号。例如，音频组件1610包括一个麦克风(MIC)，当电子设备1600处于操作模式，如呼叫模式、记录模式和语音识别模式时，麦克风被配置为接收外部音频信号。所接收的音频信号可以被进一步存储在存储器1604或经由通信组件1616发送。在一些实施例中，音频组件1610还包括一个扬声器，用于输出音频信号。
I/O接口1612为处理组件1602和外围接口模块之间提供接口，上述外围接口模块可以是键盘，点击轮，按钮等。这些按钮可包括但不限于：主页按钮、音量按钮、启动按钮和锁定按钮。
传感器组件1614包括一个或多个传感器，用于为电子设备1600提供各个方面的状态评估。例如，传感器组件1614可以检测到电子设备1600的打开/关闭状态，组件的相对定位，例如所述组件为电子设备1600的显示器和小键盘，传感器组件1614还可以检测电子设备1600或电子设备1600一个组件的位置改变，用户与电子设备1600接触的存在或不存在，电子设备1600方位或加速/减速和电子设备1600的温度变化。传感器组件1614可以包括接近传感器，被配置用来在没有任何的物理接触时检测附近物体的存在。传感器组件1614还可以包括光传感器，如互补金属氧化物半导体(CMOS)或电荷耦合装置(CCD)图像传感器，用于在成像应用中使用。在一些实施例中，该传感器组件1614还可以包括加速度传感器，陀螺仪传感器，磁传感器，压力传感器或温度传感器。
通信组件1616被配置为便于电子设备1600和其他设备之间有线或无线方式的通信。电子设备1600可以接入基于通信标准的无线网络，如无线网络(WiFi)，第二代移动通信技术(2G)或第三代移动通信技术(3G)，或它们的组合。在一个示例性实施例中，通信组件1616经由广播信道接收来自外部广播管理系统的广播信号或广播相关信息。在一个示例性实施例中，所述通信组件1616还包括近场通信(NFC)模块，以促进短程通信。例如，在NFC模块可基于射频识别(RFID)技术，红外数据协会(IrDA)技术，超宽带(UWB)技术，蓝牙(BT)技术和其他技术来实现。
在实际应用中，电子设备1600可以被一个或多个应用专用集成电路(ASIC)、数字信号处理器(DSP)、数字信号处理设备(DSPD)、可编程逻辑器件(PLD)、现场可编程门阵列(FPGA)、控制器、微控制器、微处理器或其他电子元件实现，用于执行上述方法。
本公开实施例提供了一种非易失性计算机可读存储介质，例如包括计算机程序指令的存储器1604，上述计算机程序指令可由电子设备1600的处理器1620执行以完成上述方法。
本公开实施例提供了一种计算机程序产品，包括计算机可读代码，或者承载有计算机可读代码的非易失性计算机可读存储介质，当所述计算机可读代码在电子设备的处理器中运行时，所述电子设备中的处理器执行上述方法。
图17示出本公开实施例提供的一种电子设备的框图。如图17所示，电子设备1700可以被提供为一服务器。参照图17，电子设备1700包括处理组件1722，其进一步包括一个或多个处理器，以及由存储器1732所代表的存储器资源，用于存储可由处理组件1722的执行的指令，例如应用程序。存储器1732中存储的应用程序可以包括一个或一个以上的每一个对应于一组指令的模块。此外，处理组件1722被配置为执行指令，以执行上述方法。
电子设备1700还可以包括一个电源组件1726被配置为执行电子设备1700的电源管理，一个有线或无线网络接口1750被配置为将电子设备1700连接到网络，和一个输入输出(I/O)接口1758。电子设 备1700可以操作基于存储在存储器1732的操作系统，例如微软服务器操作系统(Windows ServerTM)，苹果公司推出的基于图形用户界面操作系统(Mac OS XTM)，多用户多进程的计算机操作系统(UnixTM),自由和开放原代码的类Unix操作系统(LinuxTM)，开放原代码的类Unix操作系统(FreeBSDTM)或类似。
在示例性实施例中，还提供了一种非易失性计算机可读存储介质，例如包括计算机程序指令的存储器1732，上述计算机程序指令可由电子设备1700的处理组件1722执行以完成上述方法。
本公开可以是系统、方法和/或计算机程序产品。计算机程序产品可以包括计算机可读存储介质，其上载有用于使处理器实现本公开的各个方面的计算机可读程序指令。
计算机可读存储介质可以是可以保持和存储由指令执行设备使用的指令的有形设备。计算机可读存储介质例如可以是(但不限于)电存储设备、磁存储设备、光存储设备、电磁存储设备、半导体存储设备或者上述的任意合适的组合。计算机可读存储介质的更具体的例子(非穷举的列表)包括：便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、静态随机存取存储器(SRAM)、便携式压缩盘只读存储器(CD-ROM)、数字多功能盘(DVD)、记忆棒、软盘、机械编码设备、例如其上存储有指令的打孔卡或凹槽内凸起结构、以及上述的任意合适的组合。这里所使用的计算机可读存储介质不被解释为瞬时信号本身，诸如无线电波或者其他自由传播的电磁波、通过波导或其他传输媒介传播的电磁波(例如，通过光纤电缆的光脉冲)、或者通过电线传输的电信号。
这里所描述的计算机可读程序指令可以从计算机可读存储介质下载到各个计算/处理设备，或者通过网络、例如因特网、局域网、广域网和/或无线网下载到外部计算机或外部存储设备。网络可以包括铜传输电缆、光纤传输、无线传输、路由器、防火墙、交换机、网关计算机和/或边缘服务器。每个计算/处理设备中的网络适配卡或者网络接口从网络接收计算机可读程序指令，并转发该计算机可读程序指令，以供存储在各个计算/处理设备中的计算机可读存储介质中。
用于执行本公开操作的计算机程序指令可以是汇编指令、指令集架构(ISA)指令、机器指令、机器相关指令、微代码、固件指令、状态设置数据、或者以一种或多种编程语言的任意组合编写的源代码或目标代码，所述编程语言包括面向对象的编程语言—诸如Smalltalk、C++等，以及常规的过程式编程语言—诸如“C”语言或类似的编程语言。计算机可读程序指令可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络—包括局域网(LAN)或广域网(WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。在一些实施例中，通过利用计算机可读程序指令的状态信息来个性化定制电子电路，例如可编程逻辑电路、现场可编程门阵列(FPGA)或可编程逻辑阵列(PLA)，该电子电路可以执行计算机可读程序指令，从而实现本公开的各个方面。
这些计算机可读程序指令可以提供给通用计算机、专用计算机或其它可编程数据处理装置的处理器，从而生产出一种机器，使得这些指令在通过计算机或其它可编程数据处理装置的处理器执行时，产生了实现流程图和/或框图中的一个或多个方框中规定的功能/动作的装置。也可以把这些计算机可读程序指令存储在计算机可读存储介质中，这些指令使得计算机、可编程数据处理装置和/或其他设备以特定方式工作，从而，存储有指令的计算机可读介质则包括一个制造品，其包括实现流程图和/或框图中的一个或多个方框中规定的功能/动作的各个方面的指令。
附图中的流程图和框图显示了根据本公开的多个实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段或指令的一部分，所述模块、程序段或指令的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个连续的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或动作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
该计算机程序产品可以具体通过硬件、软件或其结合的方式实现。在一个可选实施例中，所述计算机程序产品具体体现为计算机存储介质，在另一个可选实施例中，计算机程序产品具体体现为软件产品，例如软件开发包(Software Development Kit，SDK)等等。
以上已经描述了本公开的各实施例，上述说明是示例性的，并非穷尽性的，并且也不限于所披露的各实施例。在不偏离所说明的各实施例的范围和精神的情况下，对于本技术领域的普通技术人员来 说许多修改和变更都是显而易见的。本文中所用术语的选择，旨在最好地解释各实施例的原理、实际应用或对市场中的技术的改进，或者使本技术领域的其它普通技术人员能理解本文披露的各实施例。
工业实用性
本公开实施例涉及一种图像处理方法及装置、电子设备和存储介质，所述方法包括：对待分割肋骨图像进行语义类别分割，得到待分割肋骨图像对应的语义类别分割结果；对待分割肋骨图像进行实例分割，得到待分割肋骨图像对应的实例分割结果；根据语义类别分割结果和实例分割结果，确定待分割肋骨图像对应的目标标记结果，其中，目标标记结果中包括待分割肋骨图像中的多根肋骨以及每根肋骨对应的肋骨标识。本公开实施例可以综合考虑图像全局语义信息和图像局部几何信息，提高肋骨的分割和标记精度。

Claims (18)

  1. 一种图像处理方法，包括：
    对待分割肋骨图像进行语义类别分割，得到所述待分割肋骨图像对应的语义类别分割结果；
    对所述待分割肋骨图像进行实例分割，得到所述待分割肋骨图像对应的实例分割结果；
    根据所述语义类别分割结果和所述实例分割结果，确定所述待分割肋骨图像对应的目标标记结果，其中，所述目标标记结果中包括所述待分割肋骨图像中的多根肋骨以及每根肋骨对应的肋骨标识。
  2. 根据权利要求1所述的方法，其中，所述方法还包括：
    在所述语义类别分割和所述实例分割是在第一分辨率下进行的情况下，在第二分辨率下对所述待分割肋骨图像进行二值分割，得到所述待分割肋骨图像对应的细二值分割结果，其中，所述第二分辨率大于所述第一分辨率；
    根据所述细二值分割结果，更新所述目标标记结果。
  3. 根据权利要求1或2所述的方法，其中，所述方法还包括：
    获取原始胸部扫描图像；
    对所述原始胸部扫描图像进行图像预处理，得到初始肋骨图像；
    对所述初始肋骨图像进行凸包分割，得到所述初始肋骨图像中的肋骨凸包区域；
    根据所述肋骨凸包区域，对所述初始肋骨图像进行裁剪，得到所述待分割肋骨图像。
  4. 根据权利要求3所述的方法，其中，所述对所述初始肋骨图像进行凸包分割，得到所述初始肋骨图像中的肋骨凸包区域，包括：
    确定所述初始肋骨图像中的肋骨区域是否符合预设完整性要求；
    在所述初始肋骨图像中的肋骨区域符合所述预设完整性要求的情况下，对所述初始肋骨图像进行凸包分割，得到所述肋骨凸包区域。
  5. 根据权利要求4所述的方法，其中，所述初始肋骨图像包括多个切片图像；
    所述确定所述初始肋骨图像中的肋骨区域是否符合预设完整性要求，包括：
    确定所述多个切片图像中每个切片图像的肋骨类别，其中，不同肋骨类别用于指示不同肋骨区域；
    根据每个所述切片图像的肋骨类别，确定肋骨类别序列；
    在所述肋骨类别序列中包括预设肋骨类别的情况下，确定所述初始肋骨图像中的肋骨区域符合所述预设完整性要求。
  6. 根据权利要求3至5中任意一项所述的方法，其中，所述对所述待分割肋骨图像进行实例分割，得到所述待分割肋骨图像对应的实例分割结果，包括：
    对所述待分割肋骨图像进行重采样，得到第一肋骨图像，其中，所述第一肋骨图像的分辨率是所述第一分辨率；
    对所述第一肋骨图像中的像素点进行位置编码，得到位置编码图像；
    基于所述位置编码图像，对所述第一肋骨图像进行实例分割，得到所述实例分割结果。
  7. 根据权利要求6所述的方法，其中，所述基于所述位置编码图像，对所述第一肋骨图像进行实例分割，得到所述实例分割结果，包括：
    对所述第一肋骨图像进行二值分割，得到所述第一肋骨图像对应的目标粗二值分割结果；
    基于所述位置编码图像，确定所述第一肋骨图像对应的像素点嵌入向量；
    根据所述目标粗二值分割结果和所述第一肋骨图像对应的像素点嵌入向量，确定所述第一肋骨图像中的肋骨区域对应的像素点嵌入向量；
    对所述第一肋骨图像中的肋骨区域对应的像素点嵌入向量进行聚类，得到所述实例分割结果。
  8. 根据权利要求7所述的方法，其中，所述对所述第一肋骨图像进行二值分割，得到所述第一肋骨图像对应的目标粗二值分割结果，包括：
    对所述第一肋骨图像进行二值分割，得到所述第一肋骨图像对应的初始粗二值分割结果；
    根据所述肋骨凸包区域，对所述初始粗二值分割结果中的非肋骨区域进行过滤，得到所述目标粗二值分割结果。
  9. 根据权利要求3至8中任意一项所述的方法，其中，所述根据所述语义类别分割结果和所述实例分割结果，确定所述待分割肋骨图像对应的目标标记结果，包括：
    根据所述实例分割结果和所述肋骨凸包区域，确定单侧肋骨序列，其中，所述单侧肋骨序列中包括多根单侧肋骨及其排序，所述单侧肋骨序列包括左侧肋骨序列和右侧肋骨序列；
    根据所述语义类别分割结果和所述单侧肋骨序列，确定所述待分割肋骨图像中的多根肋骨以及每根肋骨对应的肋骨标识。
  10. 根据权利要求9所述的方法，其中，所述根据所述实例分割结果和所述肋骨凸包区域，确定单侧肋骨序列，包括：
    根据所述实例分割结果和所述肋骨凸包区域，确定第一单侧肋骨集合，其中，所述第一单侧肋骨序列中包括N根单侧肋骨，N为单侧肋骨的数目；
    对所述N根单侧肋骨在z轴方向进行排序，并取排序前K根肋骨构成第二单侧肋骨集合，其中，K是大于1且小于N的整数；
    对所述第二单侧肋骨集合中的每根肋骨进行平面拟合，得到K个拟合平面；
    根据所述K个拟合平面之间的位置关系，对所述K根肋骨进行排序，得到所述单侧肋骨序列。
  11. 根据权利要求10所述的方法，其中，所述方法还包括：
    根据所述K根肋骨中每根肋骨对应的拟合平面，确定每根肋骨对应的平面法向量和肋骨质心；
    针对所述K根肋骨中的任意两根肋骨R i和R j，根据所述肋骨R i对应的平面法向量和肋骨质心，以及所述肋骨R j对应的平面法向量和肋骨质心，确定所述肋骨R i和所述肋骨R j之间的内积，其中，所述肋骨R i和所述肋骨R j之间的内积的大小，用于指示所述肋骨R i对应的拟合平面和所述肋骨R j对应的拟合平面之间的位置关系。
  12. 根据权利要求9至11中任意一项所述的方法，其中，所述根据所述语义类别分割结果和所述单侧肋骨序列，确定所述待分割肋骨图像中的多根肋骨以及每根肋骨对应的肋骨标识，包括：
    根据所述语义类别分割结果，确定所述单侧肋骨序列中的目标肋骨对应的肋骨标识，其中，所述目标肋骨是所述单侧肋骨序列中的任意一根肋骨；
    根据所述目标肋骨对应的肋骨标识以及所述单侧肋骨序列，确定所述待分割肋骨图像中的多根肋骨以及每根肋骨对应的肋骨标识。
  13. 根据权利要求2至12中任意一项所述的方法，其中，所述根据所述细二值分割结果，更新所述目标标记结果，包括：
    确定所述细二值分割结果中的多个连通域；
    根据所述多个连通域，更新所述目标标记结果。
  14. 根据权利要求1至13中任意一项所述的方法，其中，所述方法还包括：
    基于预设肋骨粘连检测条件，对所述目标标记结果进行粘连检测，确定所述目标标记结果中是否存在粘连肋骨，其中，所述粘连肋骨用于指示对应相同的肋骨标识的至少两根相邻肋骨；
    在确定所述目标标记结果中存在粘连肋骨的情况下，确定所述粘连肋骨对应的像素点嵌入向量；
    对所述粘连肋骨对应的像素点嵌入向量进行聚类，得到所述粘连肋骨解粘连后的每根肋骨对应的肋骨标识。
  15. 一种图像处理装置，包括：
    语义类别分割模块，配置为对待分割肋骨图像进行语义类别分割，得到所述待分割肋骨图像对应的语义类别分割结果；
    实例分割模块，配置为对所述待分割肋骨图像进行实例分割，得到所述待分割肋骨图像对应的实例分割结果；
    标记模块，配置为根据所述语义类别分割结果和所述实例分割结果，确定所述待分割肋骨图像对应的目标标记结果，其中，所述目标标记结果中包括所述待分割肋骨图像中的多根肋骨以及每根肋骨对应的肋骨标识。
  16. 一种电子设备，包括：
    处理器；
    用于存储处理器可执行指令的存储器；
    其中，所述处理器被配置为调用所述存储器存储的指令，以执行权利要求1至14中任意一项所述的方法。
  17. 一种计算机可读存储介质，其上存储有计算机程序指令，所述计算机程序指令被处理器执行时实现权利要求1至14中任意一项所述的方法。
  18. 一种计算机程序，包括计算机可读代码，当所述计算机可读代码在电子设备中运行时，所述电子设备中的处理器执行用于实现权利要求1至14中任一所述的方法。
PCT/CN2022/077184 2021-09-28 2022-02-22 图像处理方法、装置、电子设备、存储介质和程序 WO2023050690A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111145059.6 2021-09-28
CN202111145059.6A CN113947603A (zh) 2021-09-28 2021-09-28 图像处理方法及装置、电子设备和存储介质

Publications (1)

Publication Number Publication Date
WO2023050690A1 true WO2023050690A1 (zh) 2023-04-06

Family

ID=79328986

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/077184 WO2023050690A1 (zh) 2021-09-28 2022-02-22 图像处理方法、装置、电子设备、存储介质和程序

Country Status (2)

Country Link
CN (1) CN113947603A (zh)
WO (1) WO2023050690A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117455925A (zh) * 2023-12-26 2024-01-26 杭州健培科技有限公司 一种胸部多器官和肋骨分割方法及装置

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113947603A (zh) * 2021-09-28 2022-01-18 上海商汤智能科技有限公司 图像处理方法及装置、电子设备和存储介质

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112614134A (zh) * 2020-12-17 2021-04-06 北京迈格威科技有限公司 图像分割方法、装置、电子设备及存储介质
WO2021116011A1 (en) * 2019-12-10 2021-06-17 Koninklijke Philips N.V. Medical image segmentation and atlas image selection
CN113240696A (zh) * 2021-05-20 2021-08-10 推想医疗科技股份有限公司 图像处理方法及装置，模型的训练方法及装置，电子设备
CN113240681A (zh) * 2021-05-20 2021-08-10 推想医疗科技股份有限公司 图像处理的方法及装置
CN113255760A (zh) * 2021-05-20 2021-08-13 推想医疗科技股份有限公司 训练图像处理模型的方法、图像处理的方法及装置
CN113888548A (zh) * 2021-09-28 2022-01-04 上海商汤智能科技有限公司 图像处理方法及装置、电子设备和存储介质
CN113947603A (zh) * 2021-09-28 2022-01-18 上海商汤智能科技有限公司 图像处理方法及装置、电子设备和存储介质

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021116011A1 (en) * 2019-12-10 2021-06-17 Koninklijke Philips N.V. Medical image segmentation and atlas image selection
CN112614134A (zh) * 2020-12-17 2021-04-06 北京迈格威科技有限公司 图像分割方法、装置、电子设备及存储介质
CN113240696A (zh) * 2021-05-20 2021-08-10 推想医疗科技股份有限公司 图像处理方法及装置，模型的训练方法及装置，电子设备
CN113240681A (zh) * 2021-05-20 2021-08-10 推想医疗科技股份有限公司 图像处理的方法及装置
CN113255760A (zh) * 2021-05-20 2021-08-13 推想医疗科技股份有限公司 训练图像处理模型的方法、图像处理的方法及装置
CN113888548A (zh) * 2021-09-28 2022-01-04 上海商汤智能科技有限公司 图像处理方法及装置、电子设备和存储介质
CN113947603A (zh) * 2021-09-28 2022-01-18 上海商汤智能科技有限公司 图像处理方法及装置、电子设备和存储介质

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117455925A (zh) * 2023-12-26 2024-01-26 杭州健培科技有限公司 一种胸部多器官和肋骨分割方法及装置
CN117455925B (zh) * 2023-12-26 2024-05-17 杭州健培科技有限公司 一种胸部多器官和肋骨分割方法及装置

Also Published As

Publication number Publication date
CN113947603A (zh) 2022-01-18

Similar Documents

Publication Publication Date Title
US20220198775A1 (en) Image processing method and apparatus, electronic device, storage medium and computer program
CN110047078B (zh) 图像处理方法及装置、电子设备和存储介质
CN112767329B (zh) 图像处理方法及装置、电子设备
WO2022151755A1 (zh) 目标检测方法及装置、电子设备、存储介质、计算机程序产品和计算机程序
CN111899268B (zh) 图像分割方法及装置、电子设备和存储介质
CN111161301B (zh) 图像分割方法及装置、电子设备和存储介质
WO2023050690A1 (zh) 图像处理方法、装置、电子设备、存储介质和程序
WO2023050691A1 (zh) 图像处理方法及装置、电子设备、存储介质和程序
US20200327353A1 (en) Image processing method and apparatus, electronic device, and storage medium
US20210279473A1 (en) Video processing method and apparatus, electronic device, and storage medium
US11443438B2 (en) Network module and distribution method and apparatus, electronic device, and storage medium
CN113222038B (zh) 基于核磁图像的乳腺病灶分类和定位方法及装置
CN106557759B (zh) 一种标志牌信息获取方法及装置
CN109344703B (zh) 对象检测方法及装置、电子设备和存储介质
CN112508918A (zh) 图像处理方法及装置、电子设备和存储介质
US20220222831A1 (en) Method for processing images and electronic device therefor
CN112927239A (zh) 图像处理方法、装置、电子设备及存储介质
CN108171222B (zh) 一种基于多流神经网络的实时视频分类方法及装置
CN110909203A (zh) 视频分析方法及装置、电子设备和存储介质
CN113012146A (zh) 血管信息获取方法及装置、电子设备和存储介质
CN110852325B (zh) 图像的分割方法及装置、电子设备和存储介质
JP2022537865A (ja) 対象計数方法、装置、電子機器、記憶媒体及びプログラム
CN112906484A (zh) 一种视频帧处理方法及装置、电子设备和存储介质
CN112597944A (zh) 关键点检测方法及装置、电子设备和存储介质
CN111027617A (zh) 神经网络训练及图像识别方法、装置、设备和存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22874093

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE