WO2021238438A1

WO2021238438A1 - Tumor image processing method and apparatus, electronic device, and storage medium

Info

Publication number: WO2021238438A1
Application number: PCT/CN2021/086139
Authority: WO
Inventors: 王斯凡
Original assignee: 京东方科技集团股份有限公司
Priority date: 2020-05-29
Filing date: 2021-04-09
Publication date: 2021-12-02
Also published as: CN111640100B; CN111640100A

Abstract

A tumor image processing method and apparatus, an electronic device, and a storage medium. The method comprises: obtaining an original image by scanning a target organ (S101); performing rough recognition on the original image to obtain, from the original image, area data of an area where the target organ is located (S102); performing precise recognition on the area data to obtain first data of the target organ and second data of a tumor on the target organ (S103); performing, according to the first data and the second data, two-dimensional delineation and/or three-dimensional reconstruction on the target organ and the tumor on the target organ (S104), such that high-precision visualization results of the target organ and the tumor thereon can be obtained to enable doctors to more accurately determine the position, size, and shape of the target organ and the tumor on the target organ according to the processed identification image, thus facilitating diagnosis and treatment.

Description

[Name of invention formulated by ISA according to Rule 37.2] 　Tumor image processing method and device, electronic equipment, storage medium

Cross-references to related applications

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office, the application number is 202010474294.7, and the invention title is "Tumor Image Processing Method and Apparatus, Electronic Equipment, Storage Medium" on May 29, 2020. The entire content of the application is approved The reference is incorporated in this application.

Technical field

The present invention relates to the technical field of medical image processing, in particular to a method and device for processing tumor images, electronic equipment, and storage media.

Background technique

At present, doctors mainly use CT scans for tumor inspections. A CT scan produces at least DICOM data of more than 500M in size. Each case of DICOM data with more than 300 tomographic images requires doctors to check them one by one, which takes more than 30 minutes. If the diagnosis is made, the imaging surgeon needs to perform three-dimensional reconstruction to assist the clinical doctor in surgical planning, which usually takes more than 60 minutes. Doctors are labor intensive and inefficient. With the development of image processing technology, image processing is gradually applied to the processing of medical images to reduce the time it takes for doctors to check one by one.

Summary of the invention

The embodiment of the present disclosure proposes a method for processing a tumor image, which includes the following steps: obtaining an original image of a target organ; performing rough recognition on the original image to obtain regional data of the area where the target organ is located from the original image; Perform precise identification of the regional data, obtain the first data of the target organ and the second data of the tumor on the target organ; according to the first data and the second data, the target organ and The tumor on the target organ is delineated in two dimensions and/or reconstructed in three dimensions.

In some embodiments, the performing rough recognition on the original image to obtain the area data of the area where the target organ is located from the original image includes: inputting the original image into a U-shape with a three-layer step connection Network structure to obtain the first image; perform three-dimensional coordinate mapping on each position point in the first image to obtain the space coordinate of each position point; determine the area data of the target organ according to the space coordinate.

In some embodiments, the determining the region data of the target organ according to the spatial coordinates includes: performing a cluster analysis on the spatial coordinates to obtain the center of gravity position corresponding to each position point in the first image ; Obtain the first distance between each of the position points and the center of gravity position; identify the maximum value in the first distance, and use the maximum value as the shear radius; Clipping is performed at the center of gravity, so that the clipped area is used as the area data of the target organ.

In some embodiments, the identifying the maximum value in the first distance and using the maximum value as a shearing radius includes: obtaining a redundancy of the shearing radius, and combining the maximum value and the shearing radius. The sum of the redundancy amounts is used as the shear radius.

In some embodiments, the precise identification of the region data to obtain the first data of the target organ and the second data of the tumor on the target organ includes: obtaining a region image corresponding to the region data Input the regional image into a 3DU network structure with a sparse connection module and a multi-level residual module; use the 3DU network structure to finely identify the regional image, and obtain the 3DU network structure for precision Recognized second image; analyzing the second image to obtain the first data of the target organ and the second data of the tumor on the target organ.

In some embodiments, the acquiring the second image that has been finely identified through the 3DU-type network structure includes: using the 3DU-type network structure to up-sample the area data to acquire a first feature map; The first feature map is input to the sparse connection module to obtain a second feature map; the second feature map is input to the multi-level residual module to obtain a third feature map; the third feature map is performed Up-sampling to obtain the second image.

In some embodiments, the sparse connection module includes four cascaded branches.

In some embodiments, the multi-level residual module adopts three-scale pooling operations.

In some embodiments, the performing a two-dimensional delineation of the target organ and the tumor on the target organ according to the first data and the second data includes: obtaining a selection instruction for the target organ , Wherein the selection instruction includes a selection position of the target organ; according to the selection instruction, a target original image corresponding to the selected position is extracted from the original image; according to the first data and the The second data is a two-dimensional delineation of the target organ and the tumor on the target organ.

In some embodiments, the selection instruction further includes an image angle, and the extraction of the target original image corresponding to the selected position from the original image according to the selection instruction includes: according to the selection instruction, Extract a target original image corresponding to the selected position and conforming to the image angle from the original image.

According to the tumor image processing method proposed in this application, by recognizing the original image twice, firstly coarse and then finely, the target organ and its upper tumor can be obtained with higher precision, and the target organ and its upper tumor can be obtained in two dimensions. Delineation and/or three-dimensional reconstruction enable the doctor to more accurately determine the location, size and shape of the target organ and the tumor on it based on the identification image processed by this application, which is convenient for diagnosis and treatment.

The embodiment of the present disclosure proposes a tumor image processing device, which includes: an acquisition module for acquiring an original image taken for a target organ; a first recognition module for performing rough recognition on the original image to obtain information from the Obtain the area data of the area where the target organ is located from the original image; the second recognition module is used to accurately recognize the area data, and obtain the first data of the target organ and the second data of the tumor on the target organ; The identification module is configured to perform two-dimensional delineation and/or three-dimensional reconstruction of the target organ and the tumor on the target organ according to the first data and the second data.

The embodiment of the present disclosure proposes an electronic device including a memory, a processor, and a computer program stored on the memory and capable of running on the processor, and the processor implements the above-mentioned tumor image processing method when the program is executed.

The embodiment of the present disclosure proposes a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the above-mentioned tumor image processing method is realized.

Description of the drawings

The above-mentioned and/or additional aspects and advantages of the present invention will become obvious and easy to understand from the following description of the embodiments in conjunction with the accompanying drawings, in which:

FIG. 1 is a flowchart of a method for processing a tumor image provided by an embodiment of the present invention;

FIG. 2 is a flowchart of another tumor image processing method provided by an embodiment of the present invention;

FIG. 3 is a schematic structural diagram of a simplified Unet model provided by an embodiment of the present invention;

4 is a flowchart of another method for processing tumor images provided by an embodiment of the present invention;

FIG. 5 is a schematic diagram of the effect after the simplified Unet model processing provided by the embodiment of the present invention;

FIG. 6 is a flowchart of another method for processing a tumor image provided by an embodiment of the present invention;

FIG. 7 is a schematic structural diagram of a 3DU network structure with a sparse connection module and a multi-level residual module provided by an embodiment of the present invention;

FIG. 8 is a flowchart of another tumor image processing method provided by an embodiment of the present invention;

FIG. 9 is a schematic structural diagram of a sparse connection module provided by an embodiment of the present invention;

10 is a schematic structural diagram of a multi-level residual module provided by an embodiment of the present invention;

FIG. 11 is a schematic diagram of the principle of a training sample set according to an embodiment of the present invention;

FIG. 12 is a flowchart of yet another tumor image processing method provided by an embodiment of the present invention;

FIG. 13 is a schematic diagram of a specific two-dimensional delineation of a target organ and its tumor according to an embodiment of the present invention;

14 is another schematic diagram of specific two-dimensional delineation and three-dimensional reconstruction of a target organ and its tumor according to an embodiment of the present invention;

Fig. 15 is a schematic block diagram of a tumor image processing apparatus provided by an embodiment of the present invention.

Detailed ways

The embodiments of the present invention are described in detail below. Examples of the embodiments are shown in the accompanying drawings, in which the same or similar reference numerals indicate the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the drawings are exemplary, and are intended to explain the present invention, but should not be construed as limiting the present invention.

The following describes the tumor image processing method and device, electronic equipment, and storage medium according to the embodiments of the present invention with reference to the accompanying drawings.

FIG. 1 is a schematic flowchart of a method for processing a tumor image provided by an embodiment of the present invention. As shown in Fig. 1, the method for processing a tumor image according to an embodiment of the present invention includes the following steps:

S101: Obtain an original image of the target organ.

Among them, the original image is, for example, a CT (Computed Tomography) image taken of a patient’s target organ. A CT scan generates at least DICOM (Digital Imaging and Communications in Medicine) data with a size of more than 500M. Each case has more than 300 tomographic images.

S102: Perform rough recognition on the original image to obtain area data of the area where the target organ is located from the original image.

S103: Perform precise recognition on the area data, and obtain the first data of the target organ and the second data of the tumor on the target organ.

Among them, the first data may include information such as the location and size of the target organ, and the second information may include information such as the location, size, and shape of the tumor.

S104: Perform two-dimensional delineation and/or three-dimensional reconstruction on the target organ and the tumor on the target organ according to the first data and the second data.

Therefore, the tumor image processing method proposed in this application can obtain the target organ and its tumor with higher accuracy by recognizing the original image twice, firstly coarse and then finely. Performing two-dimensional delineation and/or three-dimensional reconstruction enables the doctor to more accurately determine the location, size, and shape of the target organ and the tumor on it according to the identification image processed by this application, which is convenient for diagnosis and treatment.

According to another embodiment of the invention, as shown in FIG. 2, performing rough recognition on the original image to obtain the area data of the area where the target organ is located from the original image includes:

S201: Input the original image into a U-shaped network structure with three-layer step connections to obtain a first image.

It should be noted that the U-shaped structure network in the example of this application is, for example, a simplified Unet model, where the Unet model has the characteristics of simple structure and close combination of deep and shallow semantics. This application optimizes the Unet model, as shown in Figure 3. Removing the fourth-layer connection in the traditional Unet model structure and adjusting it to a three-layer step connection can reduce the amount of parameters by 1/2 of the model and effectively improve the operating efficiency.

It should be noted that due to the use of CT scan images for full image recognition, large errors will occur, especially for kidney tumor recognition. There will be a lot of background space outside the kidneys, which not only increases the amount of calculation for image processing, but also affects The accuracy of the recognition result. Therefore, this application further processes the images obtained through the U-shaped network structure to determine the area data of the area where the target organ is located, that is, the input data for subsequent precision recognition is the resection irrelevant information and only contains the area where the target organ is located. Data, effectively reducing the amount of data processing for precise identification. Among them, taking the kidney as an example, the input data for fine recognition can be regional data that only includes the left kidney and the right kidney.

S202: Perform three-dimensional coordinate mapping on each position point in the first image to obtain the space coordinate of each position point.

S203: Determine the area data of the target organ according to the space coordinates.

Specifically, as shown in Fig. 4, determining the area data of the target organ according to the spatial coordinates includes:

S301: Perform cluster analysis on the spatial coordinates, and obtain the center of gravity position corresponding to each position point in the first image.

S302: Acquire the first distance between each position point and the center of gravity position.

S303: Identify the maximum value in the first distance, and use the maximum value as the shear radius.

S304: Cut the position of the center of gravity according to the cutting radius, so that the cut area is used as the area data of the target organ.

It should be understood that when the target organ is the kidney, since a person has both kidneys, there can be two positions of the center of gravity. Cut out the regional data of the two target organs, where the clustering effect can be shown in Figure 5.

Further, in order to avoid the shearing error, the elasticity scale, that is, the redundancy of the shearing radius, can be increased, where the redundancy can be The distance between the center of gravity and this point.

It should be understood that since a case of CT data contains a target organ part such as the kidney part, which only accounts for 10% of the overall data, the addition of a large amount of irrelevant information will cause the model to overfit and make the identification of the kidney and the tumor more difficult. Therefore, through this The rough recognition and removal of irrelevant information proposed by the application can improve the purity of data and make the recognition effect of the trained model better.

Therefore, the present application can extract the region data of the target organ through rough recognition, so as to reduce the amount of data processing for the subsequent fine recognition process, and effectively improve the accuracy and operation speed of image processing.

According to another embodiment of the present application, as shown in FIG. 6, performing precise recognition on the area data to generate a target organ image containing tumor data includes:

S401: Acquire an area image corresponding to the area data.

Among them, the regional data can be data blocks containing lesions and tumor information obtained through rough recognition. Since the 3DU network is an image processing network, the data blocks need to be pictured for precise recognition.

S402: Input the regional image into a 3DU network structure with a sparse connection module and a multi-level residual module.

It should be noted that the 3DU network structure with sparse connection module (S module) and multi-level residual module (P module) proposed in this application can be expressed as the SP-3DUnet model, that is, the sparse connection module is added to the 3DUnet model (S module) and a multi-level residual module (P module). Specifically, this application adds a sparse connection module (S module) and a multi-level residual module (P module) to the bottom layer of the 3DUnet model, that is, the 3DUnet model After the down-sampling and before the up-sampling.

Furthermore, this application also optimizes the 3DUnet model, that is, each convolutional layer removes a convolution operation on the basis of the original model to reduce 1/2 parameters and reduce the model volume. However, this operation will reduce the model accordingly. The ability to extract the deep semantics of images. Therefore, this application further adds a sparse connection module (S module) and a multi-level residual module (P module) to the bottom layer of the optimized 3DU model, as shown in Figure 7, to resist semantics Loss of expression ability, this operation can improve the accuracy of algorithm recognition and operating efficiency under the premise of ensuring that only 1/5 of the parameter amount is increased.

S403: Use the 3DU network structure to perform fine recognition on the regional image, and obtain a second image that has been finely recognized by the 3DU network structure.

As a specific embodiment, as shown in Figs. 8-10, step S403 acquiring a second image that has been finely recognized through a 3DU-type network structure includes:

S4031: Up-sampling the regional data using the 3DU network structure to obtain the first feature map.

S4032: Input the first feature map into the sparse connection module to obtain a second feature map.

It should be noted that this application proposes a sparse connection based on hole convolution to encode high-level semantic feature mapping, where the hole convolution is stacked in a cascaded manner. Specifically, in the embodiment of the present application, the sparse connection module is divided into four cascaded branches.

As a specific embodiment, as shown in FIG. 9, due to the requirements of the subsequent 3D model for output data and the limitation of the actual GPU (Graphics Processing Unit, graphics processor) hardware performance of this application, 1*1*1 and 3* can be used. 3*3 two kinds of hollow convolution, corresponding to the receptive field of each branch are 3*3*3, 7*7*7 and 9*9*9.

It should be understood that when the sparse connection module is used, the convolution of the large receptive field can extract and generate more abstract features for the large target, and the convolution of the small receptive field can extract and generate more abstract features for the small target. In this application, by combining hole convolutions with different expansion rates, the sparse connection module can extract features with various size targets, establish a sparse connection mode of underlying key semantic features, and offset the semantic information description loss caused by reducing the convolution operation.

S4033: Input the second feature map to the multi-level residual module to obtain a third feature map.

It should be noted that due to the different tumor sizes on the target organs, the tumor size in advanced patients may exceed 2/3 of the target organ volume. When the tumor volume is larger or smaller than the size covered by the current data set, the performance of the algorithm may decrease, so , This application proposes to use a multi-level residual module to enhance the algorithm's ability to recognize and generalize multi-size targets. The multi-level residual module uses pooling operations of different scales to extract feature information of different scales and improve the ability to recognize targets of different sizes.

As a specific embodiment, as shown in FIG. 10, a three-scale pooling operation is adopted, and the three branches output three-scale feature domains. In order to reduce the dimensionality and computational cost of weights, 1*1*1 convolution can also be used after each pooling branch, which can reduce the size of the feature domain to 1/N of the original size, where N represents the original feature domain The number of channels in.

S4034: Up-sampling the third feature map to obtain a second image.

Among them, the third feature map reduced to 1/N of the original size is subjected to bilinear interpolation to obtain the same size feature as the original feature map.

S404: Analyze the second image to obtain the first data including the target organ and the second data including the tumor on the target organ.

That is to say, the image after the precise recognition of the image through the 3DU network structure with the sparse connection module and the multi-level residual module is a characteristic image, which needs to be analyzed to obtain the first data containing the target organ and the target organ The second data on the tumor. For example, the position of the target organ can be marked as 1, the position of the tumor on the target organ is marked as 2, and the other positions are marked as 0, so that the position set of the target organ is the first data of the target organ, which can express the target The position and size of the organ, similarly, the position of the tumor on the target organ is set as the second data of the tumor on the target organ.

It should be noted that the simplified U-shaped network structure and the 3DU-type network structure with sparse connection modules and multi-level residual modules used in the above-mentioned rough recognition and fine recognition need to be trained in deep learning, so that they can be used accurately. Identify the first data of the target organ and the second data of the tumor on the target organ.

Furthermore, in the training process, multiple original images that have been manually marked with target organs and tumor locations can be input as training sample sets. Among them, in order to further make the training results more accurate, it can also be used to manually mark the training samples The training sample set is expanded by inversion, translation, rotation, deformation, etc.

This application uses the Dice coefficient loss function, where the Dice coefficient is specifically expressed by the following formula:

Among them, N is the label of the number of pixels, p(k,i)∈[0,1] and g(k,i)∈{0,1} represent the predicted probability and true label of class k, respectively, p(k,i ) Is the pre-stored probability of each pixel in the parsed target organ image, and k is the category and the weight, which can be set in the embodiment of the present application.

The loss function using the Dice coefficient can be: Lloss=Ldice+Lreg, where Lreg represents a regularization loss term used to avoid overfitting (also called weight attenuation). .

It should also be understood that when deep learning is required for the simplified U-shaped network structure and the 3DU-type network structure with sparse connection modules and multi-level residual modules, the accuracy of image recognition can also be verified by the doctor’s hand examination. If there is a recognition error, use the pen function to correct it, and replace the original recognition result with the correction result. For example, the correction result is automatically sent back to the cloud training database to update the training sample set to retrain the model, as shown in Figure 11.

As another feasible embodiment, as shown in FIG. 12, the two-dimensional delineation of the target organ and the tumor on the target organ according to the first data and the second data includes:

S501: Obtain a selection instruction for a target organ.

Among them, the selection instruction includes the selection position of the target organ.

S502: According to the selection instruction, extract the target original image corresponding to the selected position from the original image.

In some embodiments, the selection instruction may further include an image angle, and therefore, a target original image corresponding to the selected position and conforming to the image angle needs to be extracted from the original image.

S503: Perform a two-dimensional delineation of the target organ and the tumor on the target organ on the original image according to the first data and the second data.

In other words, after recognizing the original image of the target organ, the target organ and the tumor on the target organ can be outlined in two dimensions on the original image based on the first data and the second data, so that doctors and patients can clearly know the condition of the lesion . At the same time, it can help doctors focus on identifying pictures containing kidneys when performing medical diagnosis based on CT images, effectively improving the efficiency of diagnosis and treatment, achieving the purpose of precise diagnosis and treatment, and preventing missed diagnosis. Furthermore, the present application can effectively improve the visualization of doctor-patient communication by performing three-dimensional reconstruction of the target organ and its tumor.

Specifically, the target organ selection instruction can be obtained, and the position and angle selected by the doctor and/or patient can be extracted from the selection instruction, and the tomographic image corresponding to the position and angle can be extracted from the original image according to the position and angle, and then the image can be judged Whether each position point in the data belongs to the first data and/or the second data, if it is, it is identified according to the first data and/or the second data, and if it is not, no operation is performed.

For example, as shown in FIG. 13, the left image is a tomographic image of the kidney scanned from a bird's-eye view angle when the human body is upright. Specifically, the image is the image corresponding to the 286th layer of the 520 tomographic images, and the right image is The image after marking the kidney and kidney tumor in the left image. Further, the image angle may also include the side view angle and the back view angle when the human body is upright, as shown in FIG. 14, where the image in the upper left corner in FIG. The image in the corner is a tomographic image of the human body upright from the top angle, the image in the lower right corner is the tomographic image of the human body upright in the rear view angle, and the image in the lower left corner is the three-dimensional model of the target organ and its upper tumor.

Or, after the first data and the second data are obtained by fine recognition, the target organ and the tumor on the target organ are 3D modeled according to the first data and the second data to obtain a 3D model of the target organ and the tumor on the target organ, such as As shown in Figure 14, the corresponding identification data is extracted from the three-dimensional model according to the selection instruction and a two-dimensional outline is performed on the original image.

It should be understood that, as can be seen from the display in the lower left corner of FIG. 14, the embodiment of the present application can also perform three-dimensional reconstruction of the target organ and the tumor on the target organ according to the first data and the second data, and directly reconstruct the three-dimensionally reconstructed tumor. Stereoscopic images are displayed.

It should also be understood that when displaying through the display terminal, the multiple images in Figure 13 to Figure 14 above can be combined to facilitate doctors and/or patients through the correspondence between multiple images displayed at the same time. That is, the shape and size of the same target organ location and/or the same tumor location at different angles.

Further, based on the identified three-dimensional reconstruction model of the target organ and tumor area, the doctor can accurately obtain the position and shape of the tumor at the target organ through the model result, so that it is convenient for the doctor to perform medical planning, such as surgery, based on the three-dimensional reconstruction model. Planning, radiotherapy planning and chemotherapy planning, etc.

For example, during surgical planning, the tumor position determined by the method proposed in this application can be used to help doctors find precise positioning, reduce operation time, and improve the quality of the operation. Moreover, due to the accurate positioning, the dislocation of the surgical incision can be avoided, thereby reducing The size of the surgical knife edge accelerates the preset speed of the patient’s wound and reduces the patient’s pain; during radiotherapy planning, the tumor size and location can be determined by the method proposed in this application to help doctors plan the intensity of radiotherapy rays, thereby reducing the impact of radiotherapy on normal parts During chemotherapy planning, the location, size, and shape of the tumor determined by the method proposed in this application can help doctors plan the dosage of the drug, reduce the impact on the patient’s normal cells, and thereby alleviate the patient’s pain in treatment.

In summary, the tumor image processing method proposed in the present application recognizes the original image twice, firstly coarse and then finely, so as to obtain high-precision target organs and tumors on the target organs. The tumor is delineated in two dimensions and/or reconstructed in three dimensions, so that the doctor can more accurately determine the position, size, and shape of the target organ according to the target organ image processed by the application, and facilitate diagnosis and treatment.

In order to implement the above embodiments, the present invention also provides a tumor image processing device.

FIG. 15 is a schematic block diagram of a tumor image processing apparatus provided by an embodiment of the present invention. As shown in FIG. 15, the apparatus 10 for processing tumor images includes: an acquisition module 11, a first identification module 12, a second identification module 13 and an identification module 14.

Wherein, the acquisition module 11 is used to acquire the original image taken for the target organ.

The first recognition module 12 is configured to perform rough recognition on the original image to obtain the area data of the area where the target organ is located from the original image.

The second recognition module 13 is configured to perform precise recognition on the area data, and obtain the first data of the target organ and the second data of the tumor on the target organ.

The identification module 14 is configured to perform two-dimensional delineation and three-dimensional reconstruction of the target organ and the tumor on the target organ according to the first data and the second data.

Further, the first recognition module 12 is specifically configured to: input the original image into a U-shaped network structure with three layers of step connections to obtain a first image; Coordinate mapping to obtain the space coordinates of each position point; and determine the area data of the target organ according to the space coordinates.

Further, the first identification module 12 is specifically configured to: perform a cluster analysis on the spatial coordinates to obtain the position of the center of gravity corresponding to the position point in the first image; and obtain the position of the center of gravity and each position point The first distance between the positions; identify the maximum value in the first distance, and use the maximum value as the shear radius; shear the center of gravity position according to the shear radius, so that after the shear The area of is used as the area data of the target organ.

Further, the first identification module 12 is specifically configured to: obtain the redundancy of the shearing radius, and use the sum of the maximum value and the redundancy as the shearing radius.

Further, the second recognition module 13 is specifically configured to: obtain the area image corresponding to the area data; input the area image into a 3DU network structure with a sparse connection module and a multi-level residual module; use the 3DU The type network structure performs precise recognition on the regional image, and obtains a second image that has been refined through the 3DU type network structure; and analyzes the second image to generate the target organ image.

Further, the second identification module 13 is specifically configured to: use the 3DU network structure to up-sample the area data to obtain a first feature map; and input the first feature map into the sparse connection module to Obtain a second feature map; input the second feature map to the multi-level residual module to obtain a third feature map; perform up-sampling on the third feature map to obtain the second image.

Further, the sparse connection module includes four cascaded branches.

Further, the multi-level residual module adopts three-scale pooling operations.

Further, the identification module 14 is further configured to: obtain a selection instruction for the target organ, wherein the selection instruction includes a selection position of the target organ; and extract from the original image according to the selection instruction The target original image corresponding to the selected position; according to the first data and the second data, two-dimensional delineation and three-dimensional reconstruction of the target organ and the tumor on the target organ are performed.

Further, the identification module 14 is further configured to: according to the selection instruction, extract a target original image corresponding to the selected position and conforming to the image angle from the original image.

It should be noted that the foregoing explanation of the embodiment of the tumor image processing method is also applicable to the tumor image processing device of this embodiment, and will not be repeated here.

Based on the foregoing embodiments, the embodiments of the present invention also provide an electronic device, including a memory, a processor, and a computer program stored on the memory and running on the processor. When the processor executes the program, the foregoing The processing method of the tumor image.

In order to implement the above-mentioned embodiments, the present invention also provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the aforementioned tumor image processing method is realized.

In the description of this specification, descriptions with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" etc. mean specific features described in conjunction with the embodiment or example , Structures, materials or features are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above-mentioned terms do not necessarily refer to the same embodiment or example. Moreover, the described specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine the different embodiments or examples and the features of the different embodiments or examples described in this specification without contradicting each other.

In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise specifically defined.

Any process or method description in the flowchart or described in other ways herein can be understood as a module, segment or part of code that includes one or more executable instructions for implementing custom logic functions or steps of the process , And the scope of the preferred embodiments of the present invention includes additional implementations, which may not be in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order according to the functions involved. This should It is understood by those skilled in the art to which the embodiments of the present invention belong.

The logic and/or steps represented in the flowchart or described in other ways herein, for example, can be considered as a sequenced list of executable instructions for realizing logic functions, and can be embodied in any computer-readable medium, For use by instruction execution systems, devices, or equipment (such as computer-based systems, systems including processors, or other systems that can fetch instructions from and execute instructions from instruction execution systems, devices, or equipment), or combine these instruction execution systems, devices Or equipment. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transmit a program for use by an instruction execution system, device, or device or in combination with these instruction execution systems, devices, or devices. More specific examples (non-exhaustive list) of computer-readable media include the following: electrical connections (electronic devices) with one or more wiring, portable computer disk cases (magnetic devices), random access memory (RAM), Read only memory (ROM), erasable and editable read only memory (EPROM or flash memory), fiber optic devices, and portable compact disk read only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program can be printed, because it can be used, for example, by optically scanning the paper or other medium, and then editing, interpreting, or other suitable media if necessary. The program is processed in a way to obtain the program electronically and then stored in the computer memory.

It should be understood that each part of the present invention can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if it is implemented by hardware as in another embodiment, it can be implemented by any one or a combination of the following technologies known in the art: Discrete logic gate circuits for implementing logic functions on data signals Logic circuits, application specific integrated circuits with suitable combinational logic gates, programmable gate array (PGA), field programmable gate array (FPGA), etc.

Those of ordinary skill in the art can understand that all or part of the steps carried in the method of the foregoing embodiments can be implemented by a program instructing relevant hardware to complete. The program can be stored in a computer-readable storage medium. When executed, it includes one of the steps of the method embodiment or a combination thereof.

In addition, the functional units in the various embodiments of the present invention may be integrated into one processing module, or each unit may exist alone physically, or two or more units may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or software function modules. If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it may also be stored in a computer readable storage medium.

The above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present invention have been shown and described above, it can be understood that the above-mentioned embodiments are exemplary and should not be construed as limiting the present invention. Those of ordinary skill in the art can comment on the above-mentioned embodiments within the scope of the present invention. The embodiment undergoes changes, modifications, substitutions, and modifications.

Claims

A method for processing tumor images, which is characterized in that it comprises the following steps:

Obtain the original image of the target organ;

Performing rough recognition on the original image to obtain area data of the area where the target organ is located from the original image;

Performing precise recognition on the region data, and acquiring the first data of the target organ and the second data of the tumor on the target organ;

According to the first data and the second data, two-dimensional delineation and/or three-dimensional reconstruction are performed on the target organ and the tumor on the target organ.
The method for processing a tumor image according to claim 1, wherein the performing rough recognition on the original image to obtain the area data of the area where the target organ is located from the original image comprises:

Inputting the original image into a U-shaped network structure with three-layer step connections to obtain a first image;

Performing three-dimensional coordinate mapping on each position point in the first image to obtain the space coordinate of each position point;

The area data of the target organ is determined according to the space coordinates.
The tumor image processing method according to claim 2, wherein the determining the area data of the target organ according to the spatial coordinates comprises:

Performing cluster analysis on the spatial coordinates to obtain the center of gravity position corresponding to each position point in the first image;

Acquiring the first distance between each of the position points and the position of the center of gravity;

Identifying the maximum value in the first distance, and using the maximum value as the shear radius;

The position of the center of gravity is cut according to the cutting radius, so that the cut area is used as the area data of the target organ.
The tumor image processing method according to claim 3, wherein the identifying the maximum value in the first distance and using the maximum value as a cutting radius comprises:

Obtain the redundancy of the shearing radius, and use the sum of the maximum value and the redundancy as the shearing radius.
The method for processing a tumor image according to claim 1, wherein the precise recognition of the region data is performed to obtain the first data of the target organ and the second data of the tumor on the target organ, include:

Acquiring a region image corresponding to the region data;

Input the region image into a 3DU network structure with a sparse connection module and a multi-level residual module;

Using the 3DU-type network structure to perform fine recognition on the regional image, and obtain a second image that has been finely recognized through the 3DU-type network structure;

The second image is analyzed to obtain the first data including the target organ and the second data including the tumor on the target organ.
The method for processing tumor images according to claim 5, wherein the 3DU network structure is used to perform fine recognition on the area image, and the second image that has been finely recognized through the 3DU network structure is obtained. Two images, including:

Up-sampling the regional data by using the 3DU-type network structure to obtain a first feature map;

Input the first feature map to the sparse connection module to obtain a second feature map;

Input the second feature map to the multi-level residual module to obtain a third feature map;

Up-sampling the third feature map to obtain the second image.
The tumor image processing method according to claim 5 or 6, wherein the sparse connection module includes four cascaded branches.
The tumor image processing method according to claim 5 or 6, wherein the multi-level residual module adopts three-scale pooling operations.
The method for processing a tumor image according to claim 1, wherein the two-dimensional delineation of the target organ and the tumor on the target organ is performed according to the first data and the second data, include:

Acquiring a selection instruction for the target organ, wherein the selection instruction includes a selection position for the target organ;

Extracting a target original image corresponding to the selected position from the original image according to the selection instruction;

According to the first data and the second data, the target organ and the tumor on the target organ are two-dimensionally delineated on the original image.
The method for processing a tumor image according to claim 9, wherein the selection instruction further includes an image angle, and the target original image corresponding to the selected position is extracted from the original image according to the selection instruction. Images, including:

According to the selection instruction, a target original image corresponding to the selected position and conforming to the image angle is extracted from the original image.
A tumor image processing device, which is characterized in that it comprises:

The acquisition module is used to acquire the original image scanned for the target organ;

The first recognition module is configured to perform rough recognition on the original image to obtain the area data of the area where the target organ is located from the original image;

The second recognition module is used to perform precise recognition on the regional data, and obtain the first data of the target organ and the second data of the tumor on the target organ;

The identification module is used to perform two-dimensional delineation and three-dimensional reconstruction of the target organ and the tumor on the target organ according to the first data and the second data.
An electronic device, characterized in that it comprises a memory, a processor, and a computer program stored on the memory and running on the processor, wherein the processor executes the program to achieve the following: 10. The tumor image processing method described in any one of 10.
A computer-readable storage medium having a computer program stored thereon, wherein the program is executed by a processor to realize the tumor image processing method according to any one of claims 1-10.