JP7061225B2 - Focal detection methods, devices, equipment and storage media - Google Patents

Description

The present invention relates to the field of computer technology, and in particular to methods, devices, devices and storage media for lesion detection.

What is Computer Aided Diagosis (CAD)? Automatically automate lesions from images by combining visual, medical image analysis techniques and other possible physiological and biochemical means with computer-aided analysis calculations. It means to discover the target. It has been proved by practice that computer-aided diagnosis has a great positive promotion effect in terms of improving the accuracy of diagnosis, reducing omission of diagnosis, and improving the work efficiency of doctors. Here, the lesion refers to a part of a tissue or organ having a lesion due to the influence of a virulence factor, and is a part of a living body having a lesion. For example, if a part of the lungs of the human body is destroyed by Mycobacterium tuberculosis, this part becomes a lesion of pulmonary tuberculosis.

In recent years, with the rapid development of computer vision and deep learning techniques, CT image-based lesion detection methods are receiving increasing attention.

The present disclosure provides methods, devices, devices and storage media for foci detection, which can accurately detect foci status at multiple sites in a patient and preliminarily evaluate cancer throughout the patient.

On the first side, the present disclosure comprises acquiring a first image comprising a plurality of sample sections, which is a three-dimensional image including the X-axis dimension, the Y-axis dimension, and the Z-axis dimension. The features of the first image are extracted, and a first feature map including the features and positions of the lesion, which includes the three-dimensional features of the X-axis dimension, the Y-axis dimension, and the Z-axis dimension, is generated. That is, the dimension of the feature included in the first feature map is reduced to generate a second feature map including the two-dimensional features of the X-axis dimension and the Y-axis dimension, and the second feature map is detected. The present invention provides a method for detecting a lesion, including the position of each lesion in the second feature map and the reliability corresponding to the position.

In some possible embodiments, obtaining a first image containing multiple sample sections on one side resamples the captured patient CT images at the first sampling interval and multiple sample sections. Includes generating a first image containing.

In some possible embodiments, on one side, extracting the features of the first image and generating a first feature map containing the features and location of the lesion is the first by the first neural network. The image is downsampled to generate a third feature map, the third feature map is downsampled by the residual block of the second neural network to generate a fourth feature map, and the second neural network is generated. After extracting the features of lesions of different scales in the fourth feature map by the DenseASPP module of the network and processing by the DenseASPP module, a fourth predetermined feature map having the same resolution as the fourth feature map is generated, and the fourth predetermined feature map is generated. The feature map after processing by the DenseASPP module is upsampled by the inverse convolution layer of the two neural networks and the residual block to generate a third predetermined feature map having the same resolution as the third feature map, and the third The feature map and the third predetermined feature map generate a first feature map having the same resolution as the third predetermined feature map, and the fourth predetermined feature map and the fourth predetermined feature map are fused to generate the fourth predetermined feature. Including generating a first feature map with the same resolution as the map, the third predetermined feature map and the fourth predetermined feature map each include the location of the lesion, and the location of the lesion is the lesion in the first feature map. Used to generate the position of.

In some possible embodiments, on one side, extracting the features of the first image and generating a first feature map containing the features and locations of the lesions is a residual block of the second neural network. The first image is downsampled to generate a fourth feature map with a lower resolution than the first image, and the DenseASPP module of the second neural network features different scales of lesions in the fourth feature map. And after processing by the DenseASPP module, the feature map after processing by the DenseASPP module is upsampled by the inverse convolution layer of the second neural network and the residual block, and has the same resolution as the first image. The first includes generating the first predetermined feature map and generating a first feature map having the same resolution as the first predetermined feature map with the first image and the first predetermined feature map. The predetermined feature map includes the location of the lesion, and the location of the lesion is used to generate the location of the lesion in the first feature map.

In some possible embodiments, on one side, extracting the features of the first image and generating a first feature map containing the features and location of the lesion is the first by the first neural network. The image is downsampled to generate a third feature map having a lower resolution than the first image, and the third feature map is downsampled by the residual block of the second neural network to generate the third feature map. A fourth feature map with a lower resolution than the fourth feature map is generated, and the fourth feature map is downsampled by the residual block of the second neural network to obtain a fifth feature map with a lower resolution than the fourth feature map. The fifth feature map has the same resolution as the fifth feature map after being generated, extracting the features of lesions of different scales in the fifth feature map by the DenseASPP module of the second neural network, and processing by the DenseASPP module. A predetermined feature map is generated, and the feature map processed by the DenseASPP module is upsampled by the inverse convolution layer of the second neural network and the residual block, and the fourth predetermined feature map having the same resolution as the fourth feature map is generated. Or upsample the feature map processed by the DenseASPP module with the inverse convolution layer and residual block of the second neural network to generate a third predetermined feature map with the same resolution as the third feature map. The third feature map and the third predetermined feature map generate a first feature map having the same resolution as the third predetermined feature map, and the fourth feature map and the fourth predetermined feature map are fused. Then, a first feature map having the same resolution as the fourth predetermined feature map is generated, and the fifth feature map and the fifth predetermined feature map are fused to generate a first feature having the same resolution as the fifth predetermined feature map. The third predetermined feature map, the fourth predetermined feature map, and the fifth predetermined feature map each include the position of the lesion, and the position of the lesion is the position of the lesion in the first feature map. Used to generate a position.

In some possible embodiments, on one side, the first neural network comprises a convolution layer and a residual block cascaded to the convolution layer, the second neural network is a convolution layer, a deconvolution. Includes a 3D U-Net network containing layers, residual blocks and the DenseASPP module.

In some possible embodiments, on one side, the second neural network is a stack of 3D U-Net networks.

In some possible embodiments, on one side, the residual block comprises a convolutional layer, a batch normalization layer, a ReLU activation function and a maximal pooling layer.

In some possible embodiments, it is possible to reduce the dimensions of the features contained in the first feature map on one side and generate a second feature map, respectively, of all the features of the first feature map. By merging the channel dimension and the Z-axis dimension with respect to, each dimension of all the features of the first feature map consists of the X-axis dimension and the Y-axis dimension, and each dimension of all the features is The first feature map including the X-axis dimension and the Y-axis dimension is included in the second feature map.

In some possible embodiments, detecting the second feature map on one side detects the second feature map by the first detection subnet and the location of each lesion in the second feature map. It includes obtaining the coordinates and detecting the second feature map by the second detection subnet to obtain the reliability corresponding to each lesion in the second feature map.

In some possible embodiments, on one side, the first detection subnet comprises a plurality of convolution layers, each connected to one ReLU activation function, and the second detection subnet is one each. Includes multiple convolution layers connected with one ReLU activation function.

In some possible embodiments, on one side, the pre-stored lesions are labeled before extracting the features of the first image and generating a first feature map containing the features and locations of the lesions. A three-dimensional image containing a plurality of lesion labels is input to the first neural network, and the first neural network, the second neural network, the Dense ASPP module, and the first one are used by the gradient descent method. It further includes training each parameter of the detection subnet and the second detection subnet, respectively, and the position of each of the plurality of lesions is output by the first detection subnet.

In some possible embodiments, on one side, the pre-stored lesions are labeled before extracting the features of the first image and generating a first feature map containing the features and locations of the lesions. A three-dimensional image containing a plurality of lesion labels is input to the first neural network, and the second neural network, the Dense ASPP module, the first detection subnet, and the second are used by using the gradient descent method. The training of each parameter of the detection subnet is further included, and the position of each of the plurality of lesions is output by the first detection subnet.

In the second direction, the present disclosure is an acquisition unit for acquiring a first image including a plurality of sample sections, which is a three-dimensional image including an X-axis dimension, a Y-axis dimension, and a Z-axis dimension. It is a first feature map that extracts the features of the first image and includes the features and positions of the lesion, and is a first feature map that includes the three-dimensional features of the X-axis dimension, the Y-axis dimension, and the Z-axis dimension. To generate a second feature map containing two-dimensional features of the X-axis dimension and the Y-axis dimension by reducing the dimensions of the features included in the first feature map and the first generation unit for generating the above. Provided is a lesion detection device including a second generation unit, a detection unit for detecting the second feature map, and obtaining the position of each lesion in the second feature map and the reliability corresponding to the position.

In some possible embodiments, in the second direction, the acquisition unit specifically resamples the acquired CT image of the patient at the first sampling interval and produces the first image containing multiple sample sections. Used to generate.

In some possible embodiments, in the second direction, the first generation unit specifically downsamples the first image by a first neural network and has a lower resolution than the first image. To generate a feature map, to downsample the third feature map by the residual block of the second neural network, and to generate a fourth feature map having a lower resolution than the third feature map, and to generate the fourth feature map. After extracting the features of lesions of different scales in the fourth feature map by the DenseASPP module of the two neural networks and processing by the DenseASPP module, a fourth predetermined feature map having the same resolution as the fourth feature map is generated. The feature map after processing by the DenseASPP module is upsampled by the inverse convolution layer of the second neural network and the residual block to generate a third predetermined feature map having the same resolution as the third feature map. The third feature map and the third predetermined feature map generate a first feature map having the same resolution as the third predetermined feature map, and the fourth feature map and the fourth predetermined feature map are fused to generate the fourth feature map. Used to generate a first feature map with the same resolution as the predetermined feature map, the third predetermined feature map and the fourth predetermined feature map each include the location of the lesion, and the location of the lesion is the first feature map. Used to generate the location of lesions in.

In some possible embodiments, in the second direction, the first generation unit specifically downsamples the first image by a first neural network and has a lower resolution than the first image. A feature map is generated, features of lesions of different scales in the fourth feature map are extracted by the DenseASPP module of the second neural network, and after processing by the DenseASPP module, the reverse convolution of the second neural network is performed. The layer and the residual block upsample the feature map processed by the DenseASPP module to generate the first predetermined feature map having the same resolution as the first image, and the first image and the first predetermined. Used to generate a first feature map with the same resolution as the first predetermined feature map in the feature map, the first predetermined feature map includes the location of the lesion, and the location of the lesion is the lesion in the first feature map. Used to generate the position of.

In some possible embodiments, in the second direction, the first generation unit specifically downsamples the residual block of the first image by a second neural network and has a higher resolution than the first image. To generate a third feature map with a lower resolution, and to downsample the third feature map by the residual block of the second neural network to generate a fourth feature map with a lower resolution than the third feature map. Then, the fourth feature map is downsampled by the residual block of the second neural network to generate a fifth feature map having a lower resolution than the fourth feature map, and the DenseASPP module of the second neural network. After extracting the features of lesions of different scales in the fifth feature map and processing by the DenseASPP module, a fifth predetermined feature map having the same resolution as the fifth feature map is generated, and the second neural network The reverse convolution layer and the residual block upsample the feature map processed by the DenseASPP module to generate a fourth predetermined feature map with the same resolution as the fourth feature map, or the inverse of the second neural network. The feature map after processing by the DenseASPP module is upsampled by the convolution layer and the residual block to generate a third predetermined feature map having the same resolution as the third feature map, and the third feature map and the third feature map. A first feature map having the same resolution as the third predetermined feature map is generated from the predetermined feature map, and the fourth feature map and the fourth predetermined feature map are fused to form a first feature map having the same resolution as the fourth predetermined feature map. One feature map is generated, and the fifth feature map and the fifth predetermined feature map are fused to generate a first feature map having the same resolution as the fifth predetermined feature map. The predetermined feature map, the fourth predetermined feature map, and the fifth predetermined feature map each include the position of the lesion, and the position of the lesion is used to generate the position of the lesion in the first feature map.

In some possible embodiments, in the second direction, the first neural network comprises a convolution layer and a residual block cascaded to the convolution layer, the second neural network is a convolution layer, a deconvolution. Includes a 3D U-Net network containing layers, residual blocks and the DenseASPP module.

In some possible embodiments, in the second direction, the second neural network is a stack of 3D U-Net networks.

In some possible embodiments, in the second direction, the residual block comprises a convolutional layer, a batch normalization layer, a ReLU activation function and a maximal pooling layer.

In some possible embodiments, in the second direction, the third feature unit specifically merges the channel dimension and the Z-axis dimension for each of all the features in the first feature map. , A first feature map in which each dimension of all features of the first feature map consists of X-axis dimensions and Y-axis dimensions, and each dimension of all the features consists of X-axis dimensions and Y-axis dimensions. It is used to make the second feature map.

In some possible embodiments, in the second direction, the detection unit specifically detects the second feature map by the first detection subnet and coordinates the position of each lesion in the second feature map. It is used to obtain and to detect the second feature map by the second detection subnet and to obtain the reliability corresponding to each lesion in the second feature map.

In some possible embodiments, in the second direction, the first detection subnet comprises a plurality of convolution layers, each connected to one ReLU activation function, and the second detection subnet is one each. Includes multiple convolution layers connected with one ReLU activation function.

In some possible embodiments, in the second direction, the training unit further comprises, in particular, the first generation unit extracting the features of the first image and including the features of the lesion. Before generating the first feature map, a pre-stored three-dimensional image containing multiple lesion labels for labeling the lesion is input to the first neural network and the gradient descent method is used. Used to train the parameters of the first neural network, the second neural network, the first detection subnet and the second detection subnet, respectively, and the position of each of the plurality of lesions is determined by the first detection subnet. It is output.

In some possible embodiments, in the second direction, the training unit further comprises, in particular, the first generation unit extracting the features of the first image, the features and location of the lesion. Before generating the first feature map containing the above, a three-dimensional image containing a plurality of lesion labels for labeling the lesion is input to the second neural network, and the second It is used to train the parameters of the neural network, the first detection subnet and the second detection subnet, respectively, and the position of each of the plurality of lesions is output by the first detection subnet.

In the third direction, the present disclosure is a lesion detection device including a processor, a display and a memory connected to each other, wherein the display is used to display the location of the lesion and the reliability corresponding to the location. The memory is used to store the application program code, and the processor provides a lesion detection device configured to execute the one-sided lesion detection method by calling the program code.

In the fourth aspect, the present disclosure is a computer-readable storage medium for storing one or more computer programs including instructions, and when the computer program is executed on a computer, the instructions are on the first side. Provided is a computer-readable storage medium used to carry out a lesion detection method.

In the fifth aspect, the present disclosure is a computer program including a lesion detection instruction, and when the computer program is executed in a computer, the lesion detection instruction causes the lesion detection method provided by the first aspect to be executed. Provides computer programs used in.

The present disclosure provides lesion detection methods, devices, devices and storage media. First, a first image including a plurality of sample sections and a three-dimensional image including an X-axis dimension, a Y-axis dimension, and a Z-axis dimension is acquired. Furthermore, the features of the first image are extracted, and a first feature map including the features and positions of the lesion, which includes three-dimensional features of the X-axis dimension, the Y-axis dimension, and the Z-axis dimension, is generated. do. Subsequently, the dimension reduction of the features included in the first feature map is performed, and the second feature map including the two-dimensional features of the X-axis dimension and the Y-axis dimension is generated. Finally, the features of the second feature map are detected and the reliability corresponding to the features and positions of each lesion in the second feature map is obtained. By adopting the present disclosure, it is possible to accurately detect the lesion status of a plurality of sites in a patient and to preliminarily evaluate cancer throughout the patient.

Hereinafter, in order to more clearly explain the means for solving the embodiments of the present disclosure, the drawings necessary for explaining the embodiments will be briefly described. It is clear that the drawings below are some embodiments of the present disclosure and that one of ordinary skill in the art can conceive other drawings based on these drawings without any creative effort.
FIG. 1 is a schematic diagram of the network architecture of the lesion detection system provided in the present disclosure. FIG. 2 is a schematic flowchart of the lesion detection method provided in the present disclosure. FIG. 3 is a schematic block diagram of the lesion detection device provided by the present disclosure. FIG. 4 is a schematic configuration diagram of the lesion detection device provided by the present disclosure.

Hereinafter, the technical solutions in the present disclosure will be clearly and completely described with reference to the drawings in the present disclosure. The examples described are only a part of the examples of the present disclosure and are not all examples. All other examples obtained by those skilled in the art based on the examples in the present disclosure without the need for creative effort are all within the scope of the present disclosure.

As used herein and in the appended claims, the terms "included" and "included" mean that the features, whole, steps, operations, elements and / or components described are present. It should be understood that it does not exclude the existence or addition of one or more other features, whole, steps, operations, elements, components and / or groups thereof.

It should also be understood that the terms used herein are merely intended to describe a particular embodiment and are not intended to limit this disclosure. It is to be understood that the singular forms "one", "one" and "the" as used in the specification of the present disclosure and the appended claims also include the plural, unless the context clearly indicates.

As used in the specification and the appended claims of the present disclosure, the terms "and / or" mean any combination of one or more of the items listed in connection and all possible combinations. It should also be understood that these combinations are included.

The term "case" as used herein and in the appended claims is referred to as "... hour" or "... then" or "as determined / specified / determined" or "as detected", depending on the context. Can be interpreted as meaning. Similarly, the phrase "when determined / specified / determined" or "when [stated condition or event] is detected" is "determined / specified / determined" or "determined / specified / determined" depending on the context. Can be interpreted as "when" or "when [explained condition or event] is detected" or "in response to the detection of [explained condition or event]".

In a specific implementation, the devices described in the present disclosure include, but include, other portable devices such as laptop computers or tablet computers having touch sensitive surfaces (eg, touch screen displays and / or touch panels). Not limited. It should also be appreciated that in some embodiments, the device is not a portable communication device, but a desktop computer having a touch sensing surface (eg, a touch screen display and / or a touch panel).

Hereinafter, the device including the display and the touch sensing surface will be described. It should be understood that the device may include one or more other physical user interface devices such as a physical keyboard, mouse and / or joystick.

Devices include various applications such as drawing applications, demo applications, word processing applications, website creation applications, disk authoring applications, spreadsheet applications, gaming applications, phone applications, video conferencing applications, email applications, instant messaging applications, etc. Supports one or more of training support applications, photo management applications, digital camera applications, digital video camera applications, web browsing applications, digital music player applications and / or digital video player applications.

Various applications that can be run on the device may use at least one common physical user interface device, such as a touch sensitive surface. One or more functions of the touch-sensitive surface and the corresponding information displayed on the device may be coordinated and / or changed between and / or within the corresponding application. In this way, the device's common physical architecture (eg, touch-sensitive surface) can support a variety of applications with user-intuitive and transparent user interfaces.

In order to understand the present disclosure more clearly, the network architecture suitable for the present disclosure will be described below. FIG. 1 is a schematic diagram of a lesion detection system provided by the present disclosure. As shown in FIG. 1, the system 10 may include a first neural network 101, a second neural network 102, and a detection subnet 103.

In the examples of the present disclosure, a lesion refers to a part of a tissue or organ having a lesion due to the influence of a virulence factor, and is a part of a living body having a lesion. For example, if a part of the lungs of the human body is destroyed by Mycobacterium tuberculosis, this part becomes a lesion of pulmonary tuberculosis.

I would like to explain that the first neural network 101 includes a convolution layer (Conv1) and a residual block (SEResBlock) cascaded to the convolution layer. Among them, the residual block may include a batch normalization layer (Batch Normalization, BN), a modified linear unit (ReLU) activation function and a maximum pooling layer (Max-polling).

Among them, the first neural network 101 can be used to downsample the first image input to the first neural network 101 in the X-axis dimension and the Y-axis dimension to generate a third feature map. I would like to explain that the first image is a three-dimensional image including the X-axis dimension, the Y-axis dimension, and the Z-axis dimension (that is, the first image is composed of a plurality of two-dimensional images including the X-axis dimension and the Y-axis dimension. It is a three-dimensional image including the X-axis dimension, the Y-axis dimension, and the Z-axis dimension). For example, the first image may be a 512 * 512 * 9 three-dimensional image.

Specifically, the first neural network 101 processes the first image by the convolution kernel of the convolution layer to generate a feature map, and further pools a specific feature map by the residual block to better than the first image. A third feature map with low resolution may be generated. As an example, the 3D image of 512 * 512 * 9 may be processed by the first neural network 101 to obtain the 3D image of 256 * 256 * 9, or the 3D image of 512 * 512 * 9 may be obtained by the first neural network 101. A 3D image may be processed to obtain a 128 * 128 * 9 3D image. By downsampling, it is possible to extract the lesion features contained in the input first image and remove a part of the unnecessary region in the first image.

It is intended to explain that downsampling in the embodiments of the present disclosure is used to generate thumbnails of the first image to fit the size of the display area. The upsampling in the embodiments of the present disclosure is used to insert new pixels between the pixels of the original image by interpolation to enlarge the original image. This is useful for detecting small lesions.

Hereinafter, downsampling in the examples of the present disclosure will be briefly described with reference to an example. For example, an image I having a size of M * N can be downsampled by S times to obtain an image having a resolution of (M / S) * (N / S). That is, the image in the S * S window of the original image I is regarded as one pixel, and the pixel value of the pixel is the maximum value of all the pixels in the S * S window. Here, the stride that slides in the horizontal direction or the vertical direction may be 2.

The second neural network 102 may include four stacked 3D U-net networks. A development of the 3D U-net network is shown in FIG. 104. By detecting in a plurality of 3D U-net networks, the accuracy of detection can be improved. The number of 3D U-net networks in the examples of the present disclosure is exemplary only and is not limited. Here, the 3D U-Net network includes a convolution layer (conv), a deconvolution layer (deconv), a residual block and a DenseASPP module.

Among them, the residual block of the second neural network 102 is used to downsample the third feature map output from the first neural network 101 in the X-axis dimension and the Y-axis dimension to generate the fourth feature map. Can be done.

The residual block of the second neural network 102 can also be used to downsample the fourth feature map in the X-axis dimension and the Y-axis dimension to generate the fifth feature map.

Subsequently, the DenseASPP module of the second neural network 102 extracts the features of the lesions of different scales in the fifth feature map.

After processing by the DenseASPP module, a fifth predetermined feature map having the same resolution as the fifth feature map is generated, and the feature map after processing by the DenseASPP module is uploaded by the deconvolution layer of the second neural network 102 and the residual block. Sampling to generate a fourth predetermined feature map with the same resolution as the fourth feature map, or upsampling the feature map processed by the DenseASPP module with the deconvolution layer and residual block of the second neural network 102. Then, a third predetermined feature map having the same resolution as the third feature map is generated.

A first feature map with the same resolution as the third predetermined feature map is generated by concating the third feature map and the third predetermined feature map, and the fourth feature map and the fourth predetermined feature map are fused. A first feature map having the same resolution as the fourth predetermined feature map is generated, and the fifth feature map and the fifth predetermined feature map are fused to generate a first feature map having the same resolution as the fifth predetermined feature map. Here, the third predetermined feature map, the fourth predetermined feature map, and the fifth predetermined feature map each include the position of the lesion, and the position of the lesion is used to generate the position of the lesion in the first feature map. Be done.

I would like to explain that the DenseASPP module contains 5 different expansion convolutions cascaded and can extract features of lesions of different scales. Here, the expansion convolutions of the five different expansion rates (dilate) are the expansion convolution of the expansion rate d = 3, the expansion convolution of the expansion rate d = 6, the expansion convolution of the expansion rate d = 12, and the expansion rate d = 18, respectively. Extended convolution and extended convolution with an expansion rate d = 24.

The detection subnet 103 may include a first detection subnet and a second detection subnet. The first detection subnet contains a plurality of convolution layers, each connected to one ReLU activation function. Similarly, the second detection subnet contains a plurality of convolution layers, each connected to one ReLU activation function.

The first detection subnet is used to detect the second feature map with the first feature map dimensionally reduced and to obtain the coordinates of the position of each lesion in the second feature map.

Specifically, the input second feature map is processed by the four convolutional layers cascaded in the first detection subnet to identify the location of each lesion in the second feature map. Here, each convolution layer contains one Y * Y convolution kernel, and by acquiring the coordinates (x1, y1) of the upper left corner of each lesion and the coordinates (x2, y2) of the lower right corner of the lesion in order, The location of each lesion in the second feature map may be specified.

The second feature map is detected by the second detection subnet, and the reliability corresponding to each lesion in the second feature map is obtained.

Specifically, the input second feature map is processed by the four convolutional layers cascaded in the second detection subnet, the position of each lesion in the second feature map is identified, and further, at that position. Output the corresponding reliability. Here, each convolution layer contains one Y * Y convolution kernel, and by acquiring the coordinates (x1, y1) of the upper left corner of each lesion and the coordinates (x2, y2) of the lower right corner of the lesion in order, The position of each lesion in the second feature map may be specified, and the reliability corresponding to the position may be output.

It is intended to explain that in the embodiments of the present disclosure, the reliability corresponding to a position indicates the degree to which the user believes that the position is a lesion. For example, the reliability of the location of a lesion is 90%.

According to the above description, it is possible to accurately detect the lesion status of a plurality of sites in the patient and to preliminarily evaluate the cancer throughout the patient.

What I would like to explain is to further include the following steps before extracting the features of the first image and generating a first feature map containing the features and locations of the lesions.

The first three-dimensional image containing a plurality of pre-stored lesion labels for labeling the lesion (for example, labeling the lesion in the form of a box and labeling at the coordinates of the position of the lesion). One input to the neural network and the gradient descent method is used to train the parameters of the first neural network, the second neural network, the first detection subnet and the second detection subnet, respectively. Here, the position of each of the plurality of lesions is output by the first detection subnet.
I would like to explain that in the training of each parameter by the gradient descent method, the gradient descent method may be calculated by the back propagation algorithm.

Alternatively, a pre-stored 3D image containing multiple lesion labels for labeling the lesion can be input into the second neural network and the gradient descent method can be used to enter the second neural network, the first detection subnet and Train each parameter of the second detection subnet individually. Here, the position of each of the plurality of lesions is output by the first detection subnet.

FIG. 2 is a schematic flowchart of the lesion detection method provided in the present disclosure. In one possible embodiment, the lesion detection method is a user-side device (User Equipment, UE), a mobile device, a user terminal, a terminal, a cordless telephone, a personal digital assistant (PDA), a hand-held device. , A terminal device such as a computing device, an in-vehicle device, a wearable device, or an electronic device such as a server. The method may be realized by calling a computer-readable command stored in memory by the processor. Alternatively, the method may be performed by the server.

As shown in FIG. 2, the method may include at least the following steps.

S201: A first image including a plurality of sample sections, which is a three-dimensional image including an X-axis dimension, a Y-axis dimension, and a Z-axis dimension, is acquired.

Specifically, in one selectable embodiment, the acquired CT images of the patient are resampled at the first sampling interval to generate a first image containing a plurality of sample sections. Here, even if the CT image of the patient contains 130 tomographic faults in which the thickness of each layer is 2.0 mm, and the first sampling interval in the X-axis dimension and the Y-axis dimension is 2.0 mm. good.

In the embodiments of the present disclosure, the CT image of the patient is a single scan sequence containing multiple toms with respect to the patient's tissue or organ, and the number of toms may be 130.

A lesion is a part of a tissue or organ of a patient who has a lesion due to the influence of a virulence factor, and is a part of a living body having a lesion. For example, if a part of the lungs of the human body is destroyed by Mycobacterium tuberculosis, this part becomes a lesion of pulmonary tuberculosis.

I would like to explain that the first image is a three-dimensional image including the X-axis dimension, the Y-axis dimension, and the Z-axis dimension (that is, the first image is composed of N two-dimensional images including the X-axis dimension and the Y-axis dimension. It is a three-dimensional image that is configured and includes the X-axis dimension, the Y-axis dimension, and the Z-axis dimension, where N is 2 or more, and each two-dimensional image is a cross-sectional image at a different position of the detected tissue). For example, the first image may be a 512 * 512 * 9 three-dimensional image.

I would like to explain that before resampling the CT image, it may further include a step of removing the extra background in the CT image by the threshold method.

S202: The features of the first image are extracted, the features of the lesion are included, and the first feature map including the three-dimensional features of the X-axis dimension, the Y-axis dimension, and the Z-axis dimension is generated.

Specifically, extracting the features of the first image and generating a first feature map containing the features and locations of the lesions may include, but is not limited to, the following cases.

Case 1: The first image is downsampled by the first neural network to generate a third feature map.

The third feature map is downsampled by the residual block of the second neural network to generate the fourth feature map.

The DenseASPP module of the second neural network extracts the features of lesions of different scales in the fourth feature map.

After processing by the DenseASPP module, a fourth predetermined feature map with the same resolution as the fourth feature map is generated, and the feature map processed by the DenseASPP module is upsampled by the deconvolution layer and residual block of the second neural network. (3) Generate a third predetermined feature map with the same resolution as the feature map.

The third feature map and the third predetermined feature map generate a first feature map with the same resolution as the third predetermined feature map, and the fourth feature map and the fourth predetermined feature map are fused to be the same as the fourth predetermined feature map. Generate a first feature map of resolution. Here, the third predetermined feature map and the fourth predetermined feature map each include the position of the lesion, and the position of the lesion is used to generate the position of the lesion in the first feature map.

Case 2: The first image is downsampled by the residual block of the second neural network to generate the fourth feature map.

The DenseASPP module of the second neural network extracts the features of lesions of different scales in the fourth feature map.

After processing by the DenseASPP module, the feature map processed by the DenseASPP module is upsampled by the deconvolution layer and the residual block of the second neural network, and the first predetermined feature map having the same resolution as the first image is generated.

The first image and the first predetermined feature map generate a first feature map having the same resolution as the first predetermined feature map. Here, the first predetermined feature map includes the location of the lesion, and the location of the lesion is used to generate the location of the lesion in the first feature map.

Case 3: The first image is downsampled by the first neural network to generate a third feature map.

The third feature map is downsampled by the residual block of the second neural network to generate the fourth feature map.

The fourth feature map is downsampled by the residual block of the second neural network to generate the fifth feature map.

The DenseASPP module of the second neural network extracts the features of lesions of different scales in the fifth feature map.

After processing by the DenseASPP module, a fifth predetermined feature map with the same resolution as the fifth feature map is generated, and the feature map processed by the DenseASPP module is upsampled by the deconvolution layer and residual block of the second neural network. Generate a fourth predetermined feature map with the same resolution as the four feature map, or upsample the feature map processed by the DenseASPP module by the deconvolution layer and residual block of the second neural network, and be the same as the third feature map. Generate a third predetermined feature map of resolution.

The third feature map and the third predetermined feature map generate a first feature map with the same resolution as the third predetermined feature map, and the fourth feature map and the fourth predetermined feature map are fused to be the same as the fourth predetermined feature map. A first feature map with a resolution is generated, and a first feature map with the same resolution as the fifth predetermined feature map is generated by fusing the fifth feature map and the fifth predetermined feature map. Here, the third predetermined feature map, the fourth predetermined feature map, and the fifth predetermined feature map each include the position of the lesion, and the position of the lesion is used to generate the position of the lesion in the first feature map. Be done.

I would like to explain that the first neural network contains a convolution layer and a residual block cascaded to the convolution layer, and the second neural network is a 3DU containing a convolution layer, a deconvolution layer, a residual block and a DenseASP P module. -Includes Net network.

Among them, the residual block may include a convolution layer, a batch normalization layer (BN layer), a ReLU activation function and a maximum pooling layer.

Optionally, the second neural network is a stack of 3D U-Net networks. When a plurality of 3D U-Net networks in which the second neural network is stacked can be used, the stability and detection accuracy of the lesion detection system can be improved. The embodiments of the present disclosure do not limit the number of 3D U-net networks.

S203: The dimension of the feature included in the first feature map is reduced, and the second feature map including the two-dimensional features of the X-axis dimension and the Y-axis dimension is generated.

Specifically, the channel dimension and the Z-axis dimension are merged for each of the features of the first feature map, and the respective dimensions of all the features of the first feature map are set to the X-axis dimension and the Y-axis dimension. The first feature map in which each dimension of all features consists of the X-axis dimension and the Y-axis dimension is defined as the second feature map. The first feature map is a three-dimensional feature map, but when it is output to the detection subnet 103 and detected, it needs to be converted into two dimensions, so the dimension of the first feature map must be reduced.

I would like to explain that the channel of a certain feature described above represents the distribution data of a certain feature.

S204: The feature of the second feature map is detected, and the reliability corresponding to the feature and the position of each lesion in the detected second feature map is displayed.

Specifically, the second feature map is detected by the first detection subnet, and the coordinates of the position of each lesion in the second feature map are obtained.

More specifically, the input second feature map is processed by a plurality of cascaded convolution layers of the first detection subnet to identify the location of each lesion in the second feature map. Here, each convolution layer contains one Y * Y convolution kernel, and by acquiring the coordinates (x1, y1) of the upper left corner of each lesion and the coordinates (x2, y2) of the lower right corner of the lesion in order, The location of each lesion in the second feature map may be specified.

The second feature map is detected by the second detection subnet, and the reliability corresponding to each lesion in the second feature map is obtained.

More specifically, the input second feature map is processed by a plurality of cascaded convolution layers of the second detection subnet to obtain the position of each lesion in the second feature map, and further, the position. Outputs the reliability corresponding to. Here, each convolution layer contains one Y * Y convolution kernel, and by acquiring the coordinates (x1, y1) of the upper left corner of each lesion and the coordinates (x2, y2) of the lower right corner of the lesion in order, The position of each lesion in the second feature map may be acquired, and the reliability corresponding to the position may be output.

According to the above description, the examples of the present disclosure can accurately detect the foci status of a plurality of sites in a patient and preliminarily evaluate cancer throughout the patient.

What I would like to explain is to further include the following steps before extracting the features of the first image and generating the first feature map containing the features of the lesion.

Input a pre-stored 3D image containing multiple lesion labels for labeling into the first neural network and use the gradient descent method to enter the first neural network, the second neural network, and the first. Train each parameter of the discovery subnet and the second discovery subnet respectively. Here, the position of each of the plurality of lesions is output by the first detection subnet.

Alternatively, input a 3D image containing multiple lesion labels for labeling the lesion into the second neural network and use gradient descent to use the second neural network, first detection subnet, and second detection subnet. Train each parameter individually. Here, the position of each of the plurality of lesions is output by the first detection subnet.

Summarizing the above, in the present disclosure, first, a first image including a plurality of sample sections, which is a three-dimensional image including an X-axis dimension, a Y-axis dimension, and a Z-axis dimension, is acquired. Further, the features of the first image are extracted to generate a first feature map including the features of the lesion, which includes three-dimensional features of the X-axis dimension, the Y-axis dimension, and the Z-axis dimension. Subsequently, the dimension reduction of the features included in the first feature map is performed, and the second feature map including the two-dimensional features of the X-axis dimension and the Y-axis dimension is generated. Finally, the features of the second feature map are detected and the reliability corresponding to the position and position of each lesion in the second feature map is obtained. By adopting the examples of the present disclosure, it is possible to accurately detect the lesion status of a plurality of sites in a patient and to preliminarily evaluate cancer throughout the patient.

It should be understood that the relevant definitions and descriptions not provided in the embodiment of the method of FIG. 2 may be referred to in the embodiment of FIG. 1 and are omitted herein.

FIG. 3 is a lesion detection device provided by the present disclosure. As shown in FIG. 3, the lesion detection device 30 acquires a first image including a plurality of sample sections, which is a three-dimensional image including an X-axis dimension, a Y-axis dimension, and a Z-axis dimension. A first feature map that extracts the features of the first image and contains the features and positions of the lesions, including the X-axis, Y-axis, and Z-axis three-dimensional features. To generate a second feature map containing two-dimensional features of X-axis dimension and Y-axis dimension by reducing the dimensions of the first generation unit 302 for generating one feature map and the features included in the first feature map. The second generation unit 303 of the above, and the detection unit 304 for detecting the second feature map and obtaining the reliability corresponding to the position and position of each lesion in the second feature map.

The acquisition unit 302 is specifically used to resample the acquired CT images of the patient at the first sampling interval to generate a first image containing a plurality of sample sections.

Specifically, the first generation unit 303 can be used in the following three cases.

Case 1: The first image is downsampled by the first neural network to generate a third feature map.

The third feature map is downsampled by the residual block of the second neural network to generate the fourth feature map.

The DenseASPP module of the second neural network extracts the features of lesions of different scales in the fourth feature map.

After processing by the DenseASPP module, a fourth predetermined feature map with the same resolution as the fourth feature map is generated, and the feature map processed by the DenseASPP module is upsampled by the deconvolution layer and residual block of the second neural network. (3) Generate a third predetermined feature map with the same resolution as the feature map.

The third feature map and the third predetermined feature map generate a first feature map with the same resolution as the third predetermined feature map, and the fourth feature map and the fourth predetermined feature map are fused to be the same as the fourth predetermined feature map. Generate a first feature map of resolution. Here, the third predetermined feature map and the fourth predetermined feature map each include the position of the lesion, and the position of the lesion is used to generate the position of the lesion in the first feature map.

Case 2: The first image is downsampled by the residual block of the second neural network to generate a fourth feature map.

The DenseASPP module of the second neural network extracts the features of lesions of different scales in the fourth feature map.

After processing by the DenseASPP module, the feature map processed by the DenseASPP module is upsampled by the deconvolution layer and the residual block of the second neural network, and the first predetermined feature map having the same resolution as the first image is generated.

The first image and the first predetermined feature map generate a first feature map having the same resolution as the first predetermined feature map. Here, the first predetermined feature map includes the location of the lesion, and the location of the lesion is used to generate the location of the lesion in the first feature map.

Case 3: The first image is downsampled by the first neural network to generate a third feature map.

The third feature map is downsampled by the residual block of the second neural network to generate the fourth feature map.

The fourth feature map is downsampled by the residual block of the second neural network to generate the fifth feature map.

The DenseASPP module of the second neural network extracts the features of lesions of different scales in the fifth feature map.

After processing by the DenseASPP module, a fifth predetermined feature map with the same resolution as the fifth feature map is generated, and the feature map processed by the DenseASPP module is upsampled by the deconvolution layer and residual block of the second neural network. Generate a fourth predetermined feature map with the same resolution as the four feature map, or upsample the feature map processed by the DenseASPP module by the deconvolution layer and residual block of the second neural network, and be the same as the third feature map. Generate a third predetermined feature map of resolution.

The third feature map and the third predetermined feature map generate a first feature map with the same resolution as the third predetermined feature map, and the fourth feature map and the fourth predetermined feature map are fused to be the same as the fourth predetermined feature map. A first feature map with a resolution is generated, and a first feature map with the same resolution as the fifth predetermined feature map is generated by fusing the fifth feature map and the fifth predetermined feature map. Here, the third predetermined feature map, the fourth predetermined feature map, and the fifth predetermined feature map each include the position of the lesion, and the position of the lesion is used to generate the position of the lesion in the first feature map.

I would like to explain that the first neural network contains a convolution layer and a residual block cascaded to the convolution layer, and the second neural network is a 3DU containing a convolution layer, a deconvolution layer, a residual block and a DenseASP P module. -Includes Net network.

Optionally, the second neural network may include a plurality of stacked 3D U-Net networks. By detecting in a plurality of 3D U-net networks, the accuracy of detection can be improved. The number of 3D U-net networks in the examples of the present disclosure is only exemplary.

It is intended to explain that the residual block may include a convolutional layer, a batch normalization layer (BN layer), a ReLU activation function and a maximum pooling layer.

Specifically, the third feature unit 304 merges the channel dimension and the Z-axis dimension for each of the features of the first feature map, and Xs each dimension of all the features of the first feature map. It is made up of an axis dimension and a Y-axis dimension, and is used to make a first feature map in which each dimension of all features consists of an X-axis dimension and a Y-axis dimension as a second feature map.

Specifically, the detection unit 305 detects the second feature map by the first detection subnet, obtains the coordinates of the position of each lesion in the second feature map, and detects the second feature map by the second detection subnet. , Used to obtain the reliability corresponding to the lesion in the second feature map.

I would like to explain that the first detection subnet contains multiple convolution layers, each connected to one ReLU activation function, and the second detection subnet contains multiple convolutional layers, each connected to one ReLU activation function. Includes convolutional layer.

The lesion detection device 30 further includes a display unit in addition to the acquisition unit 301, the first generation unit 302, the second generation unit 303, and the detection unit 304.

The display unit is specifically used to display the position of the lesion detected by the detection unit 304 and the reliability of the position.

The lesion detection device 30 further includes a training unit in addition to the acquisition unit 301, the first generation unit 302, the second generation unit 303, and the detection unit 304.

Specifically, the training unit labels the pre-stored lesions before the first generation unit extracts the features of the first image and generates a first feature map containing the features and locations of the lesions. Enter a 3D image containing multiple lesion labels for the first neural network and use the gradient descent method to each of the first neural network, the second neural network, the first detection subnet, and the second detection subnet. Used to train each parameter, or to label the lesion before the first generation unit extracts the features of the first image and generates a first feature map containing the features and locations of the lesion. To input a 3D image containing multiple lesion labels into the second neural network and use the gradient descent method to train the parameters of the second neural network, the first detection subnet, and the second detection subnet, respectively. Used for. Here, the position of each of the plurality of lesions is output by the first detection subnet.

The lesion detection device 30 is provided only as an example in the embodiments of the present disclosure, and may include more or less members than the indicated members, and may be a combination or a combination of two or more members. It should be understood that it may be realized by different arrangements of members.

It should be understood that the specific embodiment of the functional block included in the lesion detection device 30 of FIG. 3 may refer to the embodiment of the method shown in FIG. 2, and description thereof will be omitted here.

FIG. 4 is a schematic diagram of the configuration of the lesion detection device provided by the present disclosure. In the embodiments of the present disclosure, the lesion detection device is a mobile phone, a tablet, a personal digital assistant (PDA), a mobile internet device (Mobile Internet Device, MID), a smart wearable device (for example, a smart watch, etc.). Various devices such as (smart bracelets, etc.) may be included, and the embodiments of the present disclosure are not limited thereto. As shown in FIG. 4, the lesion detection device 40 may include a baseband chip 401, a memory 402 (one or more computer-readable storage media), and a peripheral system 403. These members can communicate via one or more communication buses 404.

The baseband chip 401 includes one or more processors (CPUs) 405 and one or more image processing units (GPUs) 406 capable of processing an input normal map.

Memory 402 is coupled with processor 405 and can be used to store various software programs and / or multiple instruction sets. In a specific implementation, the memory 402 may include fast random access memory or may include non-volatile memory such as one or more magnetic disk storage devices, flash memory or other non-volatile solid storage device. .. The memory 402 may store an operating system (hereinafter simply referred to as a system), for example, an embedded operating system such as ANDROID®, IOS, WINDOWS®, or LINUX®. The memory 402 may store a network communication program used for communication with one or more additional devices, one or more devices, and one or more network devices. The memory 402 may store a user interface program that realistically displays the contents of the application by a graphical user interface and receives control operations of the application from the user by input controls such as menus, dialog boxes, and buttons.

It should be understood that the memory 402 can be used to store the program code that implements the lesion detection method.

It should be understood that the processor 405 can be used to call program code that executes the lesion detection method stored in memory 402.

The memory 402 may store one or more applications. As shown in FIG. 4, these applications include social applications (eg facebook®), image management applications (eg albums), map applications (eg Google Maps), browsers (eg Safari®), Google (eg). (Registered trademark) Chrome (registered trademark)) and the like may be included.

The peripheral system 403 is mainly used to realize an interaction function between the lesion detection device 40 and the user / external environment, and mainly includes an input / output device of the lesion detection device 40. In a specific implementation, the peripheral system 403 may include a display screen controller 407, a camera controller 408, a mouse-keyboard controller 409 and an audio controller 410. Of these, each of the controllers can be coupled with the corresponding peripherals (eg, display screen 411, camera 412, mouse-keyboard 413 and audio circuit 414). In some embodiments, the display screen may be a display screen provided with a self-capacitating floating touch panel or a display screen provided with an infrared floating touch panel. In some embodiments, the camera 412 may be a 3D camera. I would like to explain that the peripheral system 403 may include other I / O external devices.

It should be understood that the display screen 411 can be used to display the location of the detected lesion and the reliability of the location.

The lesion detection device 40 is provided only as an example in the embodiments of the present disclosure, and may include more or less members than the indicated members, and may be a combination or a combination of two or more members. It should be understood that it may be realized by different arrangements of members.

It should be understood that the specific embodiment of the functional module included in the lesion detection device 40 of FIG. 4 may refer to the embodiment of the method of FIG. 2, and the description thereof is omitted here.

The present disclosure provides a computer-readable storage medium in which a computer program is stored, which is realized when the computer program is executed by a processor.

The computer-readable storage medium may be an internal storage unit of any one of the devices described in the above embodiment, for example, a hard disk or an internal memory of the device. The computer-readable storage medium may be an external storage device of the device, for example, a plug-in hard disk deployed in the device, a SmartMedia card (SmartMedia® Card, SMC), a secure digital (SD) card, or a flash. It may be a card (FlashCard) or the like. Further, the computer-readable storage medium may include an internal storage unit and an external storage device of the device. The computer-readable storage medium is used to store computer programs and other programs and data required for equipment, and may also be used to temporarily store output or output data. good.

The present disclosure is a computer program product comprising a non-temporary computer-readable storage medium in which a computer program is stored, wherein the computer program is stored in a computer in any one of the methods described in the embodiment of the above method. Provide a computer program product that operates to perform a part or all steps. The computer program product may be a single software installation package, the computer including electronic devices.

Those skilled in the art should understand that each exemplary unit and algorithm step described in the embodiments herein can be implemented with electronic hardware, computer software, or a combination thereof. In the above description, each exemplary configuration and step is generally described according to function in order to clearly illustrate the interchangeability of hardware and software. Whether these functions are performed in hardware or software depends on the specific use of the technical solution and design constraints. A professional engineer can realize the described functions in different ways depending on the specific application. It should not be understood that such realization is beyond the scope of this disclosure.

It will be clearly understood by those skilled in the art that those skilled in the art will only need to refer to the corresponding parts of the embodiments of the method for the specific operation of the above-mentioned equipment and units in order to simplify and simplify the description. The description is omitted here.

It should be understood that the devices and methods disclosed in some embodiments of the present disclosure can be realized in other forms. For example, each exemplary configuration and step has been described, but whether these functions are performed in hardware or software depends on the specific application of the technical solution and design constraints. Be done. A professional engineer can realize the described functions in different ways depending on the specific application. It should not be understood that such realization is beyond the scope of this disclosure.

The examples of the devices described above are merely exemplary. For example, the division of the unit is merely a division of a logical function, and may be divided in another form when it is actually realized. For example, multiple units or components may be combined, integrated into another system, or some features may be omitted or not implemented. Also, the mutual coupling, direct coupling or communication connection illustrated or described may be an indirect coupling or communication connection via some interface, device or unit, or in electrical, mechanical or other form. It may be a connection of.

The unit described as a separate member may or may not be physically separated, and the member indicated as a unit may or may not be a physical unit, even if it is in one place. It may be distributed over a plurality of network units. To achieve the objectives of the solutions of this embodiment, some or all units may be selected as needed in practice.

Further, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, may be physically separated units, or may be integrated into one unit by two or more. May be done. The integrated unit may be realized by hardware or software function unit.

The integrated unit may be stored in a computer-readable storage medium when implemented as a software functional unit and sold or used as an independent product. Based on this view, substantive parts of the technical solutions of the present disclosure, parts that contribute to the prior art, or all or part of the technical solutions can be realized in the Software product. .. The computer software product is stored in a storage medium and may be stored in a single computer device (which may be a personal computer, a device that is a target node of a blockchain, a network device, or the like) as described in each embodiment of the present disclosure. Includes some instructions to perform all or part of the steps in. The storage medium can store various program codes such as a USB flash drive, a mobile hard disk, a read-only memory (Read-Only Memory, ROM), a Radham access memory (Random Access Memory, RAM), a magnetic disk, or an optical disk. Includes medium.

The above is merely a specific embodiment of the present disclosure and does not limit the scope of protection of the present disclosure. One of ordinary skill in the art can readily conceive of various equal modifications or substitutions within the technical scope described in this disclosure. All of these modifications or substitutions fall within the scope of this disclosure. Therefore, it should be understood that the scope of protection of this disclosure is in accordance with the scope of claims.

Claims (17)

To acquire a first image that is a first image containing a plurality of sample sections and is a three-dimensional image including the X-axis dimension, the Y-axis dimension, and the Z-axis dimension.
The features of the first image are extracted, and a first feature map including the features and positions of the lesion, which includes the three-dimensional features of the X-axis dimension, the Y-axis dimension, and the Z-axis dimension, is generated. To do and
To reduce the dimensions of the features included in the first feature map and generate a second feature map containing the two-dimensional features of the X-axis dimension and the Y-axis dimension.
The present invention includes detecting the second feature map and obtaining the position of each lesion in the second feature map and the reliability corresponding to the position.
Focal detection method. Obtaining a first image containing multiple sample sections is
Includes resampling the acquired CT images of the patient at the first sampling interval to generate a first image containing multiple sample sections.
The method according to claim 1. Extracting the features of the first image and generating a first feature map containing the features and locations of the lesions
To generate a third feature map by downsampling the first image by the first neural network,
To generate the fourth feature map by downsampling the third feature map by the residual block of the second neural network.
Extracting the features of lesions of different scales in the fourth feature map by the Dense ASPP module of the second neural network, and
After processing by the DenseASPP module, a fourth predetermined feature map having the same resolution as the fourth feature map is generated, and the feature map after processing by the DenseASPP module is generated by the deconvolution layer of the second neural network and the residual block. Upsampling to generate a third predetermined feature map with the same resolution as the third feature map.
The third feature map and the third predetermined feature map are fused to generate a first feature map having the same resolution as the third predetermined feature map, and the fourth feature map and the fourth predetermined feature map are fused. To generate a first feature map with the same resolution as the fourth predetermined feature map.
The third predetermined feature map and the fourth predetermined feature map each include the location of the lesion, and the location of the lesion is used to generate the location of the lesion in the first feature map.
The method according to claim 1. Extracting the features of the first image and generating a first feature map containing the features and locations of the lesions
To generate a fourth feature map by downsampling the first image with the residual block of the second neural network,
Extracting the features of lesions of different scales in the fourth feature map by the Dense ASPP module of the second neural network, and
After processing by the DenseASPP module, the feature map after processing by the DenseASPP module is upsampled by the deconvolution layer of the second neural network and the residual block, and the first predetermined feature map having the same resolution as the first image is obtained. And to generate
Including fusing the first image and the first predetermined feature map to generate a first feature map having the same resolution as the first predetermined feature map.
The first predetermined feature map includes the location of the lesion and the location of the lesion is used to generate the location of the lesion in the first feature map.
The method according to claim 1. Extracting the features of the first image and generating a first feature map containing the features and locations of the lesions
The first image is downsampled by the first neural network to generate a third feature map having a lower resolution than the first image .
To generate the fourth feature map by downsampling the third feature map by the residual block of the second neural network.
The fourth feature map is downsampled by the residual block of the second neural network to generate a fifth feature map having a lower resolution than the fourth feature map.
Extracting the features of lesions of different scales in the fifth feature map by the Dense ASPP module of the second neural network, and
After processing by the DenseASPP module, a fifth predetermined feature map having the same resolution as the fifth feature map is generated, and the feature map after processing by the DenseASPP module is generated by the deconvolution layer of the second neural network and the residual block. Upsample to generate a fourth predetermined feature map with the same resolution as the fourth feature map, or upsample the feature map processed by the DenseASPP module by the deconvolution layer and residual block of the second neural network. Then, to generate a third predetermined feature map with the same resolution as the third feature map,
The third feature map and the third predetermined feature map are fused to generate a first feature map having the same resolution as the third predetermined feature map, and the fourth feature map and the fourth predetermined feature map are fused. Then, a first feature map having the same resolution as the fourth predetermined feature map is generated, and the fifth feature map and the fifth predetermined feature map are fused to generate a first feature having the same resolution as the fifth predetermined feature map. Including, including generating a map
The third predetermined feature map, the fourth predetermined feature map, and the fifth predetermined feature map each include the position of the lesion, and the position of the lesion is used to generate the position of the lesion in the first feature map.
The method according to claim 1. The first neural network includes a convolution layer and a residual block cascaded to the convolution layer.
The second neural network includes a 3D U-Net network including a convolution layer, a deconvolution layer, a residual block and the DenseASPP module.
The method according to claim 3 or 5. The second neural network is a plurality of stacked 3D U-Net networks.
The method according to claim 5 or 6. The residual block comprises a convolution layer, a batch normalization layer, a ReLU activation function and a maximum pooling layer.
The method according to claim 5 or 6. It is possible to reduce the dimensions of the features included in the first feature map and generate the second feature map.
The channel dimension and the Z-axis dimension are merged for each of the features of the first feature map, and each dimension of all the features of the first feature map consists of the X-axis dimension and the Y-axis dimension. A first feature map in which each dimension of all the features consists of an X-axis dimension and a Y-axis dimension is used as the second feature map.
The method according to claim 1. Detecting the second feature map is
The second feature map is detected by the first detection subnet, and the coordinates of the position of each lesion in the second feature map are obtained.
The second detection subnet includes detecting the second feature map and obtaining the reliability corresponding to each lesion in the second feature map.
The method according to claim 1. The first detection subnet contains a plurality of convolution layers, each connected to one ReLU activation function.
The second detection subnet comprises a plurality of convolution layers, each connected to one ReLU activation function.
The method according to claim 10. Before extracting the features of the first image and generating a first feature map containing the features and locations of the lesions,
A pre-stored three-dimensional image containing a plurality of lesion labels for labeling the lesion is input to the first neural network, and the first neural network and the second neural network are used by the gradient descent method. Further includes training each parameter of the first detection subnet and the second detection subnet, respectively.
The position of each of the plurality of lesions is output by the first detection subnet.
The method according to any one of claims 1, 2, 3, 5, 6, 7, 8, 9, 10 and 11. Before extracting the features of the first image and generating a first feature map containing the features and locations of the lesions,
A three-dimensional image containing a plurality of lesion labels for labeling the lesion is input to the second neural network, and the second neural network, the first detection subnet, and the second detection are used by using the gradient descent method. Further including training each parameter of the subnet,
The position of each of the plurality of lesions is output by the first detection subnet.
The method according to any one of claims 1, 2, 4, 7, 9, 10 and 11. An acquisition unit for acquiring a first image that is a first image including a plurality of sample sections and is a three-dimensional image including an X-axis dimension, a Y-axis dimension, and a Z-axis dimension.
The features of the first image are extracted, and a first feature map including the features and positions of the lesion, which includes the three-dimensional features of the X-axis dimension, the Y-axis dimension, and the Z-axis dimension, is generated. With the first generation unit to
A second generation unit for reducing the dimensions of the features included in the first feature map and generating a second feature map containing two-dimensional features of the X-axis dimension and the Y-axis dimension.
A detection unit for detecting the second feature map and obtaining the position of each lesion in the second feature map and the reliability corresponding to the position.
Focal detector. A lesion detection device that includes a display, memory, and a processor coupled to the memory, wherein the display is used to display the location of the lesion and the reliability corresponding to the location, the memory storing application program code. The processor is configured to execute the lesion detection method according to any one of claims 1 to 13 by calling the program code.
Focal detection device. A computer-readable storage medium in which a computer program including a program instruction is stored, and when the program instruction is executed by the processor, the lesion detection according to any one of claims 1 to 13 is performed on the processor. Make the method run,
Computer-readable storage medium. A computer program including a computer-readable code, wherein when the computer-readable code is executed in an electronic device, the processor of the electronic device is subjected to the lesion detection method according to any one of claims 1 to 13. Execute the command to realize it,
Computer program.

Images