WO2021073120A1

WO2021073120A1 - Method and device for marking lung area shadows in medical image, server, and storage medium

Info

Publication number: WO2021073120A1
Application number: PCT/CN2020/093516
Authority: WO
Inventors: 刘新卉; 王健宗
Original assignee: 平安科技（深圳）有限公司
Priority date: 2019-10-17
Filing date: 2020-05-29
Publication date: 2021-04-22
Also published as: CN110930414A

Abstract

A method and a device for marking lung area shadows in a medical image, a server and a storage medium, the method comprising: performing lung area segmentation on a medical image containing lungs according to a pre-trained lung area division model to obtain a lung area image; detecting preset key feature points contained in the lung area image according to a pre-trained lung key feature point detection model; and partitioning the lung area image on the basis of the preset key feature points contained in the lung area image and marking shadows contained in partitioned lung areas.

Description

Shadow marking method, device, server and storage medium for lung area of medical image

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on October 17, 2019, with application number 2019109890853, and the invention titled "Method, Apparatus, Server, and Storage Medium for Shadow Marking of Lung Regions for Medical Imaging". The entire content is incorporated into this application by reference.

Technical field

This application belongs to the field of artificial intelligence technology, and in particular relates to a method, device, server and storage medium for shadow marking of lung regions in medical images.

Background technique

At present, in the areas where relevant research needs to be done based on the partition of the lung area, researchers mainly use experience to partition the lungs in the medical images (X-ray images) taken by the DR equipment, and according to the shadows of each lung part after the partition. The density of each lung area and the size of the shadow in each lung area were studied. The inventor found that the distribution of the shadows in the X-ray image is uneven, and the positions of some shadow areas are difficult to accurately locate, which makes it impossible to accurately mark the shadows of the lung area based on experience.

Summary of the invention

In view of this, the embodiments of the present application provide a method, a device, a server, and a storage medium for dividing the lung region of medical images to solve the problem of the inability to accurately mark the lungs in the prior art.

The first aspect of the embodiments of the present application provides a method for shadow marking of lung regions in medical images, including:

Perform lung region segmentation on medical images containing lungs according to the pre-trained lung region division model to obtain lung region images. The lung region division model is based on a first preset number of first medical images, training In the completed U-Net model, the first medical image includes a medical image with uneven shadow distribution;

Detect the preset key feature points contained in the lung region image according to the pre-trained lung key feature point detection model, and the lung key feature point detection model is based on a second preset number of pre-labeled lung keys The lung area image of the feature point, the trained neural network model, and the first lung area image includes a shadow area;

The lung area image is partitioned based on the preset key feature points contained in the lung area image, and the shadows contained in the lung area after the partition are marked.

A second aspect of the embodiments of the present application provides a lung region shadow marking device for medical imaging, including:

The dividing module is configured to perform lung region segmentation on the medical image containing lungs according to the pre-trained lung region division model to obtain lung region images, and the lung region division model is the first preset number according to the first preset number. A medical image, a trained U-Net model, the first medical image includes a medical image with uneven shadow distribution;

The detection module is configured to detect the preset key feature points contained in the lung region image according to the pre-trained lung key feature point detection model, and the lung key feature point detection model is based on a second preset number of pre-set key feature points. An image of a lung area marked with key feature points of the lungs, a trained neural network model, and the first lung area image includes a shadow area;

The marking module is configured to partition the lung area image based on the preset key feature points contained in the lung area image, and mark the shadows contained in the lung area after the partition.

The third aspect of the embodiments of the present application provides a server, including a memory, a processor, and a computer program stored in the memory and running on the processor. The processor executes the computer program when the computer program is executed. The steps of the method for shadow marking of lung regions in medical images as described above.

The fourth aspect of the embodiments of the present application provides a computer-readable storage medium, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the lung area shadow of the medical image is realized as described above. Mark the steps of the method.

Compared with the prior art, the embodiment of the application has the beneficial effect that the lung area segmentation is performed on the medical image containing the lungs according to the lung area division model completed in advance to obtain the lung area image. The division model is a U-Net model that has been trained based on a first preset number of first medical images; the pre-trained lung key feature point detection model is used to detect the preset key feature points contained in the lung region image, The lung key feature point detection model is a neural network model that has been trained based on a second preset number of lung area images pre-marked with lung key feature points; based on the preset key points contained in the lung area image The feature points partition the lung area map, and mark the shadows contained in the lung area after the partition. Since the lung region division model is a model trained according to a first preset number of first medical images, the first medical images include medical images with uneven shadow distribution; the pre-trained lung key feature point detection model is According to a second preset number of models trained on the first lung region image pre-marked with key feature points of the lungs, the first lung region image includes a shadow region; therefore, the lung region is divided according to the pre-trained lung region The model performs lung region segmentation on medical images containing lungs, which can accurately segment the lung regions without being affected by the uneven distribution of shadows in the medical images, and then detect the model pair based on the pre-trained lung key feature points. After segmentation, the lung area is detected with preset key feature points, and the lung area image is partitioned based on the preset key feature points contained in the lung area image, and the shadows contained in the lung area after the partition are divided Marking can accurately determine the lung area where the shadow area is located, and improve the accuracy of the position marking of the shadow area.

Description of the drawings

FIG. 1 is a flow chart of the implementation of a method for shadow marking of lung regions in medical images provided by the first embodiment of the present application;

2 is a flowchart of the implementation of a method for shadow marking of lung regions in medical images provided by the second embodiment of the present application;

FIG. 3 is a flowchart of the implementation of a method for shadow marking of lung regions in medical images provided by the third embodiment of the present application;

Figure 4 is a flow chart of the specific implementation of S103 in Figure 1;

Fig. 5 is a schematic diagram of functional modules of a lung region shadow marking device for medical images provided by the present application;

Fig. 6 is a schematic diagram of the internal functions of the server provided by the present application.

Detailed ways

In the following description, for the purpose of illustration rather than limitation, specific details such as a specific system structure and technology are proposed for a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from obstructing the description of this application.

In order to illustrate the technical solution described in the present application, specific embodiments are used for description below. As shown in Fig. 1, it is a flowchart of the implementation of the method for shadow marking of lung regions in medical images provided by the first embodiment of the present application, and the execution subject of this embodiment is a server. The details are as follows:

S101: Perform lung region segmentation on a medical image containing lungs according to a pre-trained lung region division model to obtain a lung region image, where the lung region division model is based on a first preset number of first medical images , The trained U-Net model, the first medical image includes a medical image with uneven shadow distribution. Understandably, medical images containing lungs are an important basis for the diagnosis of pneumoconiosis. Common medical images containing lungs are pneumoconiosis chest radiographs, also called chest X-rays. In the process of pneumoconiosis diagnosis, doctors need to judge the texture and characteristics of each lung area in medical images including lungs based on experience. In the embodiment of this solution, in order to improve the accuracy of the doctor's diagnosis, the medical image containing the lungs is first segmented into lung regions, specifically, segmentation is performed according to the lung region segmentation model completed in advance. The pre-trained lung region division model is a U-Net model.

S102. Detect preset key feature points contained in the lung region image according to the pre-trained lung key feature point detection model, and the lung key feature point detection model is based on a second preset number of pre-labeled lungs. The first lung region image of the key feature points, the trained neural network model, and the first lung region image includes the shadow region.

Preferably, in this embodiment, the pre-trained lung key feature point detection model is a neural network model, the neural network model includes a feature layer and a detection layer, and the detection layer is a Gaussian mixture model (Gaussian model). Mixture Model, GMM), the GMM model is used to perform cluster analysis on the features of each layer of the feature layer, perform feature positioning on the features of each layer of the feature layer from coarse to fine, and finally obtain the feature layer Accurate positioning model of each layer feature.

Preferably, in this embodiment, the GMM model includes a convolutional layer of CL1-CL4, CL1-CL3 includes a convolutional layer conv and a maximum pooling layer MP, respectively, and CL4 includes a convolution with a 2×2 convolution kernel. Layer composition, FC5-FC6 are fully linked layers.

S103, partition the lung area image based on preset key feature points included in the lung area image, and mark the shadows included in the lung area after the partition.

Further, in this embodiment, the preset key feature points include the first lung tip, the first low diaphragm, the second lung tip, and the second low diaphragm; it is understandable that in some other embodiments, The preset key feature points may also include the first phrenic top and the second phrenic top, which are not specifically limited here.

Specifically, as shown in FIG. 2, it is a flow chart of the specific implementation of S103 in FIG. 1. As can be seen from FIG. 2, S103 includes:

S1031: Generate a first straight line based on the first lung apex and the second lung apex;

Generally, the human body contains left and right lung lobes. In this embodiment, the lung apexes contained in the left and right lung lobes in the medical imaging image are preset as the first lung apex and the second lung apex, respectively. The lower diaphragm is preset as the first lower diaphragm and the second lower diaphragm respectively, and a first straight line is generated based on the first lung apex and the second lung apex.

S1032: Generate a second straight line based on the first diaphragm bottom and the second diaphragm bottom.

S1033: Determine a vertical line segment between the first straight line and the second straight line, and obtain third halves of the vertical line segment.

Specifically, in this embodiment, after the vertical line segment between the first straight line and the second straight line is determined, the vertical line segment is equally divided to make the lung partial area more accurate.

It is understandable that the number of lung areas in the lungs usually needs to be determined according to actual application scenarios. For example, in the process of diagnosing lung disease, the lung area is usually determined by the type of disease caused by the occurrence of lesions in the lung area. How many lung regions are divided into, in a possible implementation manner, the vertical distance of the lung tip is divided into three equal parts, specifically, the three equal points of the vertical line segment are obtained.

It is understandable that in different implementation processes, with different application scenarios, it can be divided into arbitrarily equal parts, and there is no specific limitation here.

S1034: Generate a horizontal line based on each equal division point, and use the first straight line, the second straight line, the vertical line segment, and each horizontal line as a dividing reference line, and compare the divided lung regions The shadow is marked.

Understandably, marking the shadows contained in the lung area is of great significance in the medical field. For example, in the diagnosis of lung diseases such as lung cancer and tuberculosis based on medical images, the lung area is accurately segmented. It is particularly important to determine the shadow contained in the lung area after each segmentation, which is the basis for accurate diagnosis of the disease. Based on the density and size of shadows in each lung area, lung disease diagnosis can be performed more accurately on medical images including lungs. This is because the severity of lung disease is not only related to the density of lung shadows, but also closely related to the distribution area of lung shadows. Therefore, after the lung area is partitioned according to preset key feature points, it can be more convenient and accurate Determine the shadow density of each lung area to improve the accuracy of lung disease diagnosis.

From the above analysis, it can be seen that the lung region shadow marking method for medical images proposed in this application performs lung region segmentation on the medical image containing the lungs according to the pre-trained lung region division model to obtain the lung region image. The lung region division model is a U-Net model that is trained based on the first preset number of first medical images; the pre-trained lung key feature point detection model detects the preset key contained in the lung region image Feature points, the lung key feature point detection model is a neural network model that has been trained based on a second preset number of lung area images pre-marked with lung key feature points; based on the lung area image contained The preset key feature points divide the lung area map, and mark the shadows contained in the divided lung areas. Since the lung region division model is a model trained according to a first preset number of first medical images, the first medical images include medical images with uneven shadow distribution; the pre-trained lung key feature point detection model is According to a second preset number of models trained on the first lung region image pre-marked with key feature points of the lungs, the first lung region image includes a shadow region; therefore, the lung region is divided according to the pre-trained lung region The model performs lung region segmentation on medical images containing lungs, which can accurately segment the lung regions without being affected by the uneven distribution of shadows in the medical images, and then detect the model pair based on the pre-trained lung key feature points. After segmentation, the lung area is detected with preset key feature points, and the lung area image is partitioned based on the preset key feature points contained in the lung area image, and the shadows contained in the lung area after the partition are divided Marking can accurately determine the lung area where the shadow area is located, and improve the accuracy of the position marking of the shadow area.

Specifically, as shown in FIG. 3, it is a flow chart of the implementation of the method for shadow marking of lung regions in medical images provided by the second embodiment of the present application. It can be seen from FIG. 3 that compared with the embodiment shown in FIG. 1, the specific implementation process of S206-S209 and S101-S103 are the same in this embodiment. The difference is that S201-S205 are included before S206. Among them, S201-S205 The specific implementation process is as follows.

S201: Acquire a first preset number of first medical images, and divide the first medical images into a first proportion of training samples and a second proportion of test samples.

Specifically, the first medical image is a medical image in which the lung area is pre-divided; it is understandable that the preset number of first medical images can be obtained from a medical image library, in order to improve the accuracy of model training , Divide the preset number of first medical images into a preset ratio (for example, 7:3), and generate a training sample set generated from training samples of the first ratio and a test sample set generated from test samples of the second ratio .

S202: Input the training samples of the first proportion into a pre-established U-Net model for training.

Preferably, before training, image transformation (resize) is performed on the training sample of the first proportion according to a preset image transformation function to obtain an image of a preset size (such as 224*224); this is due to the human body The organs of are in different states at any time, and in some cases they will be squeezed by other organs and the shape will be distorted. Therefore, before performing model training, perform image transformations on the training samples, including stretching, scaling, translation and other transformations. ; Inputting the image of the preset size into the input network of the U-Net model for training can improve the efficiency and accuracy of model training.

Preferably, the preset image size transformation function may be:

dsize=Size(round(fx*src),round(fy*src))

Among them, fx: represents the zoom ratio in the width direction, fy: represents the zoom ratio in the height direction, and src represents the image before transformation.

Optionally, it is also possible to calculate a binary mask for the pixel points corresponding to each sample in the training sample, and then generate a corresponding binary image; train the U-Net model according to the binary image.

S203: Input the test sample of the second proportion into the U-Net model, and obtain a second medical image output by the U-Net model, in which the lung region is divided.

Understandably, as the model training progresses, after the training is completed, the U-Net model will output the second medical image dividing the lung area. Specifically, the higher the accuracy of the U-Net model training, the higher the output The higher the coincidence rate of the divided lung area in the second medical image with the divided lung area in the training sample.

S204: If the coincidence rate of the second medical image and the lung region of each sample in the test sample set is greater than or equal to a preset coincidence rate threshold, it is determined that the test of the U-Net model is passed. The U-Net model is the lung region division model.

Specifically, the coincidence rate of the second medical image and the preset lung area of each sample in the test sample set is recorded as IOU, then

Where C represents the lung area of the second medical image, and G represents the lung area in the test sample.

S205: If the coincidence rate of the second medical image and the lung region of each sample in the preset test sample set is less than or equal to the preset coincidence rate threshold, it is determined that the test of the U-Net model fails , Increase the number of samples in the training sample set, and execute S202. The lung region shadow marking method for medical images proposed in this application performs lung region segmentation on a medical image containing lungs according to a pre-trained lung region division model to obtain a lung region image. The lung region division model In order to train the completed U-Net model according to the first preset number of first medical images; to detect the preset key feature points contained in the lung region image according to the pre-trained lung key feature point detection model, the The lung key feature point detection model is a neural network model that is trained based on a second preset number of lung region images pre-marked with lung key feature points; based on the preset key feature points contained in the lung region image The lung area map is divided into regions, and the shadows contained in the divided lung regions are marked. Since the lung region division model is a model trained according to a first preset number of first medical images, the first medical images include medical images with uneven shadow distribution; the pre-trained lung key feature point detection model is According to a second preset number of models trained on the first lung region image pre-marked with key feature points of the lungs, the first lung region image includes a shadow region; therefore, the lung region is divided according to the pre-trained lung region The model performs lung region segmentation on medical images containing lungs, which can accurately segment the lung regions without being affected by the uneven distribution of shadows in the medical images, and then detect the model pair based on the pre-trained lung key feature points. After segmentation, the lung area is detected with preset key feature points, and the lung area image is partitioned based on the preset key feature points contained in the lung area image, and the shadows contained in the lung area after the partition are divided Marking can accurately determine the lung area where the shadow area is located, and improve the accuracy of the position marking of the shadow area. It should be noted that the above S201 to S205 are the training process for the U-Net model. It is understandable that in different embodiments, the training process for the U-Net model is not limited to the above S201 to S205, for example In an optional implementation manner, the training of the U-Net model can be completed through the following steps, which are detailed as follows:

Acquiring a first preset number of first medical images, and dividing the first medical images into training samples of a first proportion and test samples of a second proportion;

Input the training samples of the first proportion into a pre-established U-Net model for training;

Inputting the test samples of the second ratio into the U-Net model to obtain a second medical image corresponding to each of the test samples output by the U-Net model, with a lung area divided;

Acquiring a target value of the loss function of the U-Net model, where the target value is the U-Net model analyzing each test sample, and the value of the loss function of the U-Net model;

If the rate of change of the target value is less than a preset rate of change threshold, it is determined that the test of the U-Net model is passed, and the U-Net model is a lung region division model that has been trained;

If the rate of change of the target value is greater than or equal to the preset rate of change threshold, it is determined that the test of the U-Net model fails, the number of samples in the training sample set is increased, and the training is performed. The first medical image in the sample set is input to the U-Net model for training.

In an optional implementation, the loss function of the trained U-Net model is:

Among them, w(x) is a function of measuring pixel x, which is defined as follows:

Among them, w _c (x) is a function mapping related to the target area (lung area) to which x pixels belong, d ₁ (x) is the distance between x pixels and the target area, and d ₂ (x) is x The pixel is the second closest distance to the target area. w ₀ and σ are the two parameters of the model; p ₁ (x) represents the probability that the pixel point x belongs to the target area (lung area).

Specifically, as shown in FIG. 4, it is a flow chart of the implementation of the method for shadow marking of lung regions in medical images provided by the third embodiment of the present application. It can be seen from FIG. 4 that compared with the embodiment shown in FIG. 1, the specific implementation processes of S301-S303 and S101-S103 are the same in this embodiment. The difference is that S304-307 are also included before S303, where S304 and S302 can be executed at the same time, or alternatively, the specific implementation process of S304-S307 is as follows.

S304: Input a second preset number of training samples into the feature layer of the neural network model for training, and obtain a second lung region image output by the feature layer;

Specifically, the training sample is a first lung region image pre-labeled with key feature points of the lungs, and the second lung region image is a picture of the feature layer labeled with key feature points of the lungs.

S305: The detection layer performs cluster analysis on the second lung area image, and clusters to obtain a third lung area image including the preset key feature points.

Specifically, the number of FC5 in the full link layer of the detection layer is assumed to be K. During the training process, the value of K remains unchanged at 1 until the GMM model starts to converge, and the value of K starts to change; The training samples are subjected to cluster analysis on the features obtained by the CL4 layer. In this process, with the clustering analysis of the GMM model, the value of K begins to change, and finally the first K category containing the preset key feature points is obtained. Image of three lung regions.

S306: If the similarities between all the second lung region images and the third lung region images are less than a preset similarity threshold, the neural network model is the key feature points of the lungs that have been trained Check the model.

Specifically, all the images of the second lung region are re-input to the neural network model respectively, and the CL4 layer of the GMM model is calculated. After analyzing the image of the second lung region, the image obtained contains the preset The similarity between the lung region image of the key feature point and the third lung region image of the K category including the preset key feature point.

S307: If the similarity between the second lung area image and the third lung area image is greater than or equal to a preset similarity threshold, increase the number of training samples, and perform S305 again.

It should be noted that because the features of different forms of lungs at the top of the diaphragm are significantly different, in the training process of the model, first perform cluster analysis on the features of the training samples, so that it can be more accurate for different forms of The lung finds the corresponding position of the apex of the diaphragm, and the position of the apex of the diaphragm is a key feature point of the lung.

Preferably, in this solution, the loss function of the CNN neural network model can be expressed as:

It should be noted that the preset key feature points can be preset according to the geometric characteristics of the lung and the diseased area, and no specific limitation is made here.

Preferably, in this embodiment, the preset key feature points include a first lung apex, a first diaphragm bottom, a second lung apex, and a second diaphragm bottom.

From the above analysis, it can be seen that the lung region shadow marking method for medical images proposed in this application performs lung region segmentation on the medical image containing the lungs according to the pre-trained lung region division model to obtain lung region images. The lung region division model is a U-Net model that is trained based on the first preset number of first medical images; the pre-trained lung key feature point detection model detects the preset key contained in the lung region image Feature points, the lung key feature point detection model is a neural network model that has been trained based on a second preset number of lung area images pre-marked with lung key feature points; based on the lung area image contained The preset key feature points divide the lung area map, and mark the shadows contained in the divided lung areas. Since the lung region division model is a model trained according to a first preset number of first medical images, the first medical images include medical images with uneven shadow distribution; the pre-trained lung key feature point detection model is According to a second preset number of models trained on the first lung region image pre-marked with key feature points of the lungs, the first lung region image includes a shadow region; therefore, the lung region is divided according to the pre-trained lung region The model performs lung region segmentation on medical images containing lungs, which can accurately segment the lung regions without being affected by the uneven distribution of shadows in the medical images, and then detect the model pair based on the pre-trained lung key feature points. After segmentation, the lung area is detected with preset key feature points, and the lung area image is partitioned based on the preset key feature points contained in the lung area image, and the shadows contained in the lung area after the partition are divided Marking can accurately determine the lung area where the shadow area is located, and improve the accuracy of the position marking of the shadow area. Fig. 5 is a schematic diagram of functional modules of a lung region shadow marking device for medical images provided by the present application. As shown in FIG. 5, the lung region dividing device 5 for medical imaging in this embodiment includes: a dividing module 510, a detecting module 520, and a dividing module 530. among them,

The dividing module 510 is configured to perform lung region segmentation on medical images containing lungs according to the pre-trained lung region division model to obtain lung region images, and the lung region division model is based on a first preset number A first medical image, a trained U-Net model, where the first medical image includes medical images with uneven shadow distribution;

The detection module 520 is configured to detect preset key feature points contained in the lung region image according to a pre-trained lung key feature point detection model, and the lung key feature point detection model is based on a second preset number A first lung region image pre-marked with key feature points of the lungs, a trained neural network model, and the first lung region image includes a shadow region;

The partition module 530 is configured to partition the lung area image based on preset key feature points included in the lung area image, and mark the shadows included in the lung area after the partition.

Preferably, it also includes:

The acquisition module is configured to acquire a first preset number of first medical images, and divide the first medical images into a training sample set and a test sample set of a preset ratio, and the first medical image is pre-divided into lungs Regional medical imaging;

A training module, configured to input the first medical image in the training sample set into a U-Net model for training;

An obtaining module, configured to input the first medical image in the test sample set into the U-Net model to obtain a second medical image output by the U-Net model;

The first determining module is configured to determine whether the coincidence rate of the second medical image and the lung region of each sample in the preset test sample set is greater than or equal to a preset coincidence rate threshold value, then determine that the U- The test of the Net model is passed, and the U-Net model is a trained lung region division model;

The second determining module is configured to determine whether the coincidence rate of the second medical image and the lung region of each sample in the preset test sample set is less than or equal to a preset coincidence rate threshold value If the test of the model fails, increase the number of samples in the training sample set, and execute the input of the first medical image in the training sample set into the U-Net model for training.

Preferably, the damage function of the trained U-Net model is:

Among them, w(x) is defined as follows:

Among them, w _c (x) is the preset mapping function related to the lung area to which the x-th pixel belongs, d ₁ (x) is the distance between the x-th pixel and the lung area, and d ₂ (x ) Is the second closest distance of the x-th pixel to the lung area, w ₀ and σ are the constant terms of the model, and p ₁ (x) represents the probability that the x-th pixel belongs to the lung area.

Preferably, it also includes:

An image obtaining module, configured to input a second preset number of training samples into the feature layer for training, and obtain a second lung region image output by the feature layer, where the training samples are pre-labeled with key lung feature points The first lung region image of the second lung region image is a picture in which the key feature points of the lung are marked on the feature layer;

A cluster analysis module, configured to perform cluster analysis on the first lung region image based on the detection layer to obtain a third lung region image containing the preset key feature points;

The first comparison module is configured to: if the similarities between all the second lung area images and the third lung area images are less than a preset similarity threshold, the neural network model is all the training completed Describe the key feature point detection model of the lungs;

The second comparison module is configured to increase the number of training samples if the similarity between the second lung area image and the third lung area image is greater than or equal to a preset similarity threshold, and then restart The input of the preset number of training samples into the feature layer for training is performed, and the second lung region image output by the feature layer is obtained.

Preferably, the preset key feature points include a first lung apex, a first diaphragm base, a second lung apex, and a second diaphragm base marking module 530, including:

A first generating unit, configured to generate a first straight line based on the first lung apex and the second lung apex;

A second generating unit, configured to generate a second straight line based on the first diaphragm bottom and the second diaphragm bottom;

An obtaining and generating unit, configured to determine a vertical line segment between the first straight line and the second straight line, and obtain the third halves of the vertical line segment;

The third generating unit is used to generate a horizontal line based on each bisector respectively;

The marking unit is configured to use the first straight line, the second straight line, the vertical line segment, and each of the horizontal lines as the dividing reference line to obtain a preset number of lung regions, including the divided lung regions The shadow is marked.

Fig. 6 is a schematic diagram of the internal functions of the server provided by the present application. As shown in FIG. 6, the server 6 of this embodiment includes a processor 60, a memory 61, and a computer program 62 stored in the memory 61 and running on the processor 60, such as a lung area shadow marking program for medical images. When the processor 60 executes the computer program 62, the steps in the embodiment of the shadow marking method for the lung area of each medical image described above are implemented, such as steps 101 to 103 shown in FIG. 1. Alternatively, when the processor 60 executes the computer program 62, the functions of the various modules/units in the above-mentioned embodiment of the lung region shadow marking device for medical images, such as the functions of the modules 510 to 530 shown in FIG.

Exemplarily, the computer program 62 may be divided into one or more modules/units, and the one or more modules/units are stored in the memory 61 and executed by the processor 60 to complete the application. The one or more modules/units may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program 62 in the server 6. For example, the computer program 62 can be divided into a partition module, a detection module, and a partition module (a module in a virtual device), and the specific functions of each module are as follows:

The partition module is configured to partition the lung area image based on the preset key feature points contained in the lung area image, and mark the shadows contained in the lung area after the partition.

Optionally, in some feasible implementation manners, the processor is further configured to:

Acquire a first preset number of first medical images, divide the first medical image into a training sample of a first proportion and a test sample of a second proportion, and the first medical image is a medical image with a lung area divided in advance image;

Input the training samples of the first proportion into the U-Net model for training;

If the coincidence rate of the second medical image and the lung region of each sample in the preset test sample set is greater than or equal to the preset coincidence rate threshold, it is determined that the test of the U-Net model is passed, and U-Net model is the lung region division model after training;

If the coincidence rate of the second medical image and the lung region of each sample in the preset test sample set is less than or equal to the preset coincidence rate threshold, it is determined that the test of the U-Net model is not passed, and increase The number of samples in the training sample set is executed, and the first medical image in the training sample set is input to the U-Net model for training.

Optionally, in some feasible implementation manners, the loss function of the trained U-Net model is:

Among them, w(x) is defined as follows:

Among them, L is the probability that the xth pixel belongs to the preset lung area, w(x) is, w _c (x) is the preset mapping function related to the lung area to which the xth pixel belongs, d ₁ (x) is the distance between the xth pixel and the lung area, d ₂ (x) is the second closest distance from the xth pixel to the lung area, w ₀ and σ are constant terms, p ₁ (x ) Is the probability that the xth pixel belongs to the lung area.

Optionally, in some feasible implementation manners, the pre-trained neural network model includes a feature layer and a detection layer; the processor is configured to:

A second preset number of training samples are input to the feature layer for training, and a second lung region image output by the feature layer is obtained. The training sample is a first lung region pre-labeled with key feature points of the lungs An image, where the second lung region image is a picture in which key feature points of the lung are marked on the feature layer;

The detection layer performs cluster analysis on the second lung region image to obtain a third lung region image including the preset key feature points;

If the similarities between all the second lung region images and the third lung region images are less than the preset similarity threshold, the neural network model is the trained lung key feature point detection model ；

If the similarity between the second lung area image and the third lung area image is greater than or equal to the preset similarity threshold, the number of training samples is increased, and the preset number of The training samples are input to the feature layer for training, and the second lung region image output by the feature layer is obtained.

Generating a first straight line based on the first lung tip and the second lung tip;

Generating a second straight line based on the first diaphragm bottom and the second diaphragm bottom;

Determine the vertical line segment between the first straight line and the second straight line, and obtain the third halves of the vertical line segment;

Generate a horizontal line based on each division point;

Using the first straight line, the second straight line, the vertical line segment, and each horizontal line as a dividing reference line, a preset number of lung regions are obtained, and the shadows included in the divided lung regions are marked.

Optionally, in some feasible implementation manners, the detection layer includes a Gaussian mixture model model, and the Gaussian mixture model model includes a convolutional layer of CL1-CL4, and CL1-CL3 includes a convolutional layer and a maximum pooling layer, respectively. , CL4 includes a convolutional layer with a 2×2 convolution kernel, and FC5-FC6 is a fully-linked layer.

In an exemplary embodiment, a computer-readable storage medium is also provided, and the computer-readable storage medium may be non-volatile or volatile. A computer program is stored thereon, and the computer program includes program instructions. When the program instructions are executed by the processor, they are used to implement the following steps:

According to the pre-trained lung key feature point detection model, the preset key feature points contained in the lung region image are detected, and the lung key feature point detection model is based on a second preset number of pre-labeled lung keys A first lung region image of a feature point, a trained neural network model, the first lung region image including a shadow region;

Optionally, in some feasible implementation manners, when the program instructions are executed by the processor, they are further used to implement the following steps:

Among them, w(x) is defined as follows:

Among them, L is the probability that the Xth pixel belongs to the preset lung region, w(x) is, w _c (x) is the preset mapping function related to the lung region to which the xth pixel belongs, d ₁ (x) is the distance between the xth pixel and the lung area, d ₂ (x) is the distance between the xth pixel and the lung area, w ₀ and σ are constant terms, p ₁ (x ) Is the probability that the xth pixel belongs to the lung area.

The above are only specific implementations of this application, but the protection scope of this application is not limited to this. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in this application. Should be covered within the scope of protection of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

A method for shadow marking of lung regions in medical imaging, which includes:

Perform lung region segmentation on medical images containing lungs according to the pre-trained lung region division model to obtain lung region images. The lung region division model is based on a first preset number of first medical images, training In the completed U-Net model, the first medical image includes a medical image with uneven shadow distribution;

According to the pre-trained lung key feature point detection model, the preset key feature points contained in the lung region image are detected, and the lung key feature point detection model is based on a second preset number of pre-labeled lung keys A first lung region image of a feature point, a trained neural network model, the first lung region image including a shadow region;

The lung area image is partitioned based on the preset key feature points contained in the lung area image, and the shadows contained in the lung area after the partition are marked.
The lung region shadow marking method for medical images according to claim 1, wherein before the lung region segmentation is performed on the medical image containing the lungs according to the lung region division model completed in advance, the lung region is obtained ,include:

Acquire a first preset number of first medical images, divide the first medical image into a training sample of a first proportion and a test sample of a second proportion, and the first medical image is a medical image with a lung area divided in advance image;

Input the training samples of the first proportion into the U-Net model for training;

Inputting the test samples of the second ratio into the U-Net model to obtain a second medical image corresponding to each of the test samples output by the U-Net model, with a lung area divided;

If the coincidence rate of the second medical image and the lung region of each sample in the preset test sample set is greater than or equal to the preset coincidence rate threshold, it is determined that the test of the U-Net model is passed, and U-Net model is the lung region division model after training;

If the coincidence rate of the second medical image and the lung region of each sample in the preset test sample set is less than or equal to the preset coincidence rate threshold, it is determined that the test of the U-Net model is not passed, and increase The number of samples in the training sample set is executed, and the first medical image in the training sample set is input to the U-Net model for training.
The method for marking the lung area shadows of medical images according to claim 2, wherein, in the step of inputting the test sample of the second proportion into the U-Net model, each output of the U-Net model is obtained. After the second medical image of the lung region corresponding to the test sample is divided, it includes:

Acquiring a target value of the loss function of the U-Net model, where the target value is the U-Net model analyzing each test sample, and the value of the loss function of the U-Net model;

If the rate of change of the target value is less than a preset rate of change threshold, it is determined that the test of the U-Net model is passed, and the U-Net model is a lung region division model that has been trained;

If the rate of change of the target value is greater than or equal to the preset rate of change threshold, it is determined that the test of the U-Net model fails, the number of samples in the training sample set is increased, and the training is performed. The first medical image in the sample set is input to the U-Net model for training.
The shadow marking method for lung regions of medical images according to claim 3, wherein the loss function of the trained U-Net model is:

Among them, w(x) is defined as follows:

Among them, L is the probability that the xth pixel belongs to the preset lung area, w(x) is, w c (x) is the preset mapping function related to the lung area to which the xth pixel belongs, d 1 (x) is the distance between the xth pixel and the lung area, d 2 (x) is the second closest distance from the xth pixel to the lung area, w 0 and σ are constant terms, p 1 (x ) Is the probability that the x-th pixel belongs to the lung area.
The method for marking lung area shadows in medical images according to claim 1, wherein the pre-trained neural network model includes a feature layer and a detection layer;

Before the detecting the preset key feature points contained in the lung region according to the pre-trained lung key feature point detection model, the method includes:

A second preset number of training samples are input to the feature layer for training, and a second lung region image output by the feature layer is obtained. The training sample is a first lung region pre-labeled with key feature points of the lungs An image, where the second lung region image is a picture in which key feature points of the lung are marked on the feature layer;

The detection layer performs cluster analysis on the second lung region image to obtain a third lung region image including the preset key feature points;

If the similarities between all the second lung region images and the third lung region images are less than the preset similarity threshold, the neural network model is the trained lung key feature point detection model ；

If the similarity between the second lung area image and the third lung area image is greater than or equal to the preset similarity threshold, the number of training samples is increased, and the preset number of The training samples are input to the feature layer for training, and the second lung region image output by the feature layer is obtained.
The method for marking lung area shadows in medical images according to claim 1, wherein the preset key feature points include a first lung apex, a first diaphragmatic floor, a second lung apex, and a second diaphragmatic floor; The preset key feature points included in the lung area partition the lung area to obtain a preset number of lung areas, including:

Generating a first straight line based on the first lung tip and the second lung tip;

Generating a second straight line based on the first diaphragm bottom and the second diaphragm bottom;

Determine the vertical line segment between the first straight line and the second straight line, and obtain the third halves of the vertical line segment;

Generate a horizontal line based on each division point;

Using the first straight line, the second straight line, the vertical line segment, and each horizontal line as a dividing reference line, a preset number of lung regions are obtained, and the shadows included in the divided lung regions are marked.
The method for marking lung area shadows in medical images according to claim 5, wherein the detection layer includes a Gaussian mixture model model, and the Gaussian mixture model model includes convolutional layers of CL1-CL4, and CL1-CL3 respectively include volume Build-up layer and maximum pooling layer, CL4 includes a convolutional layer with a 2×2 convolution kernel, and FC5-FC6 are fully-linked layers.
A shadow marking device for lung area in medical imaging, which includes:

The dividing module is configured to perform lung region segmentation on the medical image containing lungs according to the pre-trained lung region division model to obtain lung region images, and the lung region division model is the first preset number according to the first preset number. A medical image, a trained U-Net model, the first medical image includes a medical image with uneven shadow distribution;

The detection module is configured to detect the preset key feature points contained in the lung region image according to the pre-trained lung key feature point detection model, and the lung key feature point detection model is based on a second preset number of pre-set key feature points. An image of a lung area marked with key feature points of the lungs, a trained neural network model, and the first lung area image includes a shadow area;

The marking module is configured to partition the lung area image based on the preset key feature points contained in the lung area image, and mark the shadows contained in the lung area after the partition.
A server includes a memory and a processor, the processor and the memory are connected to each other, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the memory The program instructions of, where:

Perform lung region segmentation on medical images containing lungs according to the pre-trained lung region division model to obtain lung region images. The lung region division model is based on a first preset number of first medical images, training In the completed U-Net model, the first medical image includes a medical image with uneven shadow distribution;

According to the pre-trained lung key feature point detection model, the preset key feature points contained in the lung region image are detected, and the lung key feature point detection model is based on a second preset number of pre-labeled lung keys A first lung region image of a feature point, a trained neural network model, the first lung region image including a shadow region;

The lung area image is partitioned based on the preset key feature points contained in the lung area image, and the shadows contained in the lung area after the partition are marked.
The server of claim 9, wherein the processor is configured to:

Acquire a first preset number of first medical images, divide the first medical image into a training sample of a first proportion and a test sample of a second proportion, and the first medical image is a medical image with a lung area divided in advance image;

Input the training samples of the first proportion into the U-Net model for training;

Inputting the test samples of the second ratio into the U-Net model to obtain a second medical image corresponding to each of the test samples output by the U-Net model, which divided the lung area;

If the coincidence rate of the second medical image and the lung region of each sample in the preset test sample set is greater than or equal to the preset coincidence rate threshold, it is determined that the test of the U-Net model is passed, and U-Net model is the lung region division model after training;

If the coincidence rate of the second medical image and the lung region of each sample in the preset test sample set is less than or equal to the preset coincidence rate threshold, it is determined that the test of the U-Net model is not passed, and increase The number of samples in the training sample set is executed, and the first medical image in the training sample set is input to the U-Net model for training.
The server of claim 10, wherein the processor is configured to:

Acquiring a target value of the loss function of the U-Net model, where the target value is the U-Net model analyzing each test sample, and the value of the loss function of the U-Net model;

If the rate of change of the target value is less than a preset rate of change threshold, it is determined that the test of the U-Net model is passed, and the U-Net model is a lung region division model that has been trained;

If the rate of change of the target value is greater than or equal to the preset rate of change threshold, it is determined that the test of the U-Net model fails, the number of samples in the training sample set is increased, and the training is performed. The first medical image in the sample set is input to the U-Net model for training.
The server according to claim 11, wherein the loss function of the trained U-Net model is:

Among them, w(x) is defined as follows:

Among them, L is the probability that the xth pixel belongs to the preset lung area, w(x) is, w c (x) is the preset mapping function related to the lung area to which the xth pixel belongs, d 1 (x) is the distance between the xth pixel and the lung area, d 2 (x) is the distance between the xth pixel and the lung area, w 0 and σ are constant terms, p 1 (x ) Is the probability that the xth pixel belongs to the lung area.
9. The server of claim 9, wherein the pre-trained neural network model includes a feature layer and a detection layer;

The processor is used for:

A second preset number of training samples are input to the feature layer for training, and a second lung region image output by the feature layer is obtained. The training sample is a first lung region pre-labeled with key feature points of the lungs An image, where the second lung region image is a picture in which key feature points of the lung are marked on the feature layer;

The detection layer performs cluster analysis on the second lung region image to obtain a third lung region image including the preset key feature points;

If the similarities between all the second lung region images and the third lung region images are less than the preset similarity threshold, the neural network model is the trained lung key feature point detection model ；

If the similarity between the second lung area image and the third lung area image is greater than or equal to the preset similarity threshold, the number of training samples is increased, and the preset number of The training samples are input to the feature layer for training, and the second lung region image output by the feature layer is obtained.
The server of claim 9, wherein the processor is configured to:

Generating a first straight line based on the first lung tip and the second lung tip;

Generating a second straight line based on the first diaphragm bottom and the second diaphragm bottom;

Determine the vertical line segment between the first straight line and the second straight line, and obtain the third halves of the vertical line segment;

Generate a horizontal line based on each division point;

Using the first straight line, the second straight line, the vertical line segment, and each of the horizontal lines as a dividing reference line, a preset number of lung regions are obtained, and the shadows included in the divided lung regions are marked.
The server according to claim 13, wherein the detection layer includes a Gaussian mixture model model, the Gaussian mixture model model includes a convolutional layer of CL1-CL4, and CL1-CL3 respectively include a convolutional layer and a maximum pooling layer, CL4 includes a convolutional layer with a 2×2 convolution kernel, and FC5-FC6 are fully linked layers.
A computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, the computer program includes program instructions, and when the program instructions are executed by a processor, they are used to implement the following steps:

Perform lung region segmentation on medical images containing lungs according to the pre-trained lung region division model to obtain lung region images. The lung region division model is based on a first preset number of first medical images, training In the completed U-Net model, the first medical image includes a medical image with uneven shadow distribution;

Detect the preset key feature points contained in the lung region image according to the pre-trained lung key feature point detection model, and the lung key feature point detection model is based on a second preset number of pre-labeled lung keys A first lung region image of a feature point, a trained neural network model, the first lung region image including a shadow region;

The lung area image is partitioned based on the preset key feature points contained in the lung area image, and the shadows contained in the lung area after the partition are marked.
15. The computer-readable storage medium of claim 16, wherein when the program instructions are executed by the processor, they are further used to implement the following steps:

Acquire a first preset number of first medical images, divide the first medical image into a training sample of a first proportion and a test sample of a second proportion, and the first medical image is a medical image with a lung area divided in advance image;

Input the training samples of the first proportion into the U-Net model for training;

Inputting the test samples of the second ratio into the U-Net model to obtain a second medical image corresponding to each of the test samples output by the U-Net model, with a lung area divided;

If the coincidence rate of the second medical image and the lung region of each sample in the preset test sample set is greater than or equal to the preset coincidence rate threshold, it is determined that the test of the U-Net model is passed, and U-Net model is the lung region division model after training;

If the coincidence rate of the second medical image and the lung region of each sample in the preset test sample set is less than or equal to the preset coincidence rate threshold, it is determined that the test of the U-Net model is not passed, and increase The number of samples in the training sample set is executed, and the first medical image in the training sample set is input to the U-Net model for training.
17. The computer-readable storage medium of claim 17, wherein, when the program instructions are executed by the processor, they are further used to implement the following steps:

Acquiring a target value of the loss function of the U-Net model, where the target value is the U-Net model analyzing each test sample, and the value of the loss function of the U-Net model;

If the rate of change of the target value is less than a preset rate of change threshold, it is determined that the test of the U-Net model is passed, and the U-Net model is a lung region division model that has been trained;

If the rate of change of the target value is greater than or equal to the preset rate of change threshold, it is determined that the test of the U-Net model fails, the number of samples in the training sample set is increased, and the training is performed. The first medical image in the sample set is input to the U-Net model for training.
18. The computer-readable storage medium of claim 18, wherein when the program instructions are executed by the processor, they are further used to implement the following steps:

Among them, w(x) is defined as follows:

Among them, L is the probability that the Xth pixel belongs to the preset lung region, w(x) is, w c (x) is the preset mapping function related to the lung region to which the xth pixel belongs, d 1 (x) is the distance between the xth pixel and the lung area, d 2 (x) is the distance between the xth pixel and the lung area, w 0 and σ are constant terms, p 1 (x ) Is the probability that the xth pixel belongs to the lung area.
15. The computer-readable storage medium of claim 16, wherein when the program instructions are executed by the processor, they are further used to implement the following steps:

A second preset number of training samples are input to the feature layer for training, and a second lung region image output by the feature layer is obtained. The training sample is a first lung region pre-labeled with key feature points of the lungs An image, where the second lung region image is a picture in which key feature points of the lung are marked on the feature layer;

The detection layer performs cluster analysis on the second lung region image to obtain a third lung region image including the preset key feature points;

If the similarities between all the second lung region images and the third lung region images are less than the preset similarity threshold, the neural network model is the trained lung key feature point detection model ；

If the similarity between the second lung area image and the third lung area image is greater than or equal to the preset similarity threshold, the number of training samples is increased, and the preset number of The training samples are input to the feature layer for training, and the second lung region image output by the feature layer is obtained.