WO2021217857A1 - 切片缺陷检测方法、装置、电子设备及可读存储介质 - Google Patents

切片缺陷检测方法、装置、电子设备及可读存储介质 Download PDF

Info

Publication number
WO2021217857A1
WO2021217857A1 PCT/CN2020/098983 CN2020098983W WO2021217857A1 WO 2021217857 A1 WO2021217857 A1 WO 2021217857A1 CN 2020098983 W CN2020098983 W CN 2020098983W WO 2021217857 A1 WO2021217857 A1 WO 2021217857A1
Authority
WO
WIPO (PCT)
Prior art keywords
slice
defect
detection model
branch
category
Prior art date
Application number
PCT/CN2020/098983
Other languages
English (en)
French (fr)
Inventor
王佳平
南洋
李风仪
谢春梅
侯晓帅
Original Assignee
平安科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 平安科技（深圳）有限公司 filed Critical 平安科技（深圳）有限公司
Publication of WO2021217857A1 publication Critical patent/WO2021217857A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G01N2021/8883—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges involving the calculation of gauges, generating models
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques

Definitions

  • This application relates to the technical field of intelligent decision-making, and in particular to a method, device, electronic device, and readable storage medium for detecting slice defects.
  • Medical image slices are of great significance to three-dimensional positioning, three-dimensional visualization, surgical planning and computer-aided diagnosis.
  • the quality of slices directly affects the diagnostic efficiency and quality of diagnosis.
  • High-quality slices are the essential foundation and guarantee for correct pathological diagnosis.
  • the u-net network is usually used to detect defects in the defect slices.
  • the inventor realized that because the u-net network only uses the convolutional layer and the pooling layer, as the network deepens, the information will gradually be lost, and the gradient will disappear easily. Or the gradient explosion situation makes it impossible to accurately identify the defect area; and the u-net network mainly classifies a single pixel, and lacks the category detection of the whole image, which makes the defect classification accuracy not high. Therefore, there is an urgent need for a slice defect detection method to improve the detection accuracy of defect areas and defect categories.
  • the slice defect detection method provided by this application includes:
  • Feature extraction step receiving the slice to be tested submitted by the user, performing segmentation processing on the slice to be tested to obtain the slice set to be tested, and inputting the slice set to be tested into the feature extraction branch of the trained slice defect detection model to obtain The feature set of each slice in the slice set to be detected;
  • the first detection step input the feature set into the up-sampling branch of the slice defect detection model to obtain each defect area of each slice in the slice set to be inspected and the first defect category distribution table corresponding to each defect area, and The defect category with the largest first probability value in the first defect category distribution table corresponding to each defect area is used as the predicted defect category corresponding to each defect area;
  • the second detection step input the feature set into the classification branch of the slice defect detection model to obtain a second defect category distribution table corresponding to each slice in the slice set to be detected;
  • Defect determining step obtaining the second probability value of the predicted defect category in the second defect category distribution table corresponding to each defect area of each slice in the slice set to be inspected, when the second probability value is greater than a preset threshold
  • the predicted defect category corresponding to each defect area is taken as the target defect category corresponding to each defect area
  • each defect area of each slice in the slice set to be inspected and the target defect category corresponding to each defect area are summarized , Obtain each defect area of the slice to be inspected and the target defect category corresponding to each defect area.
  • This application also provides a slice defect detection device, including:
  • the feature extraction module is used to receive the slice to be detected submitted by the user, perform segmentation processing on the slice to be detected to obtain the slice set to be tested, and input the slice set to be tested into the feature extraction branch of the trained slice defect detection model , Obtain the feature set of each slice in the slice set to be detected;
  • the first detection module is configured to input the feature set into the up-sampling branch of the slice defect detection model to obtain each defect area of each slice in the slice set to be inspected and a first defect category distribution table corresponding to each defect area Taking the defect category with the largest first probability value in the first defect category distribution table corresponding to each defect area as the predicted defect category corresponding to each defect area;
  • the second detection module is configured to input the feature set into the classification branch of the slice defect detection model to obtain a second defect category distribution table corresponding to each slice in the slice set to be detected;
  • Defect determining step obtaining the second probability value of the predicted defect category in the second defect category distribution table corresponding to each defect area of each slice in the slice set to be inspected, when the second probability value is greater than a preset threshold
  • the predicted defect category corresponding to each defect area is taken as the target defect category corresponding to each defect area
  • each defect area of each slice in the slice set to be inspected and the target defect category corresponding to each defect area are summarized , Obtain each defect area of the slice to be inspected and the target defect category corresponding to each defect area.
  • the present application also provides an electronic device, which includes a memory and a processor, the memory stores a slice defect detection program that can be run on the processor, and the slice defect When the detection program is executed by the processor, the following steps are implemented:
  • Feature extraction step receiving the slice to be tested submitted by the user, performing segmentation processing on the slice to be tested to obtain the slice set to be tested, and inputting the slice set to be tested into the feature extraction branch of the trained slice defect detection model to obtain The feature set of each slice in the slice set to be detected;
  • the first detection step input the feature set into the up-sampling branch of the slice defect detection model to obtain each defect area of each slice in the slice set to be inspected and the first defect category distribution table corresponding to each defect area, and The defect category with the largest first probability value in the first defect category distribution table corresponding to each defect area is used as the predicted defect category corresponding to each defect area;
  • the second detection step input the feature set into the classification branch of the slice defect detection model to obtain a second defect category distribution table corresponding to each slice in the slice set to be detected;
  • Defect determining step obtaining the second probability value of the predicted defect category in the second defect category distribution table corresponding to each defect area of each slice in the slice set to be inspected, when the second probability value is greater than a preset threshold
  • the predicted defect category corresponding to each defect area is taken as the target defect category corresponding to each defect area
  • each defect area of each slice in the slice set to be inspected and the target defect category corresponding to each defect area are summarized , Obtain each defect area of the slice to be inspected and the target defect category corresponding to each defect area.
  • the present application also provides a computer-readable storage medium having a slice defect detection program stored on the computer-readable storage medium, which is implemented when the slice defect detection program is executed by one or more processors The following steps:
  • Receive the to-be-detected slice submitted by the user perform segmentation processing on the to-be-detected slice to obtain the to-be-detected slice set, and input the to-be-detected slice set into the feature extraction branch of the trained slice defect detection model to obtain the to-be-detected slice
  • the feature set of each slice in the slice set
  • the feature set is input into the up-sampling branch of the slice defect detection model to obtain each defect area of each slice in the slice set to be inspected and the first defect category distribution table corresponding to each defect area, and the each defect area
  • the defect category with the largest first probability value in the corresponding first defect category distribution table is used as the predicted defect category corresponding to each defect area;
  • the second probability value of the predicted defect category in the second defect category distribution table corresponding to each defect area of each slice in the slice set to be inspected and when the second probability value is greater than a preset threshold, all The predicted defect category corresponding to each defect area is used as the target defect category corresponding to each defect area, and each defect area of each slice in the slice set to be inspected and the target defect category corresponding to each defect area are summarized to obtain the Each defect area of the slice to be inspected and the target defect category corresponding to each defect area.
  • this application first obtains the feature set of each slice in the slice set to be tested by inputting the slice set to be detected into the feature extraction branch of the trained slice defect detection model; then, the feature set is input into the slice defect detection model The upsampling branch of each slice is obtained for each defect area of each slice and the predicted defect category corresponding to each defect area; then, the feature set is input into the classification branch of the slice defect detection model to obtain the second defect category distribution table corresponding to each slice; Finally, obtain the second probability value of the predicted defect category corresponding to each defect area of each slice in the second defect category distribution table. When the second probability value is greater than the preset threshold, the predicted defect category corresponding to each defect area is taken as The target defect category corresponding to each defect area.
  • the slice defect detection model in this application is generated by improving the u-net model.
  • the residual module replaces the feature extraction branch of the u-net model and the volume in the upsampling branch.
  • the layering ensures the integrity of the information, makes the features extracted by the feature extraction branch more complete, and the defect area restored by the upsampling branch is more accurate.
  • the classification branch is added to the output of the feature extraction branch of the u-net model to reduce the upper
  • the possibility of false positive output of the sampling branch makes the classification accuracy higher. Therefore, the present application improves the accuracy of the detection of the slice defect area and the defect category.
  • Fig. 1 is a schematic diagram of an embodiment of an electronic device of this application
  • FIG. 2 is a schematic structural diagram of an embodiment of a slice defect detection model according to the application.
  • FIG. 3 is a schematic structural diagram of an embodiment of a residual module of this application.
  • FIG. 4 is a block diagram of an embodiment of a slice defect detection device according to the present application.
  • FIG. 5 is a flowchart of an embodiment of a method for detecting slice defects according to the present application.
  • the electronic device 1 is a device that can automatically perform numerical calculation and/or information processing in accordance with pre-set or stored instructions.
  • the electronic device 1 may be a computer, a single web server, a server group composed of multiple web servers, or a cloud composed of a large number of hosts or web servers based on cloud computing, where cloud computing is a type of distributed computing, A super virtual computer composed of a group of loosely coupled computer sets.
  • the electronic device 1 includes, but is not limited to, a memory 11, a processor 12, and a network interface 13 that are communicatively connected to each other through a system bus.
  • the memory 11 stores a slice defect detection program 10, and the slice defect The detection program 10 can be executed by the processor 12.
  • FIG. 1 only shows the electronic device 1 with the components 11-13 and the slice defect detection program 10. Those skilled in the art can understand that the structure shown in FIG. 1 does not constitute a limitation on the electronic device 1, and may include ratios Fewer or more parts are shown, or some parts are combined, or different parts are arranged.
  • the memory 11 includes a memory and at least one type of readable storage medium.
  • the memory provides a cache for the operation of the electronic device 1;
  • the readable storage medium can be, for example, flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM) ), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disks, optical disks and other non-volatile storage media.
  • the readable storage medium may be an internal storage unit of the electronic device 1, such as a hard disk of the electronic device 1.
  • the nonvolatile storage medium may also be an external storage unit of the electronic device 1.
  • Storage devices such as plug-in hard disks, Smart Media Card (SMC), Secure Digital (SD) cards, flash memory cards (Flash Card), etc., equipped on the electronic device 1.
  • the readable storage medium of the memory 11 is generally used to store the operating system and various application software installed in the electronic device 1, for example, to store the code of the slice defect detection program 10 in an embodiment of the present application.
  • the memory 11 can also be used to temporarily store various types of data that have been output or will be output.
  • the processor 12 may be a central processing unit (Central Processing Unit, CPU), a controller, a microcontroller, a microprocessor, or other data processing chips.
  • the processor 12 is generally used to control the overall operation of the electronic device 1, such as performing data interaction or communication-related control and processing with other devices.
  • the processor 12 is used to run the program code or processing data stored in the memory 11, for example, to run the slice defect detection program 10 and so on.
  • the network interface 13 may include a wireless network interface or a wired network interface, and the network interface 13 is used to establish a communication connection between the electronic device 1 and a client (not shown in the figure).
  • the electronic device 1 may further include a user interface.
  • the user interface may include a display (Display) and an input unit such as a keyboard (Keyboard).
  • the optional user interface may also include a standard wired interface and a wireless interface.
  • the display may be an LED display, a liquid crystal display, a touch liquid crystal display, an OLED (Organic Light-Emitting Diode, organic light-emitting diode) touch device, etc.
  • the display can also be appropriately called a display screen or a display unit, which is used to display the information processed in the electronic device 1 and to display a visualized user interface.
  • the present application is mainly designed and implemented based on image processing and detection models in artificial intelligence technology.
  • the slice defect detection program 10 is executed by the processor 12, the following feature extraction steps are implemented: The first inspection step, the second inspection step, and the defect determination step.
  • Feature extraction step receiving the slice to be tested submitted by the user, performing segmentation processing on the slice to be tested to obtain the slice set to be tested, and inputting the slice set to be tested into the feature extraction branch of the trained slice defect detection model to obtain The feature set of each slice in the slice set to be detected.
  • the slice to be detected is a digital image obtained by scanning a pathological slice by a scanner, and the defect types of the slice include wrinkles, knife marks, impurities, bubbles, and the like.
  • the performing segmentation processing on the slice to be detected to obtain a slice set to be detected includes:
  • A1 Divide the slice to be detected into a plurality of small slices of preset size by means of a sliding window
  • the 0th layer is the clearest, and the pixels are about 100000*100000.
  • the definition of the subsequent layers will be halved in turn.
  • the third layer of pathological slices (pixels Is 20000*20000) as the slice to be tested. Because the slice to be detected is too large to be directly input to the detection model, it is necessary to divide the slice to be detected into small slices, and the pixels of each small slice are about 512*512 ⁇ 1024*1024.
  • the image is cut according to the sliding window method. If the pixels of the last small slice obtained after splitting are insufficient, this small slice needs to be complemented, and the default size of the small slice is 512 *512 as an example, for the part where the edge of the last small slice is less than 512, it needs to be filled with a value of 0, that is, to supplement the black background.
  • Correction processing is performed on the multiple small slices of preset size by using gamma correction to obtain a slice set to be detected.
  • the brightness of the slice will be uneven or the biological target is too bright or too dark, so that the contrast of the slice is not high, and the contrast can be improved, which can greatly retain the characteristics of the biological target.
  • gamma correction is used to improve the contrast of slices, and the calculation formula is:
  • V out-i A ⁇ B ⁇ V in-i
  • V out-i is the pixel value after gamma correction of the i-th slice in the slice to be detected
  • A is the brightness parameter
  • B is the contrast parameter
  • V in-i is the original pixel value of the i-th slice in the slice to be detected .
  • FIG. 2 it is a schematic structural diagram of an embodiment of a slice defect detection model of this application.
  • the slice defect detection model is an improved u-net model.
  • the original u-net model has a U-shaped structure, including feature extraction branches and up-sampling branches.
  • the feature extraction branches are used to extract the features of the picture, and the up-sampling branches are used to restore the image accuracy.
  • the feature extraction branch is located on the left side of the U-shaped structure. It is a repetitive convolutional network architecture. In each repetition, there are 2 convolutional layers with a 3*3 convolution kernel, followed by an activation layer, and a 2*2 The max pooling layer with a step size of 2 is used for down-sampling. After each down-sampling, we double the number of feature channels.
  • the up-sampling branch is located on the right side of the U-shaped structure, including a 2*2 deconvolution (that is, transposed convolution, a type of convolution). Each time the deconvolution is used, the number of feature channels is halved, and the feature map Double the size. After the deconvolution, the result of the deconvolution is spliced with the corresponding feature map in the feature extraction branch.
  • the feature map in the feature extraction branch is slightly larger in size. It needs to be trimmed and spliced.
  • the spliced map is convolved twice and 3*3.
  • Each convolution is followed by a RELU.
  • the last layer is the convolution kernel.
  • the 1*1 convolutional layer is used to map the 64-channel feature map to a class label. There are 23 convolutional layers in the entire u-net network.
  • the construction process of the slice defect detection model includes:
  • the original u-net network only uses the convolutional layer and the pooling layer.
  • problems such as information loss and loss. It is easy to cause gradients to disappear or explode. How to deepen the network Ensuring that the information does not degenerate under the circumstances is the problem to be solved by this application.
  • FIG. 3 it is a schematic structural diagram of an embodiment of the residual module of this application.
  • the residual module includes two 3*3 convolutional layers, an activation layer, and a skip connection.
  • the skip connection directly detours the input information to the output.
  • the entire network only needs to learn the part of the input and output differences, not only The learning objectives and difficulty are simplified, and the integrity of the information is also ensured, so that the features extracted by the feature extraction branch are more complete, and the image restored by the upsampling branch is more accurate.
  • the original u-net model uses the last convolutional layer of the up-sampling branch to realize the recognition of the slice defect area and defect category. It classifies the single pixel of each slice in the slice set to be detected. Type detection of graphs.
  • the input of the classification branch of the slice defect detection model of this application is the output of the feature extraction branch.
  • the classification branch performs classification detection for all the features of each slice in the slice set to be detected, and pays more attention to the global characteristics of each slice.
  • the training process of the slice defect detection model includes:
  • the slice samples in the first training set and the second training set carry labeling information, and the labeling information includes each real defect area of the slice sample and a real defect type probability value (0 or 1) corresponding to each real defect area.
  • IOU ti is the intersection ratio of the i-th type defect of the t-th slice sample in the first training set
  • a ti is the predicted defect area corresponding to the i-th type defect of the t-th slice sample in the first training set
  • B ti is The real defect area corresponding to the i-th type defect of the t-th slice sample in the first training set
  • FL(q ri ) is the balanced cross-entropy loss value of the i-th defect of the r-th slice sample in the second training set
  • ⁇ i is the category weight parameter of the i-th defect
  • ⁇ is the focus factor
  • p ri is the The predicted probability value of the i-th type defect of the r-th slice sample in the second training set
  • y ri is the true probability value of the i-th type defect of the r-th slice sample in the second training set
  • q ri is the r-th slice in the second training set
  • the first detection step input the feature set into the up-sampling branch of the slice defect detection model to obtain each defect area of each slice in the slice set to be inspected and the first defect category distribution table corresponding to each defect area, and The defect category with the largest first probability value in the first defect category distribution table corresponding to each defect area is used as the predicted defect category corresponding to each defect area.
  • the output of the up-sampling branch of the slice defect detection model is each defect area of each slice in the slice set to be inspected and the first defect category distribution table corresponding to each defect area, and the first defect category distribution table is The defect category with the largest first probability value is used as the predicted defect category.
  • the first defect category distribution table corresponding to defective area 1 is: wrinkle 0.1, knife mark 0.2, impurity 0.1, bubble 0.6, and defective area 2 corresponds to The first defect category distribution table is: wrinkle 0.2, knife mark 0.2, impurity 0.5, bubble 0.1, then the predicted defect category of defect area 1 of slice 1 is bubble, and the predicted defect category of defect area 2 is impurity.
  • the second detection step input the feature set into the classification branch of the slice defect detection model to obtain a second defect category distribution table corresponding to each slice in the slice set to be detected.
  • Defect determining step obtaining the second probability value of the predicted defect category in the second defect category distribution table corresponding to each defect area of each slice in the slice set to be inspected, when the second probability value is greater than a preset threshold
  • the predicted defect category corresponding to each defect area is taken as the target defect category corresponding to each defect area
  • each defect area of each slice in the slice set to be inspected and the target defect category corresponding to each defect area are summarized , Obtain each defect area of the slice to be inspected and the target defect category corresponding to each defect area.
  • the storage method of the blockchain system is different from the storage method of traditional information items.
  • the storage of the blockchain project is that the nodes participating in the accounting maintain the same ledger content, and each accounting node can view the data on the chain. Therefore, wait The detection slices are stored on the blockchain for storage, which is more convenient to view, and at the same time, further ensures the privacy and security of other data such as the slices to be detected.
  • the defect area corresponding to the predicted defect category is deleted from the corresponding slice.
  • the second defect category distribution table corresponding to slice 1 is: wrinkle 0.1, knife mark 0.3, impurity 0.4, bubble 0.2, then the bubble defect corresponding to defect area 1 of slice 1 has the probability in the second defect category distribution table: 0.2;
  • the impurity defect corresponding to defect area 2 has a probability of 0.4 in the respective graph of the second defect category.
  • the preset threshold is 0.3
  • the impurity is regarded as the target defect category of defect area 2
  • the probability of bubbles corresponding to defect area 1 in the second defect category distribution table is only 0.2, which is less than the preset threshold, then the defect area 1 is considered defective.
  • the corresponding bubble defect is a false positive, and the defect area 1 and its predicted defect category are deleted.
  • the electronic device 1 proposed in this application firstly inputs the feature extraction branch of the trained slice defect detection model into the slice set to be detected to obtain the feature set of each slice in the slice set to be tested;
  • the feature set is input into the up-sampling branch of the slice defect detection model, and each defect area of each slice and the predicted defect category corresponding to each defect area are obtained; then, the feature set is input into the classification branch of the slice defect detection model to obtain the corresponding branch of each slice The second defect category distribution table; finally, the second probability value of the predicted defect category corresponding to each defect area of each slice in the second defect category distribution table is obtained.
  • each defect The predicted defect category corresponding to the area is used as the target defect category corresponding to each defect area.
  • the slice defect detection model in this application is generated by improving the u-net model, and the feature extraction branch of the u-net model is replaced by the residual module And the convolutional layer in the up-sampling branch to ensure the integrity of the information, making the features extracted by the feature extraction branch more complete, and the defect area restored by the up-sampling branch more accurate, and at the same time at the output of the feature extraction branch of the u-net model Adding the classification branch reduces the possibility of false positive output from the up-sampling branch, and makes the classification accuracy higher. Therefore, the present application improves the accuracy of the detection of the slice defect area and the defect category.
  • this is a block diagram of an embodiment of a slice defect detection apparatus 100 according to the present application.
  • the slice defect detection device 100 includes a feature extraction module 110, a first detection module 120, a second detection module 130, and a defect determination module 140.
  • a feature extraction module 110 extracts a feature from a feature from a feature extraction module 110.
  • a first detection module 120 detects a defect from a defect detection module.
  • a second detection module 130 detects a defect from a defect detection module.
  • the feature extraction module 110 is configured to receive a slice to be detected submitted by a user, perform segmentation processing on the slice to be detected, to obtain a slice set to be detected, and input the slice set to be detected into the trained slice defect detection model.
  • the first detection module 120 is configured to input the feature set into the up-sampling branch of the slice defect detection model to obtain each defect area of each slice in the slice set to be inspected and the first defect corresponding to each defect area A category distribution table, using the defect category with the largest first probability value in the first defect category distribution table corresponding to each defect area as the predicted defect category corresponding to each defect area;
  • the second detection module 130 is configured to input the feature set into the classification branch of the slice defect detection model to obtain a second defect category distribution table corresponding to each slice in the slice set to be detected;
  • the defect determination module 140 is configured to obtain the second probability value of the predicted defect category in the second defect category distribution table corresponding to each defect area of each slice in the to-be-detected slice set, when the second probability When the value is greater than the preset threshold, the predicted defect category corresponding to each defect area is taken as the target defect category corresponding to each defect area, and each defect area and each defect area corresponding to each slice in the slice set to be inspected
  • the target defect categories are summarized to obtain each defect area of the slice to be inspected and the target defect category corresponding to each defect area.
  • FIG. 5 it is a flowchart of an embodiment of a method for detecting a slice defect according to the present application.
  • the method for detecting a slice defect includes steps S1-S4.
  • S1 Receive the to-be-detected slice submitted by the user, perform segmentation processing on the to-be-detected slice to obtain the to-be-detected slice set, and input the to-be-detected slice set into the feature extraction branch of the trained slice defect detection model to obtain the The feature set of each slice in the slice set to be detected.
  • the slice to be detected is a digital image obtained by scanning a pathological slice by a scanner, and the defect types of the slice include wrinkles, knife marks, impurities, bubbles, and the like.
  • the performing segmentation processing on the slice to be detected to obtain a slice set to be detected includes:
  • A1 Divide the slice to be detected into a plurality of small slices of preset size by means of a sliding window
  • the 0th layer is the clearest, and the pixels are about 100000*100000.
  • the definition of the subsequent layers will be halved in turn.
  • the third layer of pathological slices (pixels Is 20000*20000) as the slice to be tested. Because the slice to be detected is too large to be directly input to the detection model, it is necessary to divide the slice to be detected into small slices, and the pixels of each small slice are about 512*512 ⁇ 1024*1024.
  • the image is cut according to the sliding window method. If the pixels of the last small slice obtained after splitting are insufficient, this small slice needs to be complemented, and the default size of the small slice is 512 *512 as an example, for the part where the edge of the last small slice is less than 512, it needs to be filled with a value of 0, that is, to supplement the black background.
  • Correction processing is performed on the multiple small slices of preset size by using gamma correction to obtain a slice set to be detected.
  • the brightness of the slice will be uneven or the biological target is too bright or too dark, so that the contrast of the slice is not high, and the contrast can be improved, which can greatly retain the characteristics of the biological target.
  • gamma correction is used to improve the contrast of slices, and the calculation formula is:
  • V out-i A ⁇ B ⁇ V in-i
  • V out-i is the pixel value after gamma correction of the i-th slice in the slice to be detected
  • A is the brightness parameter
  • B is the contrast parameter
  • V in-i is the original pixel value of the i-th slice in the slice to be detected .
  • the slice defect detection model is an improved u-net model.
  • the original u-net model has a U-shaped structure, including feature extraction branches and up-sampling branches.
  • the feature extraction branches are used to extract the features of the picture, and the up-sampling branches are used to restore the image accuracy.
  • the feature extraction branch is located on the left side of the U-shaped structure. It is a repetitive convolutional network architecture. In each repetition, there are 2 convolutional layers with a 3*3 convolution kernel, followed by an activation layer, and a 2*2 The max pooling layer with a step size of 2 is used for down-sampling. After each down-sampling, we double the number of feature channels.
  • the up-sampling branch is located on the right side of the U-shaped structure, including a 2*2 deconvolution (that is, transposed convolution, a type of convolution). Each time the deconvolution is used, the number of feature channels is halved, and the feature map Double the size. After the deconvolution, the result of the deconvolution is spliced with the corresponding feature map in the feature extraction branch.
  • the feature map in the feature extraction branch is slightly larger in size. It needs to be trimmed and spliced.
  • the spliced map is convolved twice and 3*3.
  • Each convolution is followed by a RELU.
  • the last layer is the convolution kernel.
  • the 1*1 convolutional layer is used to map the 64-channel feature map to a class label. There are 23 convolutional layers in the entire u-net network.
  • the construction process of the slice defect detection model includes:
  • the original u-net network only uses the convolutional layer and the pooling layer.
  • problems such as information loss and loss. It is easy to cause gradients to disappear or explode. How to deepen the network Ensuring that the information does not degenerate under the circumstances is the problem to be solved by this application.
  • the residual module includes two 3*3 convolutional layers, an activation layer, and a skip connection.
  • the skip connection directly detours the input information to the output.
  • the entire network only needs to learn the part of the input and output differences, not only The learning objectives and difficulty are simplified, and the integrity of the information is also ensured, so that the features extracted by the feature extraction branch are more complete, and the image restored by the upsampling branch is more accurate.
  • the original u-net model uses the last convolutional layer of the up-sampling branch to realize the recognition of the slice defect area and defect category. It classifies the single pixel of each slice in the slice set to be detected. Type detection of graphs.
  • the input of the classification branch of the slice defect detection model of this application is the output of the feature extraction branch.
  • the classification branch performs classification detection for all the features of each slice in the slice set to be detected, and pays more attention to the global characteristics of each slice.
  • the training process of the slice defect detection model includes:
  • the slice samples in the first training set and the second training set carry labeling information, and the labeling information includes each real defect area of the slice sample and a real defect type probability value (0 or 1) corresponding to each real defect area.
  • IOU ti is the intersection ratio of the i-th type defect of the t-th slice sample in the first training set
  • a ti is the predicted defect area corresponding to the i-th type defect of the t-th slice sample in the first training set
  • B ti is The real defect area corresponding to the i-th type defect of the t-th slice sample in the first training set
  • FL(q ri ) is the balanced cross-entropy loss value of the i-th defect of the r-th slice sample in the second training set
  • ⁇ i is the category weight parameter of the i-th defect
  • ⁇ is the focus factor
  • p ri is the The predicted probability value of the i-th type defect of the r-th slice sample in the second training set
  • y ri is the true probability value of the i-th type defect of the r-th slice sample in the second training set
  • q ri is the r-th slice in the second training set
  • the output of the up-sampling branch of the slice defect detection model is each defect area of each slice in the slice set to be inspected and the first defect category distribution table corresponding to each defect area, and the first defect category distribution table is The defect category with the largest first probability value is used as the predicted defect category.
  • the first defect category distribution table corresponding to defective area 1 is: wrinkle 0.1, knife mark 0.2, impurity 0.1, bubble 0.6, and defective area 2 corresponds to The first defect category distribution table is: wrinkle 0.2, knife mark 0.2, impurity 0.5, bubble 0.1, then the predicted defect category of defect area 1 of slice 1 is bubble, and the predicted defect category of defect area 2 is impurity.
  • the method further includes:
  • the defect area corresponding to the predicted defect category is deleted from the corresponding slice.
  • the second defect category distribution table corresponding to slice 1 is: wrinkle 0.1, knife mark 0.3, impurity 0.4, bubble 0.2, then the bubble defect corresponding to defect area 1 of slice 1 has the probability in the second defect category distribution table: 0.2;
  • the impurity defect corresponding to defect area 2 has a probability of 0.4 in the respective graph of the second defect category.
  • the preset threshold is 0.3
  • the impurity is regarded as the target defect category of defect area 2
  • the probability of bubbles corresponding to defect area 1 in the second defect category distribution table is only 0.2, which is less than the preset threshold, then the defect area 1 is considered defective.
  • the corresponding bubble defect is a false positive, and the defect area 1 and its predicted defect category are deleted.
  • the slice defect detection method proposed in this application firstly inputs the slice set to be tested into the feature extraction branch of the trained slice defect detection model to obtain the feature set of each slice in the slice set to be tested; then, Input the feature set into the up-sampling branch of the slice defect detection model to obtain each defect area of each slice and the predicted defect category corresponding to each defect area; then, input the feature set into the classification branch of the slice defect detection model to obtain the corresponding segment of each slice The second defect category distribution table; finally, the second probability value of the predicted defect category corresponding to each defect area of each slice in the second defect category distribution table is obtained.
  • each The predicted defect category corresponding to the defect area is used as the target defect category corresponding to each defect area.
  • the slice defect detection model in this application is generated by improving the u-net model, and the feature extraction of the u-net model is replaced by the residual module.
  • the convolutional layer in the branch and the up-sampling branch ensures the integrity of the information, makes the features extracted by the feature extraction branch more complete, and the defect area restored by the up-sampling branch is more accurate.
  • the output of the feature extraction branch in the u-net model The classification branch is added everywhere, which reduces the possibility of false positive output from the up-sampling branch, and makes the classification accuracy higher. Therefore, the present application improves the accuracy of the detection of the slice defect area and the defect category.
  • the embodiments of the present application also propose a computer-readable storage medium.
  • the computer-readable storage medium may be non-volatile or volatile.
  • the computer-readable storage medium may be a hard disk, a multimedia card, or an SD card. , Flash memory card, SMC, read only memory (ROM), erasable programmable read only memory (EPROM), portable compact disk read only memory (CD-ROM), USB memory, etc. any one or more of them random combination.
  • the computer-readable storage medium includes a slice defect detection program 10, and the slice defect detection program 10 implements the following operations when executed by a processor:
  • Receive the to-be-detected slice submitted by the user perform segmentation processing on the to-be-detected slice to obtain the to-be-detected slice set, and input the to-be-detected slice set into the feature extraction branch of the trained slice defect detection model to obtain the to-be-detected slice
  • the feature set of each slice in the slice set
  • the feature set is input into the up-sampling branch of the slice defect detection model to obtain each defect area of each slice in the slice set to be inspected and the first defect category distribution table corresponding to each defect area, and the each defect area
  • the defect category with the largest first probability value in the corresponding first defect category distribution table is used as the predicted defect category corresponding to each defect area;
  • the second probability value of the predicted defect category in the second defect category distribution table corresponding to each defect area of each slice in the slice set to be inspected and when the second probability value is greater than a preset threshold, all The predicted defect category corresponding to each defect area is used as the target defect category corresponding to each defect area, and each defect area of each slice in the slice set to be inspected and the target defect category corresponding to each defect area are summarized to obtain the Each defect area of the slice to be inspected and the target defect category corresponding to each defect area.
  • the technical solution of this application essentially or the part that contributes to the existing technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium (such as ROM/RAM, magnetic disk, The optical disc) includes several instructions to make a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) execute the methods described in the various embodiments of the present application.
  • a terminal device which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.

Abstract

一种切片缺陷检测方法、装置、电子设备以及可读存储介质，涉及人工智能中的智能决策技术领域。上述方法包括：将待检测切片集输入训练好的切片缺陷检测模型的特征提取分支，将得到的特征集输入切片缺陷检测模型的上采样分支，得到各个缺陷区域及各个缺陷区域对应的预测缺陷类别，将特征集输入切片缺陷检测模型的分类分支，得到第二缺陷类别分布表，获取预测缺陷类别在第二缺陷类别分布表中的第二概率值，当第二概率值大于预设阈值时，将预测缺陷类别作为目标缺陷类别。上述方法提高了切片缺陷区域、缺陷类别检测的准确度。另外，上述方法还涉及区块链技术，可应用于智慧医疗领域中，从而推动智慧城市的建设。

Description

切片缺陷检测方法、装置、电子设备及可读存储介质
本申请要求于2020年4月27日提交中国专利局、申请号为CN202010343021.9、发明名称为“切片缺陷检测方法、电子装置及可读存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及智能决策技术领域，尤其涉及一种切片缺陷检测方法、装置、电子设备及可读存储介质。
背景技术
医学图像切片对三维定位、三维可视化、手术规划和计算机辅助诊断等具有重要的意义，切片的质量直接影响到诊断效率和诊断质量，高质量的切片是正确的病理诊断至关重要的基础和保证，为提升切片质量，通常需要对缺陷类切片进行缺陷区域检测、分类，用以针对性的进行改善。
目前通常使用u-net网络对缺陷类切片进行缺陷检测，发明人意识到由于u-net网络只用到了卷积层、池化层，随着网络的加深，信息会逐渐损失，容易出现梯度消失或梯度爆炸的情况，导致无法精确的识别到缺陷区域；并且u-net网络主要是针对单个像素进行分类，缺少对全图的类别检测，而使得缺陷分类准确度不高。因此，亟需一种切片缺陷检测方法，以提高缺陷区域、缺陷类别检测准确度。
发明内容
鉴于以上内容，有必要提供一种切片缺陷检测方法，旨在提高切片缺陷区域、缺陷类别检测准确度。
本申请提供的切片缺陷检测方法，包括：
特征提取步骤：接收用户提交的待检测切片，对所述待检测切片进行切分处理，得到待检测切片集，将所述待检测切片集输入训练好的切片缺陷检测模型的特征提取分支，得到所述待检测切片集中每张切片的特征集；
第一检测步骤：将所述特征集输入所述切片缺陷检测模型的上采样分支，得到所述待检测切片集中每张切片的各个缺陷区域及各个缺陷区域对应的第一缺陷类别分布表，将所述各个缺陷区域对应的第一缺陷类别分布表中第一概率值最大的缺陷类别作为所述各个缺陷区域对应的预测缺陷类别；
第二检测步骤：将所述特征集输入所述切片缺陷检测模型的分类分支，得到所述待检测切片集中每张切片对应的第二缺陷类别分布表；
缺陷确定步骤：获取所述待检测切片集中每张切片的各个缺陷区域对应的预测缺陷类别在所述第二缺陷类别分布表中的第二概率值，当所述第二概率值大于预设阈值时，将所述各个缺陷区域对应的预测缺陷类别作为所述各个缺陷区域对应的目标缺陷类别，对所述待检测切片集中每张切片的各个缺陷区域及各个缺陷区域对应的目标缺陷类别进行汇总，得到所述待检测切片的各个缺陷区域及各个缺陷区域对应的目标缺陷类别。
本申请还提供一种切片缺陷检测装置，包括：
特征提取模块，用于接收用户提交的待检测切片，对所述待检测切片进行切分处理，得到待检测切片集，将所述待检测切片集输入训练好的切片缺陷检测模型的特征提取分支，得到所述待检测切片集中每张切片的特征集；
第一检测模块，用于将所述特征集输入所述切片缺陷检测模型的上采样分支，得到所 述待检测切片集中每张切片的各个缺陷区域及各个缺陷区域对应的第一缺陷类别分布表，将所述各个缺陷区域对应的第一缺陷类别分布表中第一概率值最大的缺陷类别作为所述各个缺陷区域对应的预测缺陷类别；
第二检测模块，用于将所述特征集输入所述切片缺陷检测模型的分类分支，得到所述待检测切片集中每张切片对应的第二缺陷类别分布表；
缺陷确定步骤：获取所述待检测切片集中每张切片的各个缺陷区域对应的预测缺陷类别在所述第二缺陷类别分布表中的第二概率值，当所述第二概率值大于预设阈值时，将所述各个缺陷区域对应的预测缺陷类别作为所述各个缺陷区域对应的目标缺陷类别，对所述待检测切片集中每张切片的各个缺陷区域及各个缺陷区域对应的目标缺陷类别进行汇总，得到所述待检测切片的各个缺陷区域及各个缺陷区域对应的目标缺陷类别。
此外，为实现上述目的，本申请还提供一种电子设备，该电子设备包括：存储器、处理器，所述存储器中存储有可在所述处理器上运行的切片缺陷检测程序，所述切片缺陷检测程序被所述处理器执行时实现如下步骤：
特征提取步骤：接收用户提交的待检测切片，对所述待检测切片进行切分处理，得到待检测切片集，将所述待检测切片集输入训练好的切片缺陷检测模型的特征提取分支，得到所述待检测切片集中每张切片的特征集；
第一检测步骤：将所述特征集输入所述切片缺陷检测模型的上采样分支，得到所述待检测切片集中每张切片的各个缺陷区域及各个缺陷区域对应的第一缺陷类别分布表，将所述各个缺陷区域对应的第一缺陷类别分布表中第一概率值最大的缺陷类别作为所述各个缺陷区域对应的预测缺陷类别；
第二检测步骤：将所述特征集输入所述切片缺陷检测模型的分类分支，得到所述待检测切片集中每张切片对应的第二缺陷类别分布表；
缺陷确定步骤：获取所述待检测切片集中每张切片的各个缺陷区域对应的预测缺陷类别在所述第二缺陷类别分布表中的第二概率值，当所述第二概率值大于预设阈值时，将所述各个缺陷区域对应的预测缺陷类别作为所述各个缺陷区域对应的目标缺陷类别，对所述待检测切片集中每张切片的各个缺陷区域及各个缺陷区域对应的目标缺陷类别进行汇总，得到所述待检测切片的各个缺陷区域及各个缺陷区域对应的目标缺陷类别。
此外，为实现上述目的，本申请还提供一种计算机可读存储介质，所述计算机可读存储介质上存储有切片缺陷检测程序，所述切片缺陷检测程序被一个或者多个处理器执行时实现如下步骤：
接收用户提交的待检测切片，对所述待检测切片进行切分处理，得到待检测切片集，将所述待检测切片集输入训练好的切片缺陷检测模型的特征提取分支，得到所述待检测切片集中每张切片的特征集；
将所述特征集输入所述切片缺陷检测模型的上采样分支，得到所述待检测切片集中每张切片的各个缺陷区域及各个缺陷区域对应的第一缺陷类别分布表，将所述各个缺陷区域对应的第一缺陷类别分布表中第一概率值最大的缺陷类别作为所述各个缺陷区域对应的预测缺陷类别；
将所述特征集输入所述切片缺陷检测模型的分类分支，得到所述待检测切片集中每张切片对应的第二缺陷类别分布表；
获取所述待检测切片集中每张切片的各个缺陷区域对应的预测缺陷类别在所述第二缺陷类别分布表中的第二概率值，当所述第二概率值大于预设阈值时，将所述各个缺陷区域对应的预测缺陷类别作为所述各个缺陷区域对应的目标缺陷类别，对所述待检测切片集中每张切片的各个缺陷区域及各个缺陷区域对应的目标缺陷类别进行汇总，得到所述待检测切片的各个缺陷区域及各个缺陷区域对应的目标缺陷类别。
相较现有技术，本申请首先通过将待检测切片集输入训练好的切片缺陷检测模型的特 征提取分支，得到待检测切片集中每张切片的特征集；接着，将特征集输入切片缺陷检测模型的上采样分支，得到每张切片的各个缺陷区域及各个缺陷区域对应的预测缺陷类别；然后，将特征集输入切片缺陷检测模型的分类分支，得到每张切片对应的第二缺陷类别分布表；最后，获取每张切片的各个缺陷区域对应的预测缺陷类别在第二缺陷类别分布表中的第二概率值，当第二概率值大于预设阈值时，将各个缺陷区域对应的预测缺陷类别作为各个缺陷区域对应的目标缺陷类别，本申请中的切片缺陷检测模型是通过对u-net模型改进而生成的，通过将残差模块替换u-net模型的特征提取分支及上采样分支中的卷积层，保证了信息的完整性，使得特征提取分支提取的特征更加完整，上采样分支还原的缺陷区域更加精准，同时在u-net模型的特征提取分支的输出处增加分类分支，减少了上采样分支输出假阳性的可能性，使得分类准确度更高，因此，本申请提高了切片缺陷区域、缺陷类别检测的准确度。
附图说明
图1为本申请电子设备一实施例的示意图；
图2为本申请切片缺陷检测模型一实施例的结构示意图；
图3为本申请残差模块一实施例的结构示意图；
图4为本申请切片缺陷检测装置一实施例的模块图；
图5为本申请切片缺陷检测方法一实施例的流程图。
具体实施方式
为了使本申请的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本申请进行进一步详细说明。应当理解，此处所描述的具体实施例仅用以解释本申请，并不用于限定本申请。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
需要说明的是，在本申请中涉及“第一”、“第二”等的描述仅用于描述目的，而不能理解为指示或暗示其相对重要性或者隐含指明所指示的技术特征的数量。由此，限定有“第一”、“第二”的特征可以明示或者隐含地包括至少一个该特征。另外，各个实施例之间的技术方案可以相互结合，但是必须是以本领域普通技术人员能够实现为基础，当技术方案的结合出现相互矛盾或无法实现时应当认为这种技术方案的结合不存在，也不在本申请要求的保护范围之内。
如图1所示，为本申请电子设备1一实施例的示意图。电子设备1是一种能够按照事先设定或者存储的指令，自动进行数值计算和/或信息处理的设备。所述电子设备1可以是计算机、也可以是单个网络服务器、多个网络服务器组成的服务器组或者基于云计算的由大量主机或者网络服务器构成的云，其中云计算是分布式计算的一种，由一群松散耦合的计算机集组成的一个超级虚拟计算机。
在本实施例中，电子设备1包括，但不仅限于，可通过系统总线相互通信连接的存储器11、处理器12、网络接口13，该存储器11中存储有切片缺陷检测程序10，所述切片缺陷检测程序10可被所述处理器12执行。图1仅示出了具有组件11-13以及切片缺陷检测程序10的电子设备1，本领域技术人员可以理解的是，图1示出的结构并不构成对电子设备1的限定，可以包括比图示更少或者更多的部件，或者组合某些部件，或者不同的部件布置。
其中，存储器11包括内存及至少一种类型的可读存储介质。内存为电子设备1的运行提供缓存；可读存储介质可为如闪存、硬盘、多媒体卡、卡型存储器(例如，SD或DX存储器等)、随机访问存储器(RAM)、静态随机访问存储器(SRAM)、只读存储器(ROM)、电可擦除可编程只读存储器(EEPROM)、可编程只读存储器(PROM)、磁性存储器、磁盘、光盘等的非易失性存储介质。在一些实施例中，可读存储介质可以是电子设备1的内 部存储单元，例如该电子设备1的硬盘；在另一些实施例中，该非易失性存储介质也可以是电子设备1的外部存储设备，例如电子设备1上配备的插接式硬盘，智能存储卡(Smart Media Card,SMC)，安全数字(Secure Digital,SD)卡，闪存卡(Flash Card)等。本实施例中，存储器11的可读存储介质通常用于存储安装于电子设备1的操作系统和各类应用软件，例如存储本申请一实施例中的切片缺陷检测程序10的代码等。此外，存储器11还可以用于暂时地存储已经输出或者将要输出的各类数据。
处理器12在一些实施例中可以是中央处理器(Central Processing Unit，CPU)、控制器、微控制器、微处理器、或其他数据处理芯片。该处理器12通常用于控制所述电子设备1的总体操作，例如执行与其他设备进行数据交互或者通信相关的控制和处理等。本实施例中，所述处理器12用于运行所述存储器11中存储的程序代码或者处理数据，例如运行切片缺陷检测程序10等。
网络接口13可包括无线网络接口或有线网络接口，该网络接口13用于在所述电子设备1与客户端(图中未画出)之间建立通信连接。
可选的，所述电子设备1还可以包括用户接口，用户接口可以包括显示器(Display)、输入单元比如键盘(Keyboard)，可选的用户接口还可以包括标准的有线接口、无线接口。可选的，在一些实施例中，显示器可以是LED显示器、液晶显示器、触控式液晶显示器以及OLED(Organic Light-Emitting Diode，有机发光二极管)触摸器等。其中，显示器也可以适当的称为显示屏或显示单元，用于显示在电子设备1中处理的信息以及用于显示可视化的用户界面。
在本申请的一实施例中，本申请主要基于人工智能技术中的图像处理、检测模型等技术进行设计实现，所述切片缺陷检测程序10被所述处理器12执行时实现如下特征提取步骤、第一检测步骤、第二检测步骤及缺陷确定步骤。
特征提取步骤：接收用户提交的待检测切片，对所述待检测切片进行切分处理，得到待检测切片集，将所述待检测切片集输入训练好的切片缺陷检测模型的特征提取分支，得到所述待检测切片集中每张切片的特征集。
本实施例中，所述待检测切片为扫描仪扫描病理切片得到的数字图像，切片的缺陷类别包括褶皱、刀痕、杂质、气泡等。
所述对所述待检测切片进行切分处理，得到待检测切片集包括：
A1、采用滑窗的方式将所述待检测切片切分为多个预设尺寸的小切片；
病理切片有多层，第0层最清晰，像素大概为100000*100000，后面层清晰度会依次减半，综合考虑图像清晰度和存储容量的因素，本实施例选择第三层病理切片(像素为20000*20000)作为待检测切片。因待检测切片太大，无法直接输入至检测模型，故需将待检测切片切分为小切片，每个小切片的像素大概在512*512～1024*1024。
本实施例中，切图是按照滑窗的方式进行的，若切分后得到的最后一张小切片的像素不足，需要对这张小切片进行补全，以小切片的预设尺寸为512*512为例，对最后一张小切片的边缘不足512的部分，需使用0值进行补齐，即补充黑色背景。
A2、采用gamma校正对所述多个预设尺寸的小切片进行校正处理，得到待检测切片集。
由于扫描设备、成像条件等因素的影响，会导致切片的亮度不均匀或者生物目标过亮或过暗，从而使得切片的对比度不高，提高对比度，可以极大的保留生物目标本身特性。
本实施例采用gamma校正来提升切片的对比度，其计算公式为：
V out-i＝A×B×V in-i
其中，V out-i为待检测切片集中第i张切片gamma校正后的像素值，A为明亮度参数，B为对比度参数，V in-i为待检测切片集中第i张切片的原始像素值。
如图2所示，为本申请切片缺陷检测模型一实施例的结构示意图。本实施例中，所述 切片缺陷检测模型为改进后的u-net模型。
原u-net模型为U型结构，包括特征提取分支和上采样分支，特征提取分支用于提取图片的特征，上采样分支用来还原图像精度。
特征提取分支位于U型结构的左侧，是一种重复的卷积网络架构，每次重复中有2个卷积核为3*3的卷积层，接着是一个激活层，一个2*2的步长为2的max pooling层，用于下采样，每一次下采样后我们都把特征通道的数量加倍。
上采样分支位于U型结构的右侧，包括一个2*2的反卷积(即转置卷积，卷积的一种)，每次使用反卷积都将特征通道数量减半，特征图大小加倍。反卷积过后，将反卷积的结果与特征提取分支中对应的特征图拼接起来。特征提取分支中的特征图尺寸稍大，需将其修剪后进行拼接，对拼接后的map进行2次3*3的卷积，每个卷积后跟一个RELU，最后一层为卷积核为1*1的卷积层，用来将64通道的特征图映射到一个类标签，整个u-net网络共23个卷积层。
本实施例中，所述切片缺陷检测模型的构建过程包括：
B1、用残差模块替换u-net模型的特征提取分支及上采样分支中的卷积层，所述残差模块包括两个卷积层、一个激活层及一个跳跃式连线；
原u-net网络只用到了卷积层、池化层，在传递信息时，随着网络的加深，存在信息丢失、损耗等问题，容易出现梯度消失或梯度爆炸的情况，如何在网络加深的情况下保证信息不退化是本申请要解决的问题。
如图3所示，为本申请残差模块一实施例的结构示意图。残差模块包括2个3*3的卷积层、一个激活层、一个跳跃式连线，跳跃式连接直接将输入信息绕道传到输出，整个网络只需要学习输入、输出差别的那一部分，不仅简化了学习目标和难度，还保证了信息的完整性，使得特征提取分支提取的特征更加完整，上采样分支还原的图像更为精准。
B2、在u-net模型的特征提取分支的输出处增加分类分支，使得所述切片缺陷检测模型为Y型结构，所述分类分支包括一个残差模块和两个全连接层。
原u-net模型通过上采样分支的最后一层卷积层来实现对切片缺陷区域和缺陷类别的识别，是针对待检测切片集中每张切片的单个像素进行分类的，缺少对每张切片全图的类别检测。
而本申请切片缺陷检测模型的分类分支的输入为特征提取分支的输出，分类分支是针对待检测切片集中每张切片的全部特征来做分类检测的，更为关注每张切片的全局特征。
本实施例中，所述切片缺陷检测模型的训练过程包括：
C1、将第一训练集中的切片样本输入所述切片缺陷检测模型，训练所述切片缺陷检测模型的特征提取分支和上采样分支，通过最小化第一损失函数确定所述切片缺陷检测模型的特征分支和上采样分支对应的权重参数，得到初级切片缺陷检测模型；
C2、将第二训练集中的切片样本输入所述初级切片缺陷检测模型，训练所述初级切片缺陷检测模型的分类分支，通过最小化第二损失函数确定所述切片缺陷检测模型的分类分支对应的权重参数，得到训练好的切片缺陷检测模型。
所述第一训练集及第二训练集中的切片样本携带有标注信息，所述标注信息包括切片样本的各个真实缺陷区域及各个真实缺陷区域对应的真实缺陷类型概率值(0或1)。
所述第一损失函数的公式为：
Figure PCTCN2020098983-appb-000001
其中，IOU t-i为第一训练集中第t张切片样本的第i类缺陷的交并比值，A t-i为第一训练集中第t张切片样本的第i类缺陷对应的预测缺陷区域，B t-i为第一训练集中第t张切片样本的第i类缺陷对应的真实缺陷区域；
所述第二损失函数的公式为：
FL(q r-i)＝-α i(1-q r-i) γlog(q r-i)
Figure PCTCN2020098983-appb-000002
其中，FL(q r-i)为第二训练集中第r张切片样本的第i类缺陷的平衡交叉熵损失值，α i为第i类缺陷的类别权重参数，γ为聚焦因子，p r-i为第二训练集中第r张切片样本的第i类缺陷的预测概率值，y r-i为第二训练集中第r张切片样本的第i类缺陷的真实概率值，q r-i为第二训练集中第r张切片样本的第i类缺陷的概率调整参数。
第一检测步骤：将所述特征集输入所述切片缺陷检测模型的上采样分支，得到所述待检测切片集中每张切片的各个缺陷区域及各个缺陷区域对应的第一缺陷类别分布表，将所述各个缺陷区域对应的第一缺陷类别分布表中第一概率值最大的缺陷类别作为所述各个缺陷区域对应的预测缺陷类别。
本实施例中，切片缺陷检测模型的上采样分支输出的是，待检测切片集中每张切片的各个缺陷区域及各个缺陷区域对应的第一缺陷类别分布表，将所述第一缺陷类别分布表中第一概率值最大的缺陷类别作为预测缺陷类别。
例如，检测到待检测切片集中的切片1有缺陷区域1和缺陷区域2，缺陷区域1对应的第一缺陷类别分布表为：褶皱0.1、刀痕0.2、杂质0.1、气泡0.6，缺陷区域2对应的第一缺陷类别分布表为：褶皱0.2、刀痕0.2、杂质0.5、气泡0.1，则切片1的缺陷区域1的预测缺陷类别为气泡，缺陷区域2的预测缺陷类别为杂质。
第二检测步骤：将所述特征集输入所述切片缺陷检测模型的分类分支，得到所述待检测切片集中每张切片对应的第二缺陷类别分布表。
缺陷确定步骤：获取所述待检测切片集中每张切片的各个缺陷区域对应的预测缺陷类别在所述第二缺陷类别分布表中的第二概率值，当所述第二概率值大于预设阈值时，将所述各个缺陷区域对应的预测缺陷类别作为所述各个缺陷区域对应的目标缺陷类别，对所述待检测切片集中每张切片的各个缺陷区域及各个缺陷区域对应的目标缺陷类别进行汇总，得到所述待检测切片的各个缺陷区域及各个缺陷区域对应的目标缺陷类别。
在本申请的另一个实施例中，所述切片缺陷检测程序10被所述处理器12执行时还实现如下步骤：
将待检测切片存储于区块链节点中。也可将本方案过程中的其他数据存储在区块链中。区块链系统在存储上跟传统信息项目的存储方式不同，区块链项目的存储是参与记账的节点维护同样的账本内容，每个记账节点都能查看到链上数据，因此，待检测切片存储于区块链上进行存储，查看更加方便，同时，进一步保证待检测切片等其他数据的私密和安全性。
需知，本方案可应用于智慧医疗领域中，从而推动智慧城市的建设。在本申请的另一个实施例中，所述切片缺陷检测程序10被所述处理器12执行时还实现如下步骤：
若所述第二概率值小于预设阈值，则从对应的切片中删除所述预测缺陷类别对应的缺陷区域。
例如，切片1对应的第二缺陷类别分布表为：褶皱0.1、刀痕0.3、杂质0.4、气泡0.2，则切片1的缺陷区域1对应的气泡缺陷，在第二缺陷类别分布表中的概率为0.2；缺陷区域2对应的杂质缺陷，在第二缺陷类别分别图中的概率为0.4。假设预设阈值为0.3，则将杂质作为缺陷区域2的目标缺陷类别，而缺陷区域1对应的气泡在第二缺陷类别分布表中的概率仅为0.2，小于预设阈值，则认为缺陷区域1对应的气泡缺陷为假阳性，删除缺陷区域1及其预测缺陷类别。
由上述实施例可知，本申请提出的电子设备1，首先，通过将待检测切片集输入训练好的切片缺陷检测模型的特征提取分支，得到待检测切片集中每张切片的特征集；接着，将特征集输入切片缺陷检测模型的上采样分支，得到每张切片的各个缺陷区域及各个缺陷区域对应的预测缺陷类别；然后，将特征集输入切片缺陷检测模型的分类分支，得到每张 切片对应的第二缺陷类别分布表；最后，获取每张切片的各个缺陷区域对应的预测缺陷类别在第二缺陷类别分布表中的第二概率值，当第二概率值大于预设阈值时，将各个缺陷区域对应的预测缺陷类别作为各个缺陷区域对应的目标缺陷类别，本申请中的切片缺陷检测模型是通过对u-net模型改进而生成的，通过将残差模块替换u-net模型的特征提取分支及上采样分支中的卷积层，保证了信息的完整性，使得特征提取分支提取的特征更加完整，上采样分支还原的缺陷区域更加精准，同时在u-net模型的特征提取分支的输出处增加分类分支，减少了上采样分支输出假阳性的可能性，使得分类准确度更高，因此，本申请提高了切片缺陷区域、缺陷类别检测的准确度。
如图4所示，为本申请切片缺陷检测装置100一实施例的模块图。
在本申请的一个实施例中，切片缺陷检测装置100包括特征提取模块110、第一检测模块120、第二检测模块130及缺陷确定模块140，示例性地：
所述特征提取模块110，用于接收用户提交的待检测切片，对所述待检测切片进行切分处理，得到待检测切片集，将所述待检测切片集输入训练好的切片缺陷检测模型的特征提取分支，得到所述待检测切片集中每张切片的特征集；
所述第一检测模块120，用于将所述特征集输入所述切片缺陷检测模型的上采样分支，得到所述待检测切片集中每张切片的各个缺陷区域及各个缺陷区域对应的第一缺陷类别分布表，将所述各个缺陷区域对应的第一缺陷类别分布表中第一概率值最大的缺陷类别作为所述各个缺陷区域对应的预测缺陷类别；
所述第二检测模块130，用于将所述特征集输入所述切片缺陷检测模型的分类分支，得到所述待检测切片集中每张切片对应的第二缺陷类别分布表；
所述缺陷确定模块140，用于获取所述待检测切片集中每张切片的各个缺陷区域对应的预测缺陷类别在所述第二缺陷类别分布表中的第二概率值，当所述第二概率值大于预设阈值时，将所述各个缺陷区域对应的预测缺陷类别作为所述各个缺陷区域对应的目标缺陷类别，对所述待检测切片集中每张切片的各个缺陷区域及各个缺陷区域对应的目标缺陷类别进行汇总，得到所述待检测切片的各个缺陷区域及各个缺陷区域对应的目标缺陷类别。
上述特征提取模块110、第一检测模块120、第二检测模块130及缺陷确定模块140等模块被执行时所实现的功能或操作步骤与上述实施例大体相同，在此不再赘述。
如图5所示，为本申请切片缺陷检测方法一实施例的流程图，该切片缺陷检测方法包括步骤S1-S4。
S1、接收用户提交的待检测切片，对所述待检测切片进行切分处理，得到待检测切片集，将所述待检测切片集输入训练好的切片缺陷检测模型的特征提取分支，得到所述待检测切片集中每张切片的特征集。
本实施例中，所述待检测切片为扫描仪扫描病理切片得到的数字图像，切片的缺陷类别包括褶皱、刀痕、杂质、气泡等。
所述对所述待检测切片进行切分处理，得到待检测切片集包括：
A1、采用滑窗的方式将所述待检测切片切分为多个预设尺寸的小切片；
病理切片有多层，第0层最清晰，像素大概为100000*100000，后面层清晰度会依次减半，综合考虑图像清晰度和存储容量的因素，本实施例选择第三层病理切片(像素为20000*20000)作为待检测切片。因待检测切片太大，无法直接输入至检测模型，故需将待检测切片切分为小切片，每个小切片的像素大概在512*512～1024*1024。
本实施例中，切图是按照滑窗的方式进行的，若切分后得到的最后一张小切片的像素不足，需要对这张小切片进行补全，以小切片的预设尺寸为512*512为例，对最后一张小切片的边缘不足512的部分，需使用0值进行补齐，即补充黑色背景。
A2、采用gamma校正对所述多个预设尺寸的小切片进行校正处理，得到待检测切片集。
由于扫描设备、成像条件等因素的影响，会导致切片的亮度不均匀或者生物目标过亮或过暗，从而使得切片的对比度不高，提高对比度，可以极大的保留生物目标本身特性。
本实施例采用gamma校正来提升切片的对比度，其计算公式为：
V out-i＝A×B×V in-i
其中，V out-i为待检测切片集中第i张切片gamma校正后的像素值，A为明亮度参数，B为对比度参数，V in-i为待检测切片集中第i张切片的原始像素值。
本实施例中，所述切片缺陷检测模型为改进后的u-net模型。
原u-net模型为U型结构，包括特征提取分支和上采样分支，特征提取分支用于提取图片的特征，上采样分支用来还原图像精度。
特征提取分支位于U型结构的左侧，是一种重复的卷积网络架构，每次重复中有2个卷积核为3*3的卷积层，接着是一个激活层，一个2*2的步长为2的max pooling层，用于下采样，每一次下采样后我们都把特征通道的数量加倍。
上采样分支位于U型结构的右侧，包括一个2*2的反卷积(即转置卷积，卷积的一种)，每次使用反卷积都将特征通道数量减半，特征图大小加倍。反卷积过后，将反卷积的结果与特征提取分支中对应的特征图拼接起来。特征提取分支中的特征图尺寸稍大，需将其修剪后进行拼接，对拼接后的map进行2次3*3的卷积，每个卷积后跟一个RELU，最后一层为卷积核为1*1的卷积层，用来将64通道的特征图映射到一个类标签，整个u-net网络共23个卷积层。
本实施例中，所述切片缺陷检测模型的构建过程包括：
B1、用残差模块替换u-net模型的特征提取分支及上采样分支中的卷积层，所述残差模块包括两个卷积层、一个激活层及一个跳跃式连线；
原u-net网络只用到了卷积层、池化层，在传递信息时，随着网络的加深，存在信息丢失、损耗等问题，容易出现梯度消失或梯度爆炸的情况，如何在网络加深的情况下保证信息不退化是本申请要解决的问题。
残差模块包括2个3*3的卷积层、一个激活层、一个跳跃式连线，跳跃式连接直接将输入信息绕道传到输出，整个网络只需要学习输入、输出差别的那一部分，不仅简化了学习目标和难度，还保证了信息的完整性，使得特征提取分支提取的特征更加完整，上采样分支还原的图像更为精准。
B2、在u-net模型的特征提取分支的输出处增加分类分支，使得所述切片缺陷检测模型为Y型结构，所述分类分支包括一个残差模块和两个全连接层。
原u-net模型通过上采样分支的最后一层卷积层来实现对切片缺陷区域和缺陷类别的识别，是针对待检测切片集中每张切片的单个像素进行分类的，缺少对每张切片全图的类别检测。
而本申请切片缺陷检测模型的分类分支的输入为特征提取分支的输出，分类分支是针对待检测切片集中每张切片的全部特征来做分类检测的，更为关注每张切片的全局特征。
本实施例中，所述切片缺陷检测模型的训练过程包括：
C1、将第一训练集中的切片样本输入所述切片缺陷检测模型，训练所述切片缺陷检测模型的特征提取分支和上采样分支，通过最小化第一损失函数确定所述切片缺陷检测模型的特征分支和上采样分支对应的权重参数，得到初级切片缺陷检测模型；
C2、将第二训练集中的切片样本输入所述初级切片缺陷检测模型，训练所述初级切片缺陷检测模型的分类分支，通过最小化第二损失函数确定所述切片缺陷检测模型的分类分支对应的权重参数，得到训练好的切片缺陷检测模型。
所述第一训练集及第二训练集中的切片样本携带有标注信息，所述标注信息包括切片样本的各个真实缺陷区域及各个真实缺陷区域对应的真实缺陷类型概率值(0或1)。
所述第一损失函数的公式为：
Figure PCTCN2020098983-appb-000003
其中，IOU t-i为第一训练集中第t张切片样本的第i类缺陷的交并比值，A t-i为第一训练集中第t张切片样本的第i类缺陷对应的预测缺陷区域，B t-i为第一训练集中第t张切片样本的第i类缺陷对应的真实缺陷区域；
所述第二损失函数的公式为：
FL(q r-i)＝-α i(1-q r-i) γlog(q r-i)
Figure PCTCN2020098983-appb-000004
其中，FL(q r-i)为第二训练集中第r张切片样本的第i类缺陷的平衡交叉熵损失值，α i为第i类缺陷的类别权重参数，γ为聚焦因子，p r-i为第二训练集中第r张切片样本的第i类缺陷的预测概率值，y r-i为第二训练集中第r张切片样本的第i类缺陷的真实概率值，q r-i为第二训练集中第r张切片样本的第i类缺陷的概率调整参数。
S2、将所述特征集输入所述切片缺陷检测模型的上采样分支，得到所述待检测切片集中每张切片的各个缺陷区域及各个缺陷区域对应的第一缺陷类别分布表，将所述各个缺陷区域对应的第一缺陷类别分布表中第一概率值最大的缺陷类别作为所述各个缺陷区域对应的预测缺陷类别。
本实施例中，切片缺陷检测模型的上采样分支输出的是，待检测切片集中每张切片的各个缺陷区域及各个缺陷区域对应的第一缺陷类别分布表，将所述第一缺陷类别分布表中第一概率值最大的缺陷类别作为预测缺陷类别。
例如，检测到待检测切片集中的切片1有缺陷区域1和缺陷区域2，缺陷区域1对应的第一缺陷类别分布表为：褶皱0.1、刀痕0.2、杂质0.1、气泡0.6，缺陷区域2对应的第一缺陷类别分布表为：褶皱0.2、刀痕0.2、杂质0.5、气泡0.1，则切片1的缺陷区域1的预测缺陷类别为气泡，缺陷区域2的预测缺陷类别为杂质。
S3、将所述特征集输入所述切片缺陷检测模型的分类分支，得到所述待检测切片集中每张切片对应的第二缺陷类别分布表。
S4、获取所述待检测切片集中每张切片的各个缺陷区域对应的预测缺陷类别在所述第二缺陷类别分布表中的第二概率值，当所述第二概率值大于预设阈值时，将所述各个缺陷区域对应的预测缺陷类别作为所述各个缺陷区域对应的目标缺陷类别，对所述待检测切片集中每张切片的各个缺陷区域及各个缺陷区域对应的目标缺陷类别进行汇总，得到所述待检测切片的各个缺陷区域及各个缺陷区域对应的目标缺陷类别。
在本申请的另一个实施例中，所述方法还包括：
若所述第二概率值小于预设阈值，则从对应的切片中删除所述预测缺陷类别对应的缺陷区域。
例如，切片1对应的第二缺陷类别分布表为：褶皱0.1、刀痕0.3、杂质0.4、气泡0.2，则切片1的缺陷区域1对应的气泡缺陷，在第二缺陷类别分布表中的概率为0.2；缺陷区域2对应的杂质缺陷，在第二缺陷类别分别图中的概率为0.4。假设预设阈值为0.3，则将杂质作为缺陷区域2的目标缺陷类别，而缺陷区域1对应的气泡在第二缺陷类别分布表中的概率仅为0.2，小于预设阈值，则认为缺陷区域1对应的气泡缺陷为假阳性，删除缺陷区域1及其预测缺陷类别。
由上述实施例可知，本申请提出的切片缺陷检测方法，首先，通过将待检测切片集输入训练好的切片缺陷检测模型的特征提取分支，得到待检测切片集中每张切片的特征集；接着，将特征集输入切片缺陷检测模型的上采样分支，得到每张切片的各个缺陷区域及各个缺陷区域对应的预测缺陷类别；然后，将特征集输入切片缺陷检测模型的分类分支，得到每张切片对应的第二缺陷类别分布表；最后，获取每张切片的各个缺陷区域对应的预测缺陷类别在第二缺陷类别分布表中的第二概率值，当第二概率值大于预设阈值时，将各个 缺陷区域对应的预测缺陷类别作为各个缺陷区域对应的目标缺陷类别，本申请中的切片缺陷检测模型是通过对u-net模型改进而生成的，通过将残差模块替换u-net模型的特征提取分支及上采样分支中的卷积层，保证了信息的完整性，使得特征提取分支提取的特征更加完整，上采样分支还原的缺陷区域更加精准，同时在u-net模型的特征提取分支的输出处增加分类分支，减少了上采样分支输出假阳性的可能性，使得分类准确度更高，因此，本申请提高了切片缺陷区域、缺陷类别检测的准确度。
此外，本申请实施例还提出一种计算机可读存储介质，所述计算机可读存储介质可以是非易失性，也可以是易失性，计算机可读存储介质可以是硬盘、多媒体卡、SD卡、闪存卡、SMC、只读存储器(ROM)、可擦除可编程只读存储器(EPROM)、便携式紧致盘只读存储器(CD-ROM)、USB存储器等中的任意一种或者几种的任意组合。计算机可读存储介质中包括切片缺陷检测程序10，所述切片缺陷检测程序10被处理器执行时实现如下操作：
接收用户提交的待检测切片，对所述待检测切片进行切分处理，得到待检测切片集，将所述待检测切片集输入训练好的切片缺陷检测模型的特征提取分支，得到所述待检测切片集中每张切片的特征集；
将所述特征集输入所述切片缺陷检测模型的上采样分支，得到所述待检测切片集中每张切片的各个缺陷区域及各个缺陷区域对应的第一缺陷类别分布表，将所述各个缺陷区域对应的第一缺陷类别分布表中第一概率值最大的缺陷类别作为所述各个缺陷区域对应的预测缺陷类别；
将所述特征集输入所述切片缺陷检测模型的分类分支，得到所述待检测切片集中每张切片对应的第二缺陷类别分布表；
获取所述待检测切片集中每张切片的各个缺陷区域对应的预测缺陷类别在所述第二缺陷类别分布表中的第二概率值，当所述第二概率值大于预设阈值时，将所述各个缺陷区域对应的预测缺陷类别作为所述各个缺陷区域对应的目标缺陷类别，对所述待检测切片集中每张切片的各个缺陷区域及各个缺陷区域对应的目标缺陷类别进行汇总，得到所述待检测切片的各个缺陷区域及各个缺陷区域对应的目标缺陷类别。
本申请之计算机可读存储介质的具体实施方式与上述切片缺陷检测方法以及电子设备1的具体实施方式大致相同，在此不再赘述。
上述本申请实施例序号仅仅为了描述，不代表实施例的优劣。
需要说明的是，在本文中，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、装置、物品或者方法不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、装置、物品或者方法所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括该要素的过程、装置、物品或者方法中还存在另外的相同要素。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到上述实施例方法可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件，但很多情况下前者是更佳的实施方式。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质(如ROM/RAM、磁碟、光盘)中，包括若干指令用以使得一台终端设备(可以是手机，计算机，服务器，空调器，或者网络设备等)执行本申请各个实施例所述的方法。
以上仅为本申请的优选实施例，并非因此限制本申请的专利范围，凡是利用本申请说明书及附图内容所作的等效结构或等效流程变换，或直接或间接运用在其他相关的技术领域，均同理包括在本申请的专利保护范围内。

Claims (20)

  1. 一种切片缺陷检测方法，应用于电子设备，其中，所述方法包括：
    特征提取步骤：接收用户提交的待检测切片，对所述待检测切片进行切分处理，得到待检测切片集，将所述待检测切片集输入训练好的切片缺陷检测模型的特征提取分支，得到所述待检测切片集中每张切片的特征集；
    第一检测步骤：将所述特征集输入所述切片缺陷检测模型的上采样分支，得到所述待检测切片集中每张切片的各个缺陷区域及各个缺陷区域对应的第一缺陷类别分布表，将所述各个缺陷区域对应的第一缺陷类别分布表中第一概率值最大的缺陷类别作为所述各个缺陷区域对应的预测缺陷类别；
    第二检测步骤：将所述特征集输入所述切片缺陷检测模型的分类分支，得到所述待检测切片集中每张切片对应的第二缺陷类别分布表；
    缺陷确定步骤：获取所述待检测切片集中每张切片的各个缺陷区域对应的预测缺陷类别在所述第二缺陷类别分布表中的第二概率值，当所述第二概率值大于预设阈值时，将所述各个缺陷区域对应的预测缺陷类别作为所述各个缺陷区域对应的目标缺陷类别，对所述待检测切片集中每张切片的各个缺陷区域及各个缺陷区域对应的目标缺陷类别进行汇总，得到所述待检测切片的各个缺陷区域及各个缺陷区域对应的目标缺陷类别。
  2. 如权利要求1所述的切片缺陷检测方法，其中，所述切片缺陷检测模型的构建过程包括：
    用残差模块替换u-net模型的特征提取分支及上采样分支中的卷积层，所述残差模块包括两个卷积层、一个激活层及一个跳跃式连线；
    在u-net模型的特征提取分支的输出处增加分类分支，使得所述切片缺陷检测模型为Y型结构，所述分类分支包括一个残差模块和两个全连接层。
  3. 如权利要求2所述的切片缺陷检测方法，其中，所述切片缺陷检测模型的训练过程包括：
    将第一训练集中的切片样本输入所述切片缺陷检测模型，训练所述切片缺陷检测模型的特征提取分支和上采样分支，通过最小化第一损失函数确定所述切片缺陷检测模型的特征分支和上采样分支对应的权重参数，得到初级切片缺陷检测模型；
    将第二训练集中的切片样本输入所述初级切片缺陷检测模型，训练所述初级切片缺陷检测模型的分类分支，通过最小化第二损失函数确定所述切片缺陷检测模型的分类分支对应的权重参数，得到训练好的切片缺陷检测模型。
  4. 如权利要求3所述的切片缺陷检测方法，其中，所述第一损失函数的公式为：
    Figure PCTCN2020098983-appb-100001
    其中，IOU t-i为第一训练集中第t张切片样本的第i类缺陷的交并比值，A t-i为第一训练集中第t张切片样本的第i类缺陷对应的预测缺陷区域，B t-i为第一训练集中第t张切片样本的第i类缺陷对应的真实缺陷区域；
    所述第二损失函数的公式为：
    FL(q r-i)＝-α i(1-q r-i) γlog(q r-i)
    Figure PCTCN2020098983-appb-100002
    其中，FL(q r-i)为第二训练集中第r张切片样本的第i类缺陷的平衡交叉熵损失值，α i为第i类缺陷的类别权重参数，γ为聚焦因子，p r-i为第二训练集中第r张切片样本的第i类缺陷的预测概率值，y r-i为第二训练集中第r张切片样本的第i类缺陷的真实概率值，q r-i为第二训练集中第r张切片样本的第i类缺陷的概率调整参数。
  5. 如权利要求1-4任一项所述的切片缺陷检测方法，其中，在获取所述待检测切片集 中每张切片的各个缺陷区域对应的预测缺陷类别在所述第二缺陷类别分布表中的第二概率值之后，所述方法还包括：
    若所述第二概率值小于预设阈值，则从对应的切片中删除所述预测缺陷类别对应的缺陷区域。
  6. 如权利要求5所述的切片缺陷检测方法，其中，所述对所述待检测切片进行切分处理，得到待检测切片集包括：
    采用滑窗的方式将所述待检测切片切分为多个预设尺寸的小切片；
    采用gamma校正对所述多个预设尺寸的小切片进行校正处理，得到待检测切片集。
  7. 如权利要求6所述的切片缺陷检测方法，其中，所述gamma校正的计算公式为：
    V out-i＝A×B×V in-i
    其中，V out-i为待检测切片集中第i张切片gamma校正后的像素值，A为明亮度参数，B为对比度参数，V in-i为待检测切片集中第i张切片的原始像素值。
  8. 一种切片缺陷检测装置，其中，包括：
    特征提取模块，用于接收用户提交的待检测切片，对所述待检测切片进行切分处理，得到待检测切片集，将所述待检测切片集输入训练好的切片缺陷检测模型的特征提取分支，得到所述待检测切片集中每张切片的特征集；
    第一检测模块，用于将所述特征集输入所述切片缺陷检测模型的上采样分支，得到所述待检测切片集中每张切片的各个缺陷区域及各个缺陷区域对应的第一缺陷类别分布表，将所述各个缺陷区域对应的第一缺陷类别分布表中第一概率值最大的缺陷类别作为所述各个缺陷区域对应的预测缺陷类别；
    第二检测模块，用于将所述特征集输入所述切片缺陷检测模型的分类分支，得到所述待检测切片集中每张切片对应的第二缺陷类别分布表；
    缺陷确定模块，用于获取所述待检测切片集中每张切片的各个缺陷区域对应的预测缺陷类别在所述第二缺陷类别分布表中的第二概率值，当所述第二概率值大于预设阈值时，将所述各个缺陷区域对应的预测缺陷类别作为所述各个缺陷区域对应的目标缺陷类别，对所述待检测切片集中每张切片的各个缺陷区域及各个缺陷区域对应的目标缺陷类别进行汇总，得到所述待检测切片的各个缺陷区域及各个缺陷区域对应的目标缺陷类别。
  9. 一种电子设备，其中，该电子设备包括：存储器、处理器，所述存储器上存储有可在所述处理器上运行的切片缺陷检测程序，所述切片缺陷检测程序被所述处理器执行时实现如下步骤：
    特征提取步骤：接收用户提交的待检测切片，对所述待检测切片进行切分处理，得到待检测切片集，将所述待检测切片集输入训练好的切片缺陷检测模型的特征提取分支，得到所述待检测切片集中每张切片的特征集；
    第一检测步骤：将所述特征集输入所述切片缺陷检测模型的上采样分支，得到所述待检测切片集中每张切片的各个缺陷区域及各个缺陷区域对应的第一缺陷类别分布表，将所述各个缺陷区域对应的第一缺陷类别分布表中第一概率值最大的缺陷类别作为所述各个缺陷区域对应的预测缺陷类别；
    第二检测步骤：将所述特征集输入所述切片缺陷检测模型的分类分支，得到所述待检测切片集中每张切片对应的第二缺陷类别分布表；
    缺陷确定步骤：获取所述待检测切片集中每张切片的各个缺陷区域对应的预测缺陷类别在所述第二缺陷类别分布表中的第二概率值，当所述第二概率值大于预设阈值时，将所述各个缺陷区域对应的预测缺陷类别作为所述各个缺陷区域对应的目标缺陷类别，对所述待检测切片集中每张切片的各个缺陷区域及各个缺陷区域对应的目标缺陷类别进行汇总，得到所述待检测切片的各个缺陷区域及各个缺陷区域对应的目标缺陷类别。
  10. 如权利要求9所述的电子设备，其中，所述切片缺陷检测模型的构建过程包括：
    用残差模块替换u-net模型的特征提取分支及上采样分支中的卷积层，所述残差模块包括两个卷积层、一个激活层及一个跳跃式连线；
    在u-net模型的特征提取分支的输出处增加分类分支，使得所述切片缺陷检测模型为Y型结构，所述分类分支包括一个残差模块和两个全连接层。
  11. 如权利要求10所述的电子设备，其中，所述切片缺陷检测模型的训练过程包括：
    将第一训练集中的切片样本输入所述切片缺陷检测模型，训练所述切片缺陷检测模型的特征提取分支和上采样分支，通过最小化第一损失函数确定所述切片缺陷检测模型的特征分支和上采样分支对应的权重参数，得到初级切片缺陷检测模型；
    将第二训练集中的切片样本输入所述初级切片缺陷检测模型，训练所述初级切片缺陷检测模型的分类分支，通过最小化第二损失函数确定所述切片缺陷检测模型的分类分支对应的权重参数，得到训练好的切片缺陷检测模型。
  12. 如权利要求11所述的电子设备，其中，所述第一损失函数的公式为：
    Figure PCTCN2020098983-appb-100003
    其中，IOU t-i为第一训练集中第t张切片样本的第i类缺陷的交并比值，A t-i为第一训练集中第t张切片样本的第i类缺陷对应的预测缺陷区域，B t-i为第一训练集中第t张切片样本的第i类缺陷对应的真实缺陷区域；
    所述第二损失函数的公式为：
    FL(q r-i)＝-α i(1-q r-i) γlog(q r-i)
    Figure PCTCN2020098983-appb-100004
    其中，FL(q r-i)为第二训练集中第r张切片样本的第i类缺陷的平衡交叉熵损失值，α i为第i类缺陷的类别权重参数，γ为聚焦因子，p r-i为第二训练集中第r张切片样本的第i类缺陷的预测概率值，y r-i为第二训练集中第r张切片样本的第i类缺陷的真实概率值，q r-i为第二训练集中第r张切片样本的第i类缺陷的概率调整参数。
  13. 如权利要求9-12任一项所述的电子设备，其中，在获取所述待检测切片集中每张切片的各个缺陷区域对应的预测缺陷类别在所述第二缺陷类别分布表中的第二概率值之后，所述切片缺陷检测程序被所述处理器执行时还实现如下步骤：
    若所述第二概率值小于预设阈值，则从对应的切片中删除所述预测缺陷类别对应的缺陷区域。
  14. 如权利要求13所述的电子设备，其中，所述对所述待检测切片进行切分处理，得到待检测切片集包括：
    采用滑窗的方式将所述待检测切片切分为多个预设尺寸的小切片；
    采用gamma校正对所述多个预设尺寸的小切片进行校正处理，得到待检测切片集。
  15. 如权利要求14所述的电子设备，其中，所述gamma校正的计算公式为：
    V out-i＝A×B×V in-i
    其中，V out-i为待检测切片集中第i张切片gamma校正后的像素值，A为明亮度参数，B为对比度参数，V in-i为待检测切片集中第i张切片的原始像素值。
  16. 一种计算机可读存储介质，其中，所述计算机可读存储介质上存储有切片缺陷检测程序，所述切片缺陷检测程序被一个或者多个处理器执行时实现如下步骤：
    接收用户提交的待检测切片，对所述待检测切片进行切分处理，得到待检测切片集，将所述待检测切片集输入训练好的切片缺陷检测模型的特征提取分支，得到所述待检测切片集中每张切片的特征集；
    将所述特征集输入所述切片缺陷检测模型的上采样分支，得到所述待检测切片集中每张切片的各个缺陷区域及各个缺陷区域对应的第一缺陷类别分布表，将所述各个缺陷区域对应的第一缺陷类别分布表中第一概率值最大的缺陷类别作为所述各个缺陷区域对应的 预测缺陷类别；
    将所述特征集输入所述切片缺陷检测模型的分类分支，得到所述待检测切片集中每张切片对应的第二缺陷类别分布表；
    获取所述待检测切片集中每张切片的各个缺陷区域对应的预测缺陷类别在所述第二缺陷类别分布表中的第二概率值，当所述第二概率值大于预设阈值时，将所述各个缺陷区域对应的预测缺陷类别作为所述各个缺陷区域对应的目标缺陷类别，对所述待检测切片集中每张切片的各个缺陷区域及各个缺陷区域对应的目标缺陷类别进行汇总，得到所述待检测切片的各个缺陷区域及各个缺陷区域对应的目标缺陷类别。
  17. 如权利要求16所述的计算机可读存储介质，其中，所述切片缺陷检测模型的构建过程包括：
    用残差模块替换u-net模型的特征提取分支及上采样分支中的卷积层，所述残差模块包括两个卷积层、一个激活层及一个跳跃式连线；
    在u-net模型的特征提取分支的输出处增加分类分支，使得所述切片缺陷检测模型为Y型结构，所述分类分支包括一个残差模块和两个全连接层。
  18. 如权利要求17所述的计算机可读存储介质，其中，所述切片缺陷检测模型的训练过程包括：
    将第一训练集中的切片样本输入所述切片缺陷检测模型，训练所述切片缺陷检测模型的特征提取分支和上采样分支，通过最小化第一损失函数确定所述切片缺陷检测模型的特征分支和上采样分支对应的权重参数，得到初级切片缺陷检测模型；
    将第二训练集中的切片样本输入所述初级切片缺陷检测模型，训练所述初级切片缺陷检测模型的分类分支，通过最小化第二损失函数确定所述切片缺陷检测模型的分类分支对应的权重参数，得到训练好的切片缺陷检测模型。
  19. 如权利要求18所述的计算机可读存储介质，其中，所述第一损失函数的公式为：
    Figure PCTCN2020098983-appb-100005
    其中，IOU t-i为第一训练集中第t张切片样本的第i类缺陷的交并比值，A t-i为第一训练集中第t张切片样本的第i类缺陷对应的预测缺陷区域，B t-i为第一训练集中第t张切片样本的第i类缺陷对应的真实缺陷区域；
    所述第二损失函数的公式为：
    FL(q r-i)＝-α i(1-q r-i) γlog(q r-i)
    Figure PCTCN2020098983-appb-100006
    其中，FL(q r-i)为第二训练集中第r张切片样本的第i类缺陷的平衡交叉熵损失值，α i为第i类缺陷的类别权重参数，γ为聚焦因子，p r-i为第二训练集中第r张切片样本的第i类缺陷的预测概率值，y r-i为第二训练集中第r张切片样本的第i类缺陷的真实概率值，q r-i为第二训练集中第r张切片样本的第i类缺陷的概率调整参数。
  20. 如权利要求16-19任一项所述的计算机可读存储介质，其中，在获取所述待检测切片集中每张切片的各个缺陷区域对应的预测缺陷类别在所述第二缺陷类别分布表中的第二概率值之后，所述切片缺陷检测程序被一个或者多个处理器执行时还实现如下步骤：
    若所述第二概率值小于预设阈值，则从对应的切片中删除所述预测缺陷类别对应的缺陷区域。
PCT/CN2020/098983 2020-04-27 2020-06-29 切片缺陷检测方法、装置、电子设备及可读存储介质 WO2021217857A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010343021.9 2020-04-27
CN202010343021.9A CN111595850B (zh) 2020-04-27 2020-04-27 切片缺陷检测方法、电子装置及可读存储介质

Publications (1)

Publication Number Publication Date
WO2021217857A1 true WO2021217857A1 (zh) 2021-11-04

Family

ID=72182375

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/098983 WO2021217857A1 (zh) 2020-04-27 2020-06-29 切片缺陷检测方法、装置、电子设备及可读存储介质

Country Status (2)

Country Link
CN (1) CN111595850B (zh)
WO (1) WO2021217857A1 (zh)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114627089A (zh) * 2022-03-21 2022-06-14 成都数之联科技股份有限公司 缺陷识别方法、装置、计算机设备及计算机可读存储介质
CN115272310A (zh) * 2022-09-26 2022-11-01 江苏智云天工科技有限公司 工件的缺陷检测方法及装置
CN115984268A (zh) * 2023-03-20 2023-04-18 杭州百子尖科技股份有限公司 基于机器视觉的目标检测方法、装置、电子设备以及介质
CN116046790A (zh) * 2023-01-31 2023-05-02 北京百度网讯科技有限公司 缺陷检测方法、装置、系统、电子设备以及存储介质
CN116433664A (zh) * 2023-06-13 2023-07-14 成都数之联科技股份有限公司 面板缺陷检测方法、装置、存储介质、设备及程序产品
CN116452791A (zh) * 2023-03-27 2023-07-18 广州市斯睿特智能科技有限公司 多相机点位的缺陷区域定位方法、系统、装置及存储介质

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111915609B (zh) * 2020-09-22 2023-07-14 平安科技（深圳）有限公司 病灶检测分析方法、装置、电子设备及计算机存储介质
CN112184665A (zh) * 2020-09-27 2021-01-05 苏州嘉展科技有限公司 一种应用于纸塑行业的人工智能缺陷侦测系统
CN113743445A (zh) * 2021-07-15 2021-12-03 上海朋熙半导体有限公司 目标对象识别方法、装置、计算机设备和存储介质
CN117765533B (zh) * 2024-02-22 2024-04-26 天津医科大学第二医院 一种用于口腔黏膜癌预测的图像处理方法及系统

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109523521A (zh) * 2018-10-26 2019-03-26 复旦大学 基于多切片ct图像的肺结节分类和病灶定位方法和系统
WO2019057944A1 (en) * 2017-09-22 2019-03-28 F. Hoffmann-La Roche Ag ELIMINATION OF ARTIFACTS FROM IMAGES OF FABRIC
CN109829880A (zh) * 2018-12-07 2019-05-31 清影医疗科技（深圳）有限公司 一种基于深度学习的ct图像检测方法、装置和控制设备
CN110796637A (zh) * 2019-09-29 2020-02-14 郑州金惠计算机系统工程有限公司 图像缺陷检测模型的训练、测试方法、装置及存储介质
CN111008953A (zh) * 2019-10-29 2020-04-14 平安科技（深圳）有限公司 一种数字病理图像质控的方法及装置

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7379175B1 (en) * 2002-10-15 2008-05-27 Kla-Tencor Technologies Corp. Methods and systems for reticle inspection and defect review using aerial imaging
JP5081590B2 (ja) * 2007-11-14 2012-11-28 株式会社日立ハイテクノロジーズ 欠陥観察分類方法及びその装置
EP2566384A1 (en) * 2010-05-03 2013-03-13 STI Medical Systems, LLC Image analysis for cervical neoplasia detection and diagnosis
KR101759496B1 (ko) * 2015-08-20 2017-07-19 충북대학교 산학협력단 Ｐｃｂ 결함 및 결함종류 분류 시스템 및 방법
FR3045156B1 (fr) * 2015-12-11 2017-12-22 Soitec Silicon On Insulator Procede de detection de defauts et dispositif associe
KR20190111644A (ko) * 2018-03-23 2019-10-02 (주)브릭 반복 패턴 영상의 결함 유형 분류 장치 및 방법
EP3553740A1 (en) * 2018-04-13 2019-10-16 Koninklijke Philips N.V. Automatic slice selection in medical imaging
CN108802051B (zh) * 2018-07-10 2020-05-19 华南理工大学 一种柔性ic基板直线线路气泡及折痕缺陷检测系统及方法
CN111044525B (zh) * 2019-12-30 2021-10-29 歌尔股份有限公司 一种产品缺陷检测方法、装置与系统

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019057944A1 (en) * 2017-09-22 2019-03-28 F. Hoffmann-La Roche Ag ELIMINATION OF ARTIFACTS FROM IMAGES OF FABRIC
CN109523521A (zh) * 2018-10-26 2019-03-26 复旦大学 基于多切片ct图像的肺结节分类和病灶定位方法和系统
CN109829880A (zh) * 2018-12-07 2019-05-31 清影医疗科技（深圳）有限公司 一种基于深度学习的ct图像检测方法、装置和控制设备
CN110796637A (zh) * 2019-09-29 2020-02-14 郑州金惠计算机系统工程有限公司 图像缺陷检测模型的训练、测试方法、装置及存储介质
CN111008953A (zh) * 2019-10-29 2020-04-14 平安科技（深圳）有限公司 一种数字病理图像质控的方法及装置

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114627089A (zh) * 2022-03-21 2022-06-14 成都数之联科技股份有限公司 缺陷识别方法、装置、计算机设备及计算机可读存储介质
CN115272310A (zh) * 2022-09-26 2022-11-01 江苏智云天工科技有限公司 工件的缺陷检测方法及装置
CN115272310B (zh) * 2022-09-26 2023-01-03 江苏智云天工科技有限公司 工件的缺陷检测方法及装置
CN116046790A (zh) * 2023-01-31 2023-05-02 北京百度网讯科技有限公司 缺陷检测方法、装置、系统、电子设备以及存储介质
CN116046790B (zh) * 2023-01-31 2023-10-27 北京百度网讯科技有限公司 缺陷检测方法、装置、系统、电子设备以及存储介质
CN115984268A (zh) * 2023-03-20 2023-04-18 杭州百子尖科技股份有限公司 基于机器视觉的目标检测方法、装置、电子设备以及介质
CN115984268B (zh) * 2023-03-20 2023-06-30 杭州百子尖科技股份有限公司 基于机器视觉的目标检测方法、装置、电子设备以及介质
CN116452791A (zh) * 2023-03-27 2023-07-18 广州市斯睿特智能科技有限公司 多相机点位的缺陷区域定位方法、系统、装置及存储介质
CN116452791B (zh) * 2023-03-27 2024-03-22 广州市斯睿特智能科技有限公司 多相机点位的缺陷区域定位方法、系统、装置及存储介质
CN116433664A (zh) * 2023-06-13 2023-07-14 成都数之联科技股份有限公司 面板缺陷检测方法、装置、存储介质、设备及程序产品
CN116433664B (zh) * 2023-06-13 2023-09-01 成都数之联科技股份有限公司 面板缺陷检测方法、装置、存储介质、设备及程序产品

Also Published As

Publication number Publication date
CN111595850A (zh) 2020-08-28
CN111595850B (zh) 2022-07-15

Similar Documents

Publication Publication Date Title
WO2021217857A1 (zh) 切片缺陷检测方法、装置、电子设备及可读存储介质
CN110705583B (zh) 细胞检测模型训练方法、装置、计算机设备及存储介质
WO2019174130A1 (zh) 票据识别方法、服务器及计算机可读存储介质
CN110059697B (zh) 一种基于深度学习的肺结节自动分割方法
EP3358476A1 (en) Method and apparatus for constructing decision model, computer device and storage device
US11010883B2 (en) Automated analysis of petrographic thin section images using advanced machine learning techniques
CN109285174A (zh) 基于众包和深度学习的染色体分割和核型分析
RU2760471C1 (ru) Способы и системы идентификации полей в документе
WO2020151340A1 (zh) 一种目标细胞标记方法、装置、存储介质及终端设备
CN113837151B (zh) 表格图像处理方法、装置、计算机设备及可读存储介质
US10803586B1 (en) Image analysis in pathology
CN113065609B (zh) 图像分类方法、装置、电子设备及可读存储介质
CN111192678A (zh) 病理显微图像诊断、模型训练方法、装置、设备及介质
CN114445356A (zh) 基于多分辨率的全视野病理切片图像肿瘤快速定位方法
RU2732895C1 (ru) Метод для выделения и классификации типов клеток крови с помощью глубоких сверточных нейронных сетей
CN111768405A (zh) 处理标注图像的方法、装置、设备和存储介质
US20220207724A1 (en) Method of determining a distribution of stem cells in a cell image, electronic device, and storage medium
US11893784B2 (en) Assessment of image quality for optical character recognition using machine learning
CN114511523A (zh) 基于自监督学习的胃癌分子亚型分类方法及装置
CN111428770A (zh) 网络模型训练及电感缺陷识别方法、装置、电子设备
CN112464892B (zh) 票据区域识别方法、装置、电子设备及可读存储介质
CN112784189A (zh) 一种识别页面图像的方法和装置
CN111179226A (zh) 视野图的识别方法、装置及计算机存储介质
CN110458070A (zh) 基于机动车年检检验表图片识别检验次数的方法与系统
CN117576109B (zh) 一种缺陷检测方法、装置、设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20933301

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20933301

Country of ref document: EP

Kind code of ref document: A1