WO2021179484A1 - 多标签多示例的图像检测方法、装置、设备及存储介质 - Google Patents

多标签多示例的图像检测方法、装置、设备及存储介质 Download PDF

Info

Publication number
WO2021179484A1
WO2021179484A1 PCT/CN2020/098968 CN2020098968W WO2021179484A1 WO 2021179484 A1 WO2021179484 A1 WO 2021179484A1 CN 2020098968 W CN2020098968 W CN 2020098968W WO 2021179484 A1 WO2021179484 A1 WO 2021179484A1
Authority
WO
WIPO (PCT)
Prior art keywords
target
pathological
label
model
preset
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2020/098968
Other languages
English (en)
French (fr)
Inventor
梁志成
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ping An Technology Shenzhen Co Ltd
Original Assignee
Ping An Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ping An Technology Shenzhen Co Ltd filed Critical Ping An Technology Shenzhen Co Ltd
Publication of WO2021179484A1 publication Critical patent/WO2021179484A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/69—Microscopic objects, e.g. biological cells or cellular parts
    • G06V20/693—Acquisition
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G06F18/232—Non-hierarchical techniques
    • G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
    • G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
    • G06F18/24155—Bayesian classification
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/69—Microscopic objects, e.g. biological cells or cellular parts
    • G06V20/695—Preprocessing, e.g. image segmentation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/69—Microscopic objects, e.g. biological cells or cellular parts
    • G06V20/698—Matching; Classification
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/467—Encoded features or binary features, e.g. local binary patterns [LBP]

Definitions

  • This application relates to the technical field of region extraction, and in particular to a method, device, device, and storage medium for image detection with multiple tags and multiple instances.
  • a biopsy image is an image obtained by slicing the tissue of the patient's diseased part and performing magnified imaging under a microscope.
  • Biopsy images directly reflect the lesions that occur within the tissue, and are an important basis for doctors to diagnose diseases, and even the ultimate basis for diagnosis of certain serious diseases.
  • the imaging methods X-ray, CT, MRI, etc.
  • the living tissue of the lesion must be extracted, observed under the microscope and biochemical examination, this process is called biopsy.
  • the inventor realizes that in clinical diagnosis, doctors will analyze the entire pathological image to obtain a diagnosis report. In most cases, the diagnosis report will not clearly indicate which area has which pathological characteristics. In order to combine the characteristics in the diagnosis report Corresponding to different areas in the image, it is necessary to call the corresponding model for marking. N features require N models. The number of models used is large and the marking process is cumbersome.
  • This application provides an image detection method, device, device and storage medium with multiple labels and multiple examples, which reduces the analysis time of pathological images, improves analysis efficiency, and improves the accuracy of analysis results.
  • the first aspect of the embodiments of the present application provides a multi-label and multi-example image detection method, including: acquiring a pathological area of a confirmed biopsy image and a pathological feature description corresponding to the pathological area; according to the pathological area and the pathological area
  • the corresponding pathological feature description trains the offline training multi-label multi-instance model to obtain the preset multi-instance multi-label model; obtains the original biopsy image, the original biopsy image is used to indicate the pathological characteristics of the human tissue; the original biopsy The image is subjected to texture feature extraction, and the extracted texture features are structurally expressed to generate the target image structure; the preset training model is called to recognize the target image structure to obtain the target feature vector group; through the preset multiple
  • the example multi-label model labels the target feature vector group to obtain multiple target pathological labels.
  • the second aspect of the embodiments of the present application provides a multi-label and multi-example image detection device, including: a first acquisition unit for acquiring a pathological area of a confirmed biopsy image and a pathological feature description corresponding to the pathological area; a training unit , Used to train the offline training multi-label multi-example model according to the pathological area and the pathological feature description corresponding to the pathological area to obtain a preset multi-example multi-label model; the second acquisition unit is used to acquire the original biopsy image
  • the original biopsy image is used to indicate the pathological characteristics of the human tissue;
  • the extraction unit is used to extract texture features of the original biopsy image, and perform structural feature expression on the extracted texture features to generate the target image structure; recognition;
  • the unit is used to call a preset training model to identify the target graph structure to obtain the target feature vector group; the labeling unit is used to label the target feature vector group through the preset multi-example multi-label model to obtain Multiple target pathology labels.
  • the third aspect of the embodiments of the present application provides an image detection device with multiple tags and multiple instances, including a memory, a processor, and a computer program stored on the memory and running on the processor.
  • the processor When the computer program is executed, the multi-label and multi-example image detection method described in any of the above embodiments is implemented, for example, the following steps are implemented: Obtain the pathological area of the diagnosed biopsy image and the pathological feature description corresponding to the pathological area; The pathological area and the pathological feature description corresponding to the pathological area are trained on the offline training multi-label multi-example model to obtain the preset multi-example multi-label model; the original biopsy image is obtained, and the original biopsy image is used to indicate the status of the human tissue Pathological characteristics; extract texture features from the original biopsy image, and express the extracted texture features in structured features to generate the target map structure; call the preset training model to recognize the target map structure, and obtain the target feature vector group ; Mark the target feature vector group through the preset multi-example
  • the fourth aspect of the embodiments of the present application provides a computer-readable storage medium, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the multiple operations described in any of the above embodiments.
  • the steps of the image detection method with multiple examples of tags implement the following steps: obtain the pathological area of the diagnosed biopsy image and the pathological feature description corresponding to the pathological area; according to the pathological area and the pathological feature description corresponding to the pathological area Offline training of the multi-label and multi-instance model for training to obtain a preset multi-instance and multi-label model; obtain the original biopsy image, which is used to indicate the pathological characteristics of the human tissue; perform texture feature extraction on the original biopsy image, The extracted texture features are expressed in structured features to generate the target graph structure; the preset training model is called to identify the target graph structure, and the target feature vector group is obtained; The target feature vector group is labeled, and multiple target pathological labels are obtained.
  • the embodiments of the present application reduce the analysis time of pathological images, improve analysis efficiency, and improve the accuracy of analysis results.
  • FIG. 1 is a schematic diagram of an embodiment of an image detection method with multiple tags and multiple examples in an embodiment of the application;
  • FIG. 2 is a schematic diagram of another embodiment of an image detection method with multiple tags and multiple examples in an embodiment of the present application
  • FIG. 3 is a schematic diagram of an embodiment of an image detection device with multiple tags and multiple examples in an embodiment of the application;
  • FIG. 4 is a schematic diagram of another embodiment of an image detection device with multiple tags and multiple examples in an embodiment of the application;
  • FIG. 5 is a schematic diagram of an embodiment of an image detection device with multiple tags and multiple examples in an embodiment of the application.
  • This application provides a multi-label and multi-instance image detection method, device, device, and storage medium, which are used for real-time pathological image detection through a multi-label and multi-instance model, generate pathological characteristics of multiple biopsy images, and reduce the risk of pathology.
  • the analysis time of the image improves the analysis efficiency and the accuracy of the analysis result.
  • the technical solution of the present application can be applied to the field of artificial intelligence or digital medical technology, and can perform disease assessment through rapid and reliable analysis of pathological images to realize health management.
  • the flowchart of the multi-label and multi-instance image detection method provided by the embodiment of the present application specifically includes:
  • the server obtains the pathological area of the confirmed biopsy image and the pathological feature description corresponding to the pathological area.
  • the pathological feature description is a textual description of biopsy images in different states, and is used to reflect the medical features of the pathological area in the biopsy image.
  • the text description of the biopsy image corresponding to the stomach of the patient and the biopsy image corresponding to the heart of the patient must be different, and they are used to reflect the medical characteristics of the stomach and the heart, respectively.
  • the execution subject of the present application may be an image detection device with multiple tags and multiple examples, and may also be a terminal or a server, which is not specifically limited here.
  • the embodiment of the present application takes the server as the execution subject as an example for description.
  • the server extracts texture and structural features according to the pathological area to generate a feature vector group; the server uses the feature vector group as the input of the offline training multi-label multi-instance model, and uses the pathological feature description corresponding to the pathological area as the offline training multi-label multi-instance model
  • ⁇ 1 , ⁇ 2 , ⁇ 3 ... ⁇ ni ); the server adjusts ⁇ 1 , ⁇ 2 , ⁇ 3 ... ⁇ ni to minimize the likelihood function value, and the likelihood function is like( ⁇ ) F D (f 1 ( ⁇ 1 x 1 ),f 1 ( ⁇ 2 x 2 ),f 1 ( ⁇ 3 x 3 )...f 1 ( ⁇ ni x ni )
  • the set multi-example multi-label model is F D (f s ( ⁇ 1 x 1 ), f s ( ⁇ 2 x 2 ), f s ( ⁇ 3 x 3 )...f s ( ⁇ ni x ni )
  • the pathological image and the corresponding diagnosis description are used as the training data training model.
  • the advantage of this model is that it can introduce a prior distribution, that is, the empirical relationship between examples and labels can be introduced through the prior probability distribution, so that the model can be Better use of medical experience and knowledge.
  • this model can also use a support vector machine as the final feature annotation output.
  • the server obtains the original biopsy image, and the original biopsy image is used to indicate the pathological characteristics of the human tissue.
  • the laboratory will observe and take pictures of it under a microscope. It is possible to use higher magnification to take pictures of the parts with significant characteristics. There may be 5-10 biopsy images with different magnifications for the whole or some parts of the submitted part.
  • the server performs texture feature extraction on the original biopsy image, and performs structured feature expression on the extracted texture feature to generate the target image structure.
  • the server extracts the texture features of the original biopsy image to obtain multiple target texture feature images; step (1) specifically includes: the server divides the original biopsy image into n*n target regions, where n is a positive integer ; The server compares a target pixel in each target area with 8 pixels in the circular neighborhood around the target pixel clockwise or counterclockwise; if the pixel value of the target pixel is greater than the pixel value of the adjacent pixel, the server Assign the adjacent pixel to 1; if the pixel value of the target pixel is less than or equal to the pixel value of the adjacent pixel, the server assigns the adjacent pixel to 0; the server generates each pixel based on multiple assignment results for each pixel 8-bit binary number of each pixel and converted into decimal number; the server calculates the histogram of each target area, and normalizes it to obtain the statistical histogram; the server connects
  • the server uses a preset clustering algorithm to classify pixels in multiple target texture feature images to generate multiple local homogeneous pixel clusters
  • the server uses the K-means algorithm to cluster the k points in the target texture feature image as the center to obtain k classifications. For example, the server gathers pixels based on the similar properties of pixels in the same object area, starting from the initial area (such as a small neighborhood or even each pixel), and merges adjacent pixels or other areas with the same properties into In the current area, the area gradually grows until there are no pixels or target areas that can be merged.
  • the server uses the centers of multiple local homogeneous pixel clusters as vertices and triangulates them to generate the target graph structure.
  • the server triangulates each pixel group to obtain the three-dimensional information of the pixel group, prevents the loss of the three-dimensional space information, and generates a three-dimensional target map structure.
  • the server calls the preset training model to identify the target graph structure, and obtains the target feature vector group. Specifically, the server calls a preset training model to identify the target graph structure and generates multiple candidate features; the server divides the multiple candidate features into target feature vector groups x 1 , x 2 , K, x ni , where the vector x ni contains ni candidate features.
  • the server determines the number of prediction models according to the number of vectors of the target feature vector group x 1 , x 2 , K, x ni ; the server divides the target feature vector group into multiple prediction vector groups x 1 , x 2 , according to preset candidate case data K,x ni , each predictive vector group corresponds to a preset candidate case data; the server calls a preset multi-example multi-label model to obtain multiple predictive values F D (T) for the pathological label T s of each predictive vector group ,
  • the preset multi-example multi-label model is: F D (f s ( ⁇ 1 x 1 ), f s ( ⁇ 2 x 2 ), f s ( ⁇ 3 x 3 )...f s ( ⁇ ni x ni )
  • a pathological label corresponds to a paragraph of descriptive text.
  • a certain biopsy image corresponds to four pathological labels, which are: thickening of the epidermal spinous layer, elongation of the dermal process, dense perivascularization of the superficial dermis, and infiltration of lymphoid tissue cells.
  • another biopsy image corresponds to six pathological labels, which are: hyperkeratosis of the epidermis; papilloma-like hyperplasia; pseudocorneal cysts can be seen; the dermal cells are thick or thin, intertwined into a network; dermis The superficial perivascular area is dense; lymphoid tissue cells are infiltrated.
  • pathological image detection is performed in real time through a multi-label and multi-instance model to generate pathological characteristics of multiple biopsy images, which reduces the analysis time of pathological images, improves analysis efficiency, and improves the accuracy of analysis results.
  • FIG. 2 another flowchart of the multi-label multi-instance image detection method provided by the embodiment of the present application specifically includes:
  • the server obtains the pathological area of the confirmed biopsy image and the pathological feature description corresponding to the pathological area.
  • the pathological feature description is a textual description of biopsy images in different states, and is used to reflect the medical features of the pathological area in the biopsy image.
  • the text description of the biopsy image corresponding to the patient's stomach and the biopsy image corresponding to the patient's heart must be different, and they are used to reflect the medical characteristics of the stomach and the heart, respectively.
  • the execution subject of the present application may be an image detection device with multiple tags and multiple examples, and may also be a terminal or a server, which is not specifically limited here.
  • the embodiment of the present application takes the server as the execution subject as an example for description.
  • the server extracts texture and structural features according to the pathological area to generate a feature vector group; the server uses the feature vector group as the input of the offline training multi-label multi-instance model, and uses the pathological feature description corresponding to the pathological area as the offline training multi-label multi-instance model
  • ⁇ 1 , ⁇ 2 , ⁇ 3 ... ⁇ ni ); the server adjusts ⁇ 1 , ⁇ 2 , ⁇ 3 ... ⁇ ni to minimize the likelihood function value, and the likelihood function is like( ⁇ ) F D (f 1 ( ⁇ 1 x 1 ),f 1 ( ⁇ 2 x 2 ),f 1 ( ⁇ 3 x 3 )...f 1 ( ⁇ ni x ni )
  • the set multi-example multi-label model is F D (f s ( ⁇ 1 x 1 ), f s ( ⁇ 2 x 2 ), f s ( ⁇ 3 x 3 )...f s ( ⁇ ni x ni )
  • the pathological image and the corresponding diagnosis description are used as the training data training model.
  • the advantage of this model is that it can introduce a prior distribution, that is, the empirical relationship between examples and labels can be introduced through the prior probability distribution, so that the model can be Better use of medical experience and knowledge.
  • this model can also use a support vector machine as the final feature annotation output.
  • the server obtains the original biopsy image, and the original biopsy image is used to indicate the pathological characteristics of the human tissue.
  • the laboratory will observe and take pictures of it under a microscope. It is possible to use higher magnification to take pictures of the parts with significant characteristics. There may be 5-10 biopsy images with different magnifications for the whole or some parts of the submitted part.
  • the server performs texture feature extraction on the original biopsy image, and performs structured feature expression on the extracted texture feature to generate the target image structure.
  • the server extracts the texture features of the original biopsy image to obtain multiple target texture feature images; step (1) specifically includes: the server divides the original biopsy image into n*n target regions, where n is a positive integer ; The server compares a target pixel in each target area with 8 pixels in the circular neighborhood around the target pixel clockwise or counterclockwise; if the pixel value of the target pixel is greater than the pixel value of the adjacent pixel, the server Assign the adjacent pixel to 1; if the pixel value of the target pixel is less than or equal to the pixel value of the adjacent pixel, the server assigns the adjacent pixel to 0; the server generates each pixel based on multiple assignment results for each pixel 8-bit binary number of each pixel and converted into decimal number; the server calculates the histogram of each target area, and normalizes it to obtain the statistical histogram; the server connects
  • the server uses a preset clustering algorithm to classify pixels in multiple target texture feature images to generate multiple local homogeneous pixel clusters
  • the server uses the K-means algorithm to cluster the k points in the target texture feature image as the center to obtain k classifications. For example, the server gathers pixels based on the similar properties of pixels in the same object area, starting from the initial area (such as a small neighborhood or even each pixel), and merges adjacent pixels or other areas with the same properties into In the current area, the area gradually grows until there are no pixels or target areas that can be merged.
  • the server uses the centers of multiple local homogeneous pixel clusters as vertices and triangulates them to generate the target graph structure.
  • the server triangulates each pixel group to obtain the three-dimensional information of the pixel group, prevents the loss of the three-dimensional space information, and generates a three-dimensional target map structure.
  • the server calls the preset training model to identify the target graph structure, and obtains the target feature vector group. Specifically, the server calls a preset training model to identify the target graph structure and generates multiple candidate features; the server divides the multiple candidate features into target feature vector groups x 1 , x 2 , K, x ni , where the vector x ni contains ni candidate features.
  • the server determines the number of prediction models according to the number of vectors of the target feature vector group x 1 , x 2 , K, x ni ; the server divides the target feature vector group into multiple prediction vector groups x 1 , x 2 , according to preset candidate case data K,x ni , each predictive vector group corresponds to a preset candidate case data; the server calls a preset multi-example multi-label model to obtain multiple predictive values F D (T) for the pathological label T s of each predictive vector group ,
  • the preset multi-example multi-label model is: F D (f s ( ⁇ 1 x 1 ), f s ( ⁇ 2 x 2 ), f s ( ⁇ 3 x 3 )...f s ( ⁇ ni x ni )
  • a pathological label corresponds to a paragraph of descriptive text.
  • a certain biopsy image corresponds to four pathological labels, which are: thickening of the epidermal spinous layer, elongation of the dermal process, dense perivascularization of the superficial dermis, and infiltration of lymphoid tissue cells.
  • another biopsy image corresponds to six pathological labels, which are: hyperkeratosis of the epidermis; papilloma-like hyperplasia; pseudocorneal cysts can be seen; the dermal cells are thick or thin, intertwined into a network; dermis The superficial perivascular area is dense; lymphoid tissue cells are infiltrated.
  • the server verifies multiple target pathological tags, and initializes the model parameters of the preset multi-example multi-label model. Among them, after the multi-example multi-label learning model is designed and trained using historical data, the parameters of this model can be solidified, a Web application can be implemented in Java language, and the biopsy image uploaded by the doctor can be accepted. Probability of its various characteristics.
  • the pathological area of the confirmed biopsy image and the pathological feature description corresponding to the pathological area are obtained; the offline training multi-label multi-example model is trained according to the pathological feature description corresponding to the pathological area and the pathological area, Obtain a preset multi-example multi-label model; obtain the original biopsy image, which is used to indicate the pathological characteristics of human tissue; extract the texture feature of the original biopsy image, and perform structured feature expression on the extracted texture feature to generate Target map structure; call the preset training model to identify the target map structure to obtain the target feature vector group; use the preset multi-example multi-label model to label the target feature vector group to obtain multiple target pathological labels.
  • pathological image detection is performed in real time through a multi-label and multi-instance model to generate pathological characteristics of multiple biopsy images, which reduces the analysis time of pathological images, improves analysis efficiency, and improves the accuracy of analysis results.
  • An embodiment of the image detection device includes:
  • the first acquiring unit 301 is configured to acquire the pathological area of the diagnosed biopsy image and the pathological feature description corresponding to the pathological area;
  • the training unit 302 is configured to train the offline training multi-label multi-example model according to the pathological area and the pathological feature description corresponding to the pathological area, to obtain a preset multi-example multi-label model;
  • the second acquiring unit 303 is configured to acquire an original biopsy image, where the original biopsy image is used to indicate the pathological characteristics of the human tissue;
  • the extraction unit 304 is configured to perform texture feature extraction on the original biopsy image, and perform structured feature expression on the extracted texture feature to generate a target image structure;
  • the recognition unit 305 is configured to call a preset training model to recognize the structure of the target graph, and obtain a target feature vector group;
  • the labeling unit 306 is configured to label the target feature vector group through the preset multi-example multi-label model to obtain multiple target pathology labels.
  • pathological image detection is performed in real time through a multi-label and multi-instance model to generate pathological characteristics of multiple biopsy images, which reduces the analysis time of pathological images, improves analysis efficiency, and improves the accuracy of analysis results.
  • another embodiment of the image detection device with multiple tags and multiple instances in the embodiment of the present application includes:
  • the first acquiring unit 301 is configured to acquire the pathological area of the diagnosed biopsy image and the pathological feature description corresponding to the pathological area;
  • the training unit 302 is configured to train the offline training multi-label multi-example model according to the pathological area and the pathological feature description corresponding to the pathological area, to obtain a preset multi-example multi-label model;
  • the second acquiring unit 303 is configured to acquire an original biopsy image, where the original biopsy image is used to indicate the pathological characteristics of the human tissue;
  • the extraction unit 304 is configured to perform texture feature extraction on the original biopsy image, and perform structured feature expression on the extracted texture feature to generate a target image structure;
  • the recognition unit 305 is configured to call a preset training model to recognize the structure of the target graph, and obtain a target feature vector group;
  • the labeling unit 306 is configured to label the target feature vector group through the preset multi-example multi-label model to obtain multiple target pathology labels.
  • the training unit 302 is specifically used for:
  • the multi-example multi-label model is F D (f s ( ⁇ 1 x 1 ), f s ( ⁇ 2 x 2 ), f s ( ⁇ 3x 3 )... fs ( ⁇ ni x ni )
  • the extraction unit 304 includes:
  • the extraction module 3041 is configured to extract texture features of the original biopsy image to obtain multiple target texture feature images
  • the classification module 3042 is configured to use a preset clustering algorithm to classify the pixels in the multiple target texture feature images to generate multiple local homogeneous pixel clusters;
  • the generating module 3043 is configured to use the centers of the multiple local homogeneous pixel clusters as vertices and perform triangulation to generate the target graph structure.
  • the extraction module 3041 is specifically used for:
  • n is a positive integer; compare one target pixel in each target area with 8 pixels in the annular neighborhood around the target pixel clockwise or counterclockwise ; If the pixel value of the target pixel is greater than the pixel value of the adjacent pixel, the adjacent pixel is assigned a value of 1; if the pixel value of the target pixel is less than or equal to the pixel value of the adjacent pixel, the adjacent pixel is assigned It is 0; According to the multiple assignment results of each pixel, an 8-bit binary number of each pixel is generated and converted into a decimal number; the histogram of each target area is calculated and normalized to obtain the statistical histogram Figure; Connect the statistical histograms of each target area to generate multiple target texture feature images of the original biopsy image.
  • the identification unit 305 is specifically configured to:
  • the marking unit 306 is specifically configured to:
  • the preset multi-example multi-label model is: F D (f s ( ⁇ 1 x 1 ), f s ( ⁇ 2 x 2 ), f s ( ⁇ 3 x 3 )...f s ( ⁇ ni x ni )
  • the label corresponds
  • the multi-label multi-instance image detection device further includes:
  • the verification initialization unit 307 is configured to verify the multiple target pathological labels and initialize the model parameters of the preset multi-example multi-label model.
  • the pathological area of the confirmed biopsy image and the pathological feature description corresponding to the pathological area are obtained; the offline training multi-label multi-example model is trained according to the pathological feature description corresponding to the pathological area and the pathological area, Obtain a preset multi-example multi-label model; obtain the original biopsy image, which is used to indicate the pathological characteristics of human tissue; extract the texture feature of the original biopsy image, and perform structured feature expression on the extracted texture feature to generate Target map structure; call the preset training model to identify the target map structure to obtain the target feature vector group; use the preset multi-example multi-label model to label the target feature vector group to obtain multiple target pathological labels.
  • pathological image detection is performed in real time through a multi-label and multi-instance model to generate pathological characteristics of multiple biopsy images, which reduces the analysis time of pathological images, improves analysis efficiency, and improves the accuracy of analysis results.
  • FIG. 5 is a schematic structural diagram of a multi-label and multi-instance image detection device provided by an embodiment of the present application.
  • the multi-label and multi-instance image detection device 500 may have relatively large differences due to different configurations or performances, and may include one or More than one processor (central processing units, CPU) 501 (for example, one or more processors) and memory 509, one or more storage media 508 for storing application programs 507 or data 506 (for example, one or one storage device with a large amount of storage ).
  • the memory 509 and the storage medium 508 may be short-term storage or persistent storage.
  • the program stored in the storage medium 508 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the image detection device with multiple tags and multiple instances. Further, the processor 501 may be configured to communicate with the storage medium 508, and execute a series of instruction operations in the storage medium 508 on the image detection device 500 with multiple tags and multiple instances.
  • the multi-label and multi-instance image detection device 500 may also include one or more power supplies 502, one or more wired or wireless network interfaces 503, one or more input and output interfaces 504, and/or, one or more operating systems 505 , Such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD and so on.
  • Windows Serve Windows Serve
  • Mac OS X Unix
  • Linux FreeBSD
  • FIG. 5 does not constitute a limitation on the multi-label and multi-instance image detection device, and may include more or less components than shown in the figure. Or some parts are combined, or different parts are arranged.
  • the processor 501 can perform the functions of the first acquisition unit 301, the training unit 302, the second acquisition unit 303, the extraction unit 304, the recognition unit 305, the labeling unit 306, and the inspection initialization unit 307 in the foregoing embodiment.
  • the processor 501 is the control center of the multi-label and multi-instance image detection device, and can perform processing according to the set multi-label and multi-instance image detection method.
  • the processor 501 uses various interfaces and lines to connect various parts of the entire multi-label and multi-instance image detection device, and by running or executing software programs and/or modules stored in the memory 509, and calling data stored in the memory 509,
  • the various functions and processing data of the image detection device that executes multiple labels and multiple examples, thereby reducing the analysis time of pathological images, improving the analysis efficiency, and improving the accuracy of the analysis results.
  • the storage medium 508 and the memory 509 are both carriers for storing data.
  • the storage medium 508 may refer to an internal memory with a small storage capacity but a fast speed, and the storage medium 509 may have a large storage capacity but a slow storage speed. External memory.
  • the memory 509 may be used to store software programs and modules.
  • the processor 501 executes various functional applications and data processing of the image detection device 500 with multiple tags and multiple instances by running the software programs and modules stored in the memory 509.
  • the memory 509 may mainly include a storage program area and a storage data area.
  • the storage program area may store an operating system and at least one application program required by a function (such as extracting texture features of the original biopsy image, and performing the extracted texture features). Structured feature expression, generating target map structure), etc.; the storage data area can store data (such as target feature vector groups, etc.) created based on the use of image detection equipment with multiple labels and multiple examples.
  • the memory 509 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices.
  • a non-volatile memory such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices.
  • the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.
  • the computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium.
  • the computer instructions may be transmitted from a website, computer, server, or data center. Transmission to another website, computer, server or data center via wired (such as coaxial cable, optical fiber, twisted pair) or wireless (such as infrared, wireless, microwave, etc.).
  • the computer-readable storage medium may be any available medium that can be stored by a computer or a data storage device such as a server or a data center integrated with one or more available media.
  • the usable medium may be a magnetic medium (for example, a floppy disk, a hard disk, and a magnetic tape), an optical medium (for example, an optical disc), or a semiconductor medium (for example, a solid state disk (SSD)).
  • the computer-readable storage medium may be non-volatile or volatile.
  • the disclosed system, device, and method can be implemented in other ways.
  • the device embodiments described above are only illustrative.
  • the division of the units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components may be combined or It can be integrated into another system, or some features can be ignored or not implemented.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
  • the functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.
  • the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium.
  • the technical solution of the present application essentially or the part that contributes to the existing technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , Including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (read-only memory, ROM), random access memory (random access memory, RAM), magnetic disks or optical disks and other media that can store program codes. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Evolutionary Computation (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Probability & Statistics with Applications (AREA)
  • Image Analysis (AREA)

Abstract

涉及人工智能领域，一种多标签多示例的图像检测方法、装置、设备及存储介质，用于减少对病理图像的分析时长，并提高分析结果的准确性。该方法包括：获取已确诊的活检图像的病理区域以及病理区域对应的病理特征描述（101）；根据病理区域和病理区域对应的病理特征描述对离线训练多标签多示例模型进行训练，得到预置的多示例多标签模型（102）；获取原始活检图像，原始活检图像用于指示人体组织的病理特性（103）；对原始活检图像进行纹理特征提取，并将提取到的纹理特征进行结构化特征表达，生成目标图结构（104）；调用预置的训练模型对目标图结构进行识别，得到目标特征向量组（105）；通过预置的多示例多标签模型对目标特征向量组进行标注，得到多个目标病理标签（106）。

Description

多标签多示例的图像检测方法、装置、设备及存储介质
本申请要求于2020年3月9日提交中国专利局、申请号为202010156699.6，发明名称为“多标签多示例的图像检测方法、装置、设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及区域提取技术领域，尤其涉及一种多标签多示例的图像检测方法、装置、设备及存储介质。
背景技术
活检图像是对患者病变部位的组织进行切片，在显微镜下进行放大成像得到的图像。活检图像直接反映组织内部所发生的病变，是医生进行疾病诊断的重要依据，甚至是进行某些严重疾病诊断的最终依据。例如在癌症的诊断中，通过对病灶的影像学手段(X光、CT、MRI等)进行观察，包括观察是否有阴影、斑块、血管的扩张情况等，结合血液化验手段，可以得出初步的诊断。但要最后确诊，必须提取病灶的活体组织，进行显微镜下的观察及进行生物化学检验，该过程称为活检。
以皮肤科为例，常见的皮肤病有3000多种，而通过观察皮肤表面能做出诊断的仅有几百种，其它种类大多需要取出病变部位的组织进行显微镜下观察。另一方面，活检图像的病理特性种类和数量均比较稳定，每一个医科都有其需要关注的病理特性，大多数在100-200个之间。随着病人数量的日益增加和对疾病准确诊断率的更高要求，病理检查特别是对其显微图像分析的需求成倍增加，要精确判断这100至200个不同的病理特性，对医生的专业知识、判断力及其读片时的专注程度要求极高。
发明人意识到，在临床诊断中，医生会对整个病理图像进行分析得到诊断报告，绝大多数情况下不会在诊断报告中明确指出哪个区域具有何种病理特性，为了将诊断报告中的特性对应到图像中不同区域，需要调用对应的模型进行标记，N个特性就需要N个模型，使用模型数量多，标记过程繁琐。
发明内容
本申请提供了一种多标签多示例的图像检测方法、装置、设备及存储介质，减少了对病理图像的分析时长，提高了分析效率，并提高了分析结果的准确性。
本申请实施例的第一方面提供一种多标签多示例的图像检测方法，包括：获取已确诊的活检图像的病理区域以及病理区域对应的病理特征描述；根据所述病理区域和所述病理区域对应的病理特征描述对离线训练多标签多示例模型进行训练，得到预置的多示例多标签模型；获取原始活检图像，所述原始活检图像用于指示人体组织的病理特性；对所述原始活检图像进行纹理特征提取，并将提取到的纹理特征进行结构化特征表达，生成目标图结构；调用预置的训练模型对目标图结构进行识别，得到目标特征向量组；通过所述预置的多示例多标签模型对所述目标特征向量组进行标注，得到多个目标病理标签。
本申请实施例的第二方面提供了一种多标签多示例的图像检测装置，包括：第一获取单元，用于获取已确诊的活检图像的病理区域以及病理区域对应的病理特征描述；训练单元，用于根据所述病理区域和所述病理区域对应的病理特征描述对离线训练多标签多示例模型进行训练，得到预置的多示例多标签模型；第二获取单元，用于获取原始活检图像，所述原始活检图像用于指示人体组织的病理特性；提取单元，用于对所述原始活检图像进行纹理特征提取，并将提取到的纹理特征进行结构化特征表达，生成目标图结构；识别单元，用于调用预置的训练模型对目标图结构进行识别，得到目标特征向量组；标注单元，用于通过所述预置的多示例多标签模型对所述目标特征向量组进行标注，得到多个目标病理标签。
本申请实施例的第三方面提供了一种多标签多示例的图像检测设备，包括存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现上述任一实施方式所述的多标签多示例的图像检测方法，例如，实现以下步骤：获取已确诊的活检图像的病理区域以及病理区域对应的病理特征描述；根据所述病理区域和所述病理区域对应的病理特征描述对离线训练多标签多示例模型进行训练，得到预置的多示例多标签模型；获取原始活检图像，所述原始活检图像用于指示人体组织的病理特性；对所述原始活检图像进行纹理特征提取，并将提取到的纹理特征进行结构化特征表达，生成目标图结构；调用预置的训练模型对目标图结构进行识别，得到目标特征向量组；通过所述预置的多示例多标签模型对所述目标特征向量组进行标注，得到多个目标病理标签。
本申请实施例的第四方面提供了一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，当所述计算机程序被处理器执行时实现上述任一实施方式所述的多标签多示例的图像检测方法的步骤，例如，实现以下步骤：获取已确诊的活检图像的病理区域以及病理区域对应的病理特征描述；根据所述病理区域和所述病理区域对应的病理特征描述对离线训练多标签多示例模型进行训练，得到预置的多示例多标签模型；获取原始活检图像，所述原始活检图像用于指示人体组织的病理特性；对所述原始活检图像进行纹理特征提取，并将提取到的纹理特征进行结构化特征表达，生成目标图结构；调用预置的训练模型对目标图结构进行识别，得到目标特征向量组；通过所述预置的多示例多标签模型对所述目标特征向量组进行标注，得到多个目标病理标签。
本申请实施例减少了对病理图像的分析时长，提高了分析效率，并提高了分析结果的准确性。
附图说明
图1为本申请实施例中多标签多示例的图像检测方法的一个实施例示意图；
图2为本申请实施例中多标签多示例的图像检测方法的另一个实施例示意图；
图3为本申请实施例中多标签多示例的图像检测装置的一个实施例示意图；
图4为本申请实施例中多标签多示例的图像检测装置的另一个实施例示意图；
图5为本申请实施例中多标签多示例的图像检测设备的一个实施例示意图。
具体实施方式
本申请提供了一种多标签多示例的图像检测方法、装置、设备及存储介质，用于通过一个多标签多示例模型实时进行病理图像检测，生成多个活检图像的病理特性，减少了对病理图像的分析时长，提高了分析效率，并提高了分析结果的准确性。
本申请的技术方案可应用于人工智能或数字医疗技术领域，能够通过对病理图像的快速、可靠分析，进行疾病评估，以实现健康管理。
为了使本技术领域的人员更好地理解本申请方案，下面将结合本申请实施例中的附图，对本申请实施例进行描述。
本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”、“第三”、“第四”等(如果存在)是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便这里描述的实施例能够以除了在这里图示或描述的内容以外的顺序实施。此外，术语“包括”或“具有”及其任何变形，意图在于覆盖不排他的包含，例如，包含了一系列步骤或单元的过程、方法、系统、产品或设备不必限于清楚地列出的那些步骤或单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。
请参阅图1，本申请实施例提供的多标签多示例的图像检测方法的流程图，具体包括：
101、获取已确诊的活检图像的病理区域以及病理区域对应的病理特征描述。
服务器获取已确诊的活检图像的病理区域以及病理区域对应的病理特征描述。
需要说明的是，病理特征描述是针对不同状态下的活检图像的文字描述，用于体现活检图像中病理区域的医学特征。例如，对应病人的胃的活检图像与对应病人的心脏的活检图像的文字描述肯定不同，分别用来反映胃和心脏的医学特征。
可以理解的是，本申请的执行主体可以为多标签多示例的图像检测装置，还可以是终端或者服务器，具体此处不做限定。本申请实施例以服务器为执行主体为例进行说明。
102、根据病理区域和病理区域对应的病理特征描述对离线训练多标签多示例模型进行训练，得到预置的多示例多标签模型。
具体的，服务器根据病理区域进行纹理和结构特征提取，生成特征向量组；服务器将特征向量组作为离线训练多标签多示例模型的输入，将病理区域对应的病理特征描述作为离线训练多标签多示例模型的输出，训练得到离线训练多标签多示例模型F D(f 1(θ 1x 1),f 1(θ 2x 2),f 1(θ 3x 3)...f 1(θ nix ni)|θ 1,θ 2,θ 3...θ ni)；服务器调整θ 1,θ 2,θ 3...θ ni，使得似然函数值最小，似然函数为like(θ)＝F D(f 1(θ 1x 1),f 1(θ 2x 2),f 1(θ 3x 3)...f 1(θ nix ni)|θ 1,θ 2,θ 3...θ ni)-F D(T)；服务器生成预置的多示例多标签模型，并保存最小似然函数值对应的模型参数θ 1,θ 2,θ 3...θ ni，预置的多示例多标签模型为F D(f s(θ 1x 1),f s(θ 2x 2),f s(θ 3x 3)...f s(θ nix ni)|θ 1,θ 2,θ 3...θ ni)。
需要说明的是，将病理图像及对应的诊断描述作为训练数据训练模型。采用基于贝叶斯学习的多示例多标签学习模型，这个模型的好处是可以引入先验分布，也就是示例之间和标签之间的经验关系都可以通过先验概率分布进行引入，使得模型可以更好的使用医学经验和知识。同时这个模型也可以使用支持向量机作为最终的特性标注输出。
103、获取原始活检图像，原始活检图像用于指示人体组织的病理特性。
服务器获取原始活检图像，原始活检图像用于指示人体组织的病理特性。一般情况下，一个病人活体组织送检时，实验室会对其进行显微镜下观察并拍照，对于有显著特性的局部有可能使用更高的放大倍数进行拍照，因此一个病人根据其送检部位的情况可能会有5-10幅不同放大倍数、针对送检部位全局或某几个局部的活检图像。
104、对原始活检图像进行纹理特征提取，并将提取到的纹理特征进行结构化特征表达，生成目标图结构。
服务器对原始活检图像进行纹理特征提取，并将提取到的纹理特征进行结构化特征表达，生成目标图结构。具体的，(1)服务器对原始活检图像的纹理特征进行提取，得到多个目标纹理特征图像；步骤(1)具体包括：服务器将原始活检图像划分为n*n个目标区域，n为正整数；服务器将每个目标区域中的一个目标像素和目标像素周围环形邻域内的8个像素点进行顺时针或逆时针的比较；若目标像素的像素值大于相邻像素点的像素值，则服务器将相邻像素点赋值为1；若目标像素的像素值小于或等于相邻像素点的像素值，则服务器将相邻像素点赋值为0；服务器根据每个像素点的多个赋值结果生成每个像素点的8位二进制数，并转换成十进制数的数值；服务器计算每个目标区域的直方图，并进行归一化处理得到统计直方图；服务器将每个目标区域的统计直方图进行连接，生成原始活检图像的多个目标纹理特征图像。
(2)服务器采用预置聚类算法对多个目标纹理特征图像中的像素点进行分类，生成多个局部同类像素团；
服务器采用K-means算法以目标纹理特征图像中的k个点为中心进行聚类，得到k个分类。例如，服务器根据同一物体区域内像素的相似性质来聚集像素点的方法，从初始区域开始(如小邻域或甚至于每个像素)，将相邻的具有同样性质的像素或其它区域归并到目前的区域中从而逐步增长区域，直至没有可以归并的像素点或目标区域为止。
(3)服务器将多个局部同类像素团中的中心作为顶点并进行三角化，生成目标图结构。
服务器对每个像素团进行三角化，得到像素团的三维信息，防止丢失三维空间信息，生成三维的目标图结构。
需要说明的是，在进行纹理特征提取的过程中，需要排除被填充像素点对特征提取结果的影响，还可以采用二维小波变换(2D discrete Wavelet transformation，2D-DWT)、灰度共生矩阵(gray-level co-occurrence matrix，GLCM)、局部二值模式(local binary pattern，LBP)等方法对于单个区域进行特征提取，其中，输入图像分辨率与训练图像分辨率相同时，可选择灰度共生矩阵方法进行纹理特征提取。
105、调用预置的训练模型对目标图结构进行识别，得到目标特征向量组。
服务器调用预置的训练模型对目标图结构进行识别，得到目标特征向量组。具体的，服务器调用预置的训练模型对目标图结构进行识别，生成多个候选特征；服务器将多个候选特征划分为目标特征向量组x 1,x 2,K,x ni，其中，向量x ni包含ni个候选特征。
106、通过预置的多示例多标签模型对目标特征向量组进行标注，得到多个目标病理标签。
服务器根据目标特征向量组x 1,x 2,K,x ni的向量数量确定预测模型的数量；服务器根据预置候选病例数据将目标特征向量组划分为多个预测向量组x 1,x 2,K,x ni，每个预测向量组对应一个预置候选病例数据；服务器调用预置的多示例多标签模型对每个预测向量组的病理标签T s，得到多个预测值F D(T)，预置的多示例多标签模型为：F D(f s(θ 1x 1),f s(θ 2x 2),f s(θ 3x 3)...f s(θ nix ni)|θ 1,θ 2,θ 3...θ ni)；服务器根据多个预测值F D(T)生成多个病理标签T 1,T 2,T 3...T s，每个病理标签对应一段病理描述文字。
可以理解的是，一个病理标签对应一段描述文字，例如，某一个活检图像对应有四个病理标签，分别是：表皮棘层增厚，皮突延长，真皮浅层血管周致密，淋巴组织细胞浸润。又例如，另一个活检图像对应有六个病理标签，分别是：表皮角化过度；乳头瘤样增生；可见假性角囊肿；皮突细胞呈粗条状或细条索状，交织成网；真皮浅层血管周致密；淋巴组织细胞浸润。
本申请实施例，通过一个多标签多示例模型实时进行病理图像检测，生成多个活检图像的病理特性，减少了对病理图像的分析时长，提高了分析效率，并提高了分析结果的准确性。
请参阅图2，本申请实施例提供的多标签多示例的图像检测方法的另一个流程图，具体包括：
201、获取已确诊的活检图像的病理区域以及病理区域对应的病理特征描述。
服务器获取已确诊的活检图像的病理区域以及病理区域对应的病理特征描述。
需要说明的是，病理特征描述是针对不同状态下的活检图像的文字描述，用于体现活检图像中病理区域的医学特征。例如，对应病人的胃的活检图像与对应病人的心脏的活检 图像的文字描述肯定不同，分别用来反映胃和心脏的医学特征。
可以理解的是，本申请的执行主体可以为多标签多示例的图像检测装置，还可以是终端或者服务器，具体此处不做限定。本申请实施例以服务器为执行主体为例进行说明。
202、根据病理区域和病理区域对应的病理特征描述对离线训练多标签多示例模型进行训练，得到预置的多示例多标签模型。
具体的，服务器根据病理区域进行纹理和结构特征提取，生成特征向量组；服务器将特征向量组作为离线训练多标签多示例模型的输入，将病理区域对应的病理特征描述作为离线训练多标签多示例模型的输出，训练得到离线训练多标签多示例模型F D(f 1(θ 1x 1),f 1(θ 2x 2),f 1(θ 3x 3)...f 1(θ nix ni)|θ 1,θ 2,θ 3...θ ni)；服务器调整θ 1,θ 2,θ 3...θ ni，使得似然函数值最小，似然函数为like(θ)＝F D(f 1(θ 1x 1),f 1(θ 2x 2),f 1(θ 3x 3)...f 1(θ nix ni)|θ 1,θ 2,θ 3...θ ni)-F D(T)；服务器生成预置的多示例多标签模型，并保存最小似然函数值对应的模型参数θ 1,θ 2,θ 3...θ ni，预置的多示例多标签模型为F D(f s(θ 1x 1),f s(θ 2x 2),f s(θ 3x 3)...f s(θ nix ni)|θ 1,θ 2,θ 3...θ ni)。
需要说明的是，将病理图像及对应的诊断描述作为训练数据训练模型。采用基于贝叶斯学习的多示例多标签学习模型，这个模型的好处是可以引入先验分布，也就是示例之间和标签之间的经验关系都可以通过先验概率分布进行引入，使得模型可以更好的使用医学经验和知识。同时这个模型也可以使用支持向量机作为最终的特性标注输出。
203、获取原始活检图像，原始活检图像用于指示人体组织的病理特性。
服务器获取原始活检图像，原始活检图像用于指示人体组织的病理特性。一般情况下，一个病人活体组织送检时，实验室会对其进行显微镜下观察并拍照，对于有显著特性的局部有可能使用更高的放大倍数进行拍照，因此一个病人根据其送检部位的情况可能会有5-10幅不同放大倍数、针对送检部位全局或某几个局部的活检图像。
204、对原始活检图像进行纹理特征提取，并将提取到的纹理特征进行结构化特征表达，生成目标图结构。
服务器对原始活检图像进行纹理特征提取，并将提取到的纹理特征进行结构化特征表达，生成目标图结构。具体的，(1)服务器对原始活检图像的纹理特征进行提取，得到多个目标纹理特征图像；步骤(1)具体包括：服务器将原始活检图像划分为n*n个目标区域，n为正整数；服务器将每个目标区域中的一个目标像素和目标像素周围环形邻域内的8个像素点进行顺时针或逆时针的比较；若目标像素的像素值大于相邻像素点的像素值，则服务器将相邻像素点赋值为1；若目标像素的像素值小于或等于相邻像素点的像素值，则服务器将相邻像素点赋值为0；服务器根据每个像素点的多个赋值结果生成每个像素点的8位二进制数，并转换成十进制数的数值；服务器计算每个目标区域的直方图，并进行归一化处理得到统计直方图；服务器将每个目标区域的统计直方图进行连接，生成原始活检图像的多个目标纹理特征图像。
(2)服务器采用预置聚类算法对多个目标纹理特征图像中的像素点进行分类，生成多个局部同类像素团；
服务器采用K-means算法以目标纹理特征图像中的k个点为中心进行聚类，得到k个分类。例如，服务器根据同一物体区域内像素的相似性质来聚集像素点的方法，从初始区域开始(如小邻域或甚至于每个像素)，将相邻的具有同样性质的像素或其它区域归并到目前的区域中从而逐步增长区域，直至没有可以归并的像素点或目标区域为止。
(3)服务器将多个局部同类像素团中的中心作为顶点并进行三角化，生成目标图结构。
服务器对每个像素团进行三角化，得到像素团的三维信息，防止丢失三维空间信息，生成三维的目标图结构。
需要说明的是，在进行纹理特征提取的过程中，需要排除被填充像素点对特征提取结果的影响，还可以采用二维小波变换(2D discrete Wavelet transformation，2D-DWT)、灰度共生矩阵(gray-level co-occurrence matrix，GLCM)、局部二值模式(local binary pattern，LBP)等方法对于单个区域进行特征提取，其中，输入图像分辨率与训练图像分辨率相同时，可选择灰度共生矩阵方法进行纹理特征提取。
205、调用预置的训练模型对目标图结构进行识别，得到目标特征向量组。
服务器调用预置的训练模型对目标图结构进行识别，得到目标特征向量组。具体的，服务器调用预置的训练模型对目标图结构进行识别，生成多个候选特征；服务器将多个候选特征划分为目标特征向量组x 1,x 2,K,x ni，其中，向量x ni包含ni个候选特征。
206、通过预置的多示例多标签模型对目标特征向量组进行标注，得到多个目标病理标签。
服务器根据目标特征向量组x 1,x 2,K,x ni的向量数量确定预测模型的数量；服务器根据预置候选病例数据将目标特征向量组划分为多个预测向量组x 1,x 2,K,x ni，每个预测向量组对应一个预置候选病例数据；服务器调用预置的多示例多标签模型对每个预测向量组的病理标签T s，得到多个预测值F D(T)，预置的多示例多标签模型为：F D(f s(θ 1x 1),f s(θ 2x 2),f s(θ 3x 3)...f s(θ nix ni)|θ 1,θ 2,θ 3...θ ni)；服务器根据多个预测值F D(T)生成多个病理标签T 1,T 2,T 3...T s，每个病理标签对应一段病理描述文字。
可以理解的是，一个病理标签对应一段描述文字，例如，某一个活检图像对应有四个病理标签，分别是：表皮棘层增厚，皮突延长，真皮浅层血管周致密，淋巴组织细胞浸润。又例如，另一个活检图像对应有六个病理标签，分别是：表皮角化过度；乳头瘤样增生；可见假性角囊肿；皮突细胞呈粗条状或细条索状，交织成网；真皮浅层血管周致密；淋巴组织细胞浸润。
207、对多个目标病理标签进行校验，初始化预置的多示例多标签模型的模型参数。
服务器对多个目标病理标签进行校验，初始化预置的多示例多标签模型的模型参数。其中，多示例多标签学习模型设计并使用历史数据训练好之后，可以固化这个模型的参数，用Java语言实现一个Web应用，接受医生上传的活检图像，实时通过模型得到这个活检图像的特性标注或其各种特性的概率。
用历史已确诊的活检图像的病理区域和对应的病理特征离线训练多标签多示例学习器，并固化训练后的参数以便下次预测加载使用。定期进行离线训练，可以每星期或每月离线训练一次，以保证模型准确率。
本申请实施例提供的技术方案中，获取已确诊的活检图像的病理区域以及病理区域对应的病理特征描述；根据病理区域和病理区域对应的病理特征描述对离线训练多标签多示例模型进行训练，得到预置的多示例多标签模型；获取原始活检图像，原始活检图像用于指示人体组织的病理特性；对原始活检图像进行纹理特征提取，并将提取到的纹理特征进行结构化特征表达，生成目标图结构；调用预置的训练模型对目标图结构进行识别，得到 目标特征向量组；通过预置的多示例多标签模型对目标特征向量组进行标注，得到多个目标病理标签。本申请实施例，通过一个多标签多示例模型实时进行病理图像检测，生成多个活检图像的病理特性，减少了对病理图像的分析时长，提高了分析效率，并提高了分析结果的准确性。
上面对本申请实施例中多标签多示例的图像检测方法进行了描述，下面对本申请实施例中多标签多示例的图像检测装置进行描述，请参阅图3，本申请实施例中多标签多示例的图像检测装置的一个实施例包括：
第一获取单元301，用于获取已确诊的活检图像的病理区域以及病理区域对应的病理特征描述；
训练单元302，用于根据所述病理区域和所述病理区域对应的病理特征描述对离线训练多标签多示例模型进行训练，得到预置的多示例多标签模型；
第二获取单元303，用于获取原始活检图像，所述原始活检图像用于指示人体组织的病理特性；
提取单元304，用于对所述原始活检图像进行纹理特征提取，并将提取到的纹理特征进行结构化特征表达，生成目标图结构；
识别单元305，用于调用预置的训练模型对目标图结构进行识别，得到目标特征向量组；
标注单元306，用于通过所述预置的多示例多标签模型对所述目标特征向量组进行标注，得到多个目标病理标签。
本申请实施例，通过一个多标签多示例模型实时进行病理图像检测，生成多个活检图像的病理特性，减少了对病理图像的分析时长，提高了分析效率，并提高了分析结果的准确性。
请参阅图4，本申请实施例中多标签多示例的图像检测装置的另一个实施例包括：
第一获取单元301，用于获取已确诊的活检图像的病理区域以及病理区域对应的病理特征描述；
训练单元302，用于根据所述病理区域和所述病理区域对应的病理特征描述对离线训练多标签多示例模型进行训练，得到预置的多示例多标签模型；
第二获取单元303，用于获取原始活检图像，所述原始活检图像用于指示人体组织的病理特性；
提取单元304，用于对所述原始活检图像进行纹理特征提取，并将提取到的纹理特征进行结构化特征表达，生成目标图结构；
识别单元305，用于调用预置的训练模型对目标图结构进行识别，得到目标特征向量组；
标注单元306，用于通过所述预置的多示例多标签模型对所述目标特征向量组进行标注，得到多个目标病理标签。
可选的，训练单元302具体用于：
根据所述病理区域进行纹理和结构特征提取，生成特征向量组；将所述特征向量组作为离线训练多标签多示例模型的输入，将所述病理区域对应的病理特征描述作为离线训练多标签多示例模型的输出，训练得到离线训练多标签多示例模型F D(f 1(θ 1x 1),f 1(θ 2x 2),f 1(θ 3x 3)...f 1(θ nix ni)|θ 1,θ 2,θ 3...θ ni)；调整θ 1,θ 2,θ 3...θ ni，使得似然函数值最小，似然函数为like(θ)＝F D(f 1(θ 1x 1),f 1(θ 2x 2),f 1(θ 3x 3)...f 1(θ nix ni)|θ 1,θ 2,θ 3...θ ni)-F D(T)；生成预置的多示例多标签模型，并保存最小似然函数值对应的模型参数θ 1,θ 2,θ 3...θ ni，预置的多示例多标签 模型为F D(f s(θ 1x 1),f s(θ 2x 2),f s(θ3x 3)... fs(θ nix ni)|θ 1,θ 2,θ 3...θ ni)。
可选的，提取单元304包括：
提取模块3041，用于对所述原始活检图像的纹理特征进行提取，得到多个目标纹理特征图像；
分类模块3042，用于采用预置聚类算法对所述多个目标纹理特征图像中的像素点进行分类，生成多个局部同类像素团；
生成模块3043，用于将所述多个局部同类像素团中的中心作为顶点并进行三角化，生成目标图结构。
可选的，提取模块3041具体用于：
将所述原始活检图像划分为n*n个目标区域，n为正整数；将每个目标区域中的一个目标像素和目标像素周围环形邻域内的8个像素点进行顺时针或逆时针的比较；若目标像素的像素值大于相邻像素点的像素值，则将相邻像素点赋值为1；若目标像素的像素值小于或等于相邻像素点的像素值，则将相邻像素点赋值为0；根据每个像素点的多个赋值结果生成每个像素点的8位二进制数，并转换成十进制数的数值；计算每个目标区域的直方图，并进行归一化处理得到统计直方图；将每个目标区域的统计直方图进行连接，生成原始活检图像的多个目标纹理特征图像。
可选的，识别单元305具体用于：
调用预置的训练模型对目标图结构进行识别，生成多个候选特征；将多个候选特征划分为目标特征向量组x 1,x 2,K,x ni，其中，向量x ni包含ni个候选特征。
可选的，标注单元306具体用于：
根据目标特征向量组x 1,x 2,K,x ni的向量数量确定预测模型的数量；根据预置候选病例数据将目标特征向量组划分为多个预测向量组x 1,x 2,K,x ni，每个预测向量组对应一个预置候选病例数据；调用所述预置的多示例多标签模型对每个预测向量组的病理标签T s，得到多个预测值F D(T)，所述预置的多示例多标签模型为：F D(f s(θ 1x 1),f s(θ 2x 2),f s(θ 3x 3)...f s(θ nix ni)|θ 1,θ 2,θ 3...θ ni)；根据多个预测值F D(T)生成多个病理标签T 1,T 2,T 3...T s，每个病理标签对应一段病理描述文字。
可选的，多标签多示例的图像检测装置还包括：
检验初始化单元307，用于对所述多个目标病理标签进行校验，初始化所述预置的多示例多标签模型的模型参数。
本申请实施例提供的技术方案中，获取已确诊的活检图像的病理区域以及病理区域对应的病理特征描述；根据病理区域和病理区域对应的病理特征描述对离线训练多标签多示例模型进行训练，得到预置的多示例多标签模型；获取原始活检图像，原始活检图像用于指示人体组织的病理特性；对原始活检图像进行纹理特征提取，并将提取到的纹理特征进行结构化特征表达，生成目标图结构；调用预置的训练模型对目标图结构进行识别，得到目标特征向量组；通过预置的多示例多标签模型对目标特征向量组进行标注，得到多个目标病理标签。本申请实施例，通过一个多标签多示例模型实时进行病理图像检测，生成多 个活检图像的病理特性，减少了对病理图像的分析时长，提高了分析效率，并提高了分析结果的准确性。
上面图3至图4从模块化功能实体的角度对本申请实施例中的多标签多示例的图像检测装置进行详细描述，下面从硬件处理的角度对本申请实施例中多标签多示例的图像检测设备进行详细描述。
图5是本申请实施例提供的一种多标签多示例的图像检测设备的结构示意图，该多标签多示例的图像检测设备500可因配置或性能不同而产生比较大的差异，可以包括一个或一个以上处理器(central processing units，CPU)501(例如，一个或一个以上处理器)和存储器509，一个或一个以上存储应用程序507或数据506的存储介质508(例如一个或一个以上海量存储设备)。其中，存储器509和存储介质508可以是短暂存储或持久存储。存储在存储介质508的程序可以包括一个或一个以上模块(图示没标出)，每个模块可以包括对多标签多示例的图像检测设备中的一系列指令操作。更进一步地，处理器501可以设置为与存储介质508通信，在多标签多示例的图像检测设备500上执行存储介质508中的一系列指令操作。
多标签多示例的图像检测设备500还可以包括一个或一个以上电源502，一个或一个以上有线或无线网络接口503，一个或一个以上输入输出接口504，和/或，一个或一个以上操作系统505，例如Windows Serve，Mac OS X，Unix，Linux，FreeBSD等等。本领域技术人员可以理解，图5中示出的多标签多示例的图像检测设备结构并不构成对多标签多示例的图像检测设备的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置。处理器501可以执行上述实施例中第一获取单元301、训练单元302、第二获取单元303、提取单元304、识别单元305、标注单元306和检验初始化单元307的功能。
下面结合图5对多标签多示例的图像检测设备的各个构成部件进行具体的介绍：
处理器501是多标签多示例的图像检测设备的控制中心，可以按照设置的多标签多示例的图像检测方法进行处理。处理器501利用各种接口和线路连接整个多标签多示例的图像检测设备的各个部分，通过运行或执行存储在存储器509内的软件程序和/或模块，以及调用存储在存储器509内的数据，执行多标签多示例的图像检测设备的各种功能和处理数据，从而减少了对病理图像的分析时长，提高了分析效率，并提高了分析结果的准确性。存储介质508和存储器509都是存储数据的载体，本申请实施例中，存储介质508可以是指储存容量较小，但速度快的内存储器，而存储器509可以是储存容量大，但储存速度慢的外存储器。
存储器509可用于存储软件程序以及模块，处理器501通过运行存储在存储器509的软件程序以及模块，从而执行多标签多示例的图像检测设备500的各种功能应用以及数据处理。存储器509可主要包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需的应用程序(比如对原始活检图像进行纹理特征提取，并将提取到的纹理特征进行结构化特征表达，生成目标图结构)等；存储数据区可存储根据多标签多示例的图像检测设备的使用所创建的数据(比如目标特征向量组等)等。此外，存储器509可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件、闪存器件、或其他非易失性固态存储器件。在本申请实施例中提供的多标签多示例的图像检测方法程序和接收到的数据流存储在存储器中，当需要使用时，处理器501从存储器509中调用。
在计算机上加载和执行所述计算机程序指令时，全部或部分地产生按照本申请实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读 存储介质向另一计算机可读存储介质传输，例如，所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、双绞线)或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存储的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质，(例如，软盘、硬盘、磁带)、光介质(例如，光盘)、或者半导体介质(例如固态硬盘(solid state disk，SSD))等。
可选的，该计算机可读存储介质(或存储介质)可以是非易失性的，也可以是易失性的。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统，装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统，装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(read-only memory，ROM)、随机存取存储器(random access memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述，以上实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的精神和范围。

Claims (20)

  1. 一种多标签多示例的图像检测方法，其中，包括：
    获取已确诊的活检图像的病理区域以及病理区域对应的病理特征描述；
    根据所述病理区域和所述病理区域对应的病理特征描述对离线训练多标签多示例模型进行训练，得到预置的多示例多标签模型；
    获取原始活检图像，所述原始活检图像用于指示人体组织的病理特性；
    对所述原始活检图像进行纹理特征提取，并将提取到的纹理特征进行结构化特征表达，生成目标图结构；
    调用预置的训练模型对目标图结构进行识别，得到目标特征向量组；
    通过所述预置的多示例多标签模型对所述目标特征向量组进行标注，得到多个目标病理标签。
  2. 根据权利要求1所述的多标签多示例的图像检测方法，其中，所述根据所述病理区域和所述病理区域对应的病理特征描述对离线训练多标签多示例模型进行训练，得到预置的多示例多标签模型，包括：
    根据所述病理区域进行纹理和结构特征提取，生成特征向量组；
    将所述特征向量组作为离线训练多标签多示例模型的输入，将所述病理区域对应的病理特征描述作为离线训练多标签多示例模型的输出，训练得到离线训练多标签多示例模型F D(f 1(θ 1x 1),f 1(θ 2x 2),f 1(θ 3x 3)...f 1(θ nix ni)|θ 1,θ 2,θ 3...θ ni)；
    调整θ 1,θ 2,θ 3...θ ni，使得似然函数值最小，似然函数为
    like(θ)＝F D(f 1(θ 1x 1),f 1(θ 2x 2),f 1(θ 3x 3)...f 1(θ nix ni)|θ 1,θ 2,θ 3...θ ni)-F D(T)；
    生成预置的多示例多标签模型，并保存最小似然函数值对应的模型参数θ 1,θ 2,θ 3...θ ni，预置的多示例多标签模型为F D(f s(θ 1x 1),f s(θ 2x 2),f s(θ 3x 3)...f s(θ nix ni)|θ 1,θ 2,θ 3...θ ni)。
  3. 根据权利要求1所述的多标签多示例的图像检测方法，其中，所述对所述原始活检图像进行纹理特征提取，并将提取到的纹理特征进行结构化特征表达，生成目标图结构，包括：
    对所述原始活检图像的纹理特征进行提取，得到多个目标纹理特征图像；
    采用预置聚类算法对所述多个目标纹理特征图像中的像素点进行分类，生成多个局部同类像素团；
    将所述多个局部同类像素团中的中心作为顶点并进行三角化，生成目标图结构。
  4. 根据权利要求3所述的多标签多示例的图像检测方法，其中，所述对所述原始活检图像的纹理特征进行提取，得到多个目标纹理特征图像，包括：
    将所述原始活检图像划分为n*n个目标区域，n为正整数；
    将每个目标区域中的一个目标像素和目标像素周围环形邻域内的8个像素点进行顺时针或逆时针的比较；
    若目标像素的像素值大于相邻像素点的像素值，则将相邻像素点赋值为1；
    若目标像素的像素值小于或等于相邻像素点的像素值，则将相邻像素点赋值为0；
    根据每个像素点的多个赋值结果生成每个像素点的8位二进制数，并转换成十进制数的数值；
    计算每个目标区域的直方图，并进行归一化处理得到统计直方图；
    将每个目标区域的统计直方图进行连接，生成原始活检图像的多个目标纹理特征图像。
  5. 根据权利要求1所述的多标签多示例的图像检测方法，其中，所述调用预置的训练模型对目标图结构进行识别，得到目标特征向量组，包括：
    调用预置的训练模型对目标图结构进行识别，生成多个候选特征；
    将多个候选特征划分为目标特征向量组x 1,x 2,K,x ni，其中，向量x ni包含ni个候选特征。
  6. 根据权利要求1所述的多标签多示例的图像检测方法，其中，所述通过所述预置的多示例多标签模型对所述目标特征向量组进行标注，得到多个目标病理标签，包括：
    根据目标特征向量组x 1,x 2,K,x ni的向量数量确定预测模型的数量；
    根据预置候选病例数据将目标特征向量组划分为多个预测向量组x 1,x 2,K,x ni，每个预测向量组对应一个预置候选病例数据；
    调用所述预置的多示例多标签模型对每个预测向量组的病理标签T s，得到多个预测值F D(T)，所述预置的多示例多标签模型为：
    F D(f s(θ 1x 1),f s(θ 2x 2),f s(θ 3x 3)...f s(θ nix ni)|θ 1,θ 2,θ 3...θ ni)；
    根据多个预测值F D(T)生成多个病理标签T 1,T 2,T 3...T s，每个病理标签对应一段病理描述文字。
  7. 根据权利要求1-6中任一项所述的多标签多示例的图像检测方法，其中，在所述通过所述预置的多示例多标签模型对所述目标特征向量组进行标注，得到多个目标病理标签之后，所述方法还包括：
    对所述多个目标病理标签进行校验，初始化所述预置的多示例多标签模型的模型参数。
  8. 一种多标签多示例的图像检测装置，其中，包括：
    第一获取单元，用于获取已确诊的活检图像的病理区域以及病理区域对应的病理特征描述；
    训练单元，用于根据所述病理区域和所述病理区域对应的病理特征描述对离线训练多标签多示例模型进行训练，得到预置的多示例多标签模型；
    第二获取单元，用于获取原始活检图像，所述原始活检图像用于指示人体组织的病理特性；
    提取单元，用于对所述原始活检图像进行纹理特征提取，并将提取到的纹理特征进行结构化特征表达，生成目标图结构；
    识别单元，用于调用预置的训练模型对目标图结构进行识别，得到目标特征向量组；
    标注单元，用于通过所述预置的多示例多标签模型对所述目标特征向量组进行标注，得到多个目标病理标签。
  9. 一种多标签多示例的图像检测设备，其中，包括存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现以下步骤：
    获取已确诊的活检图像的病理区域以及病理区域对应的病理特征描述；
    根据所述病理区域和所述病理区域对应的病理特征描述对离线训练多标签多示例模型进行训练，得到预置的多示例多标签模型；
    获取原始活检图像，所述原始活检图像用于指示人体组织的病理特性；
    对所述原始活检图像进行纹理特征提取，并将提取到的纹理特征进行结构化特征表达，生成目标图结构；
    调用预置的训练模型对目标图结构进行识别，得到目标特征向量组；
    通过所述预置的多示例多标签模型对所述目标特征向量组进行标注，得到多个目标病理标签。
  10. 根据权利要求9所述的多标签多示例的图像检测设备，其中，所述根据所述病理区域和所述病理区域对应的病理特征描述对离线训练多标签多示例模型进行训练，得到预置的多示例多标签模型时，具体实现以下步骤：
    根据所述病理区域进行纹理和结构特征提取，生成特征向量组；
    将所述特征向量组作为离线训练多标签多示例模型的输入，将所述病理区域对应的病理特征描述作为离线训练多标签多示例模型的输出，训练得到离线训练多标签多示例模型F D(f 1(θ 1x 1),f 1(θ 2x 2),f 1(θ 3x 3)...f 1(θ nix ni)|θ 1,θ 2,θ 3...θ ni)；
    调整θ 1,θ 2,θ 3...θ ni，使得似然函数值最小，似然函数为
    like(θ)＝F D(f 1(θ 1x 1),f 1(θ 2x 2),f 1(θ 3x 3)...f 1(θ nix ni)|θ 1,θ 2,θ 3...θ ni)-F D(T)；
    生成预置的多示例多标签模型，并保存最小似然函数值对应的模型参数θ 1,θ 2,θ 3...θ ni，预置的多示例多标签模型为F D(f s(θ 1x 1),f s(θ 2x 2),f s(θ 3x 3)...f s(θ nix ni)|θ 1,θ 2,θ 3...θ ni)。
  11. 根据权利要求9所述的多标签多示例的图像检测设备，其中，所述对所述原始活检图像进行纹理特征提取，并将提取到的纹理特征进行结构化特征表达，生成目标图结构时，具体实现以下步骤：
    对所述原始活检图像的纹理特征进行提取，得到多个目标纹理特征图像；
    采用预置聚类算法对所述多个目标纹理特征图像中的像素点进行分类，生成多个局部同类像素团；
    将所述多个局部同类像素团中的中心作为顶点并进行三角化，生成目标图结构。
  12. 根据权利要求11所述的多标签多示例的图像检测设备，其中，所述对所述原始活检图像的纹理特征进行提取，得到多个目标纹理特征图像时，具体实现以下步骤：
    将所述原始活检图像划分为n*n个目标区域，n为正整数；
    将每个目标区域中的一个目标像素和目标像素周围环形邻域内的8个像素点进行顺时针或逆时针的比较；
    若目标像素的像素值大于相邻像素点的像素值，则将相邻像素点赋值为1；
    若目标像素的像素值小于或等于相邻像素点的像素值，则将相邻像素点赋值为0；
    根据每个像素点的多个赋值结果生成每个像素点的8位二进制数，并转换成十进制数的数值；
    计算每个目标区域的直方图，并进行归一化处理得到统计直方图；
    将每个目标区域的统计直方图进行连接，生成原始活检图像的多个目标纹理特征图像。
  13. 根据权利要求9所述的多标签多示例的图像检测设备，其中，所述调用预置的训练模型对目标图结构进行识别，得到目标特征向量组时，具体实现以下步骤：
    调用预置的训练模型对目标图结构进行识别，生成多个候选特征；
    将多个候选特征划分为目标特征向量组x 1,x 2,K,x ni，其中，向量x ni包含ni个候选特征。
  14. 根据权利要求9所述的多标签多示例的图像检测设备，其中，所述通过所述预置的多示例多标签模型对所述目标特征向量组进行标注，得到多个目标病理标签时，具体实现以下步骤：
    根据目标特征向量组x 1,x 2,K,x ni的向量数量确定预测模型的数量；
    根据预置候选病例数据将目标特征向量组划分为多个预测向量组x 1,x 2,K,x ni，每个预测向量组对应一个预置候选病例数据；
    调用所述预置的多示例多标签模型对每个预测向量组的病理标签T s，得到多个预测值F D(T)，所述预置的多示例多标签模型为：
    F D(f s(θ 1x 1),f s(θ 2x 2),f s(θ 3x 3)...f s(θ nix ni)|θ 1,θ 2,θ 3...θ ni)；
    根据多个预测值F D(T)生成多个病理标签T 1,T 2,T 3...T s，每个病理标签对应一段病理描述文字。
  15. 一种计算机可读存储介质，其中，所述计算机可读存储介质存储有计算机程序，当所述计算机程序被处理器执行时实现以下步骤：
    获取已确诊的活检图像的病理区域以及病理区域对应的病理特征描述；
    根据所述病理区域和所述病理区域对应的病理特征描述对离线训练多标签多示例模型进行训练，得到预置的多示例多标签模型；
    获取原始活检图像，所述原始活检图像用于指示人体组织的病理特性；
    对所述原始活检图像进行纹理特征提取，并将提取到的纹理特征进行结构化特征表达，生成目标图结构；
    调用预置的训练模型对目标图结构进行识别，得到目标特征向量组；
    通过所述预置的多示例多标签模型对所述目标特征向量组进行标注，得到多个目标病理标签。
  16. 根据权利要求15所述的计算机可读存储介质，其中，所述根据所述病理区域和所述病理区域对应的病理特征描述对离线训练多标签多示例模型进行训练，得到预置的多示例多标签模型时，具体实现以下步骤：
    根据所述病理区域进行纹理和结构特征提取，生成特征向量组；
    将所述特征向量组作为离线训练多标签多示例模型的输入，将所述病理区域对应的病理特征描述作为离线训练多标签多示例模型的输出，训练得到离线训练多标签多示例模型F D(f 1(θ 1x 1),f 1(θ 2x 2),f 1(θ 3x 3)...f 1(θ nix ni)|θ 1,θ 2,θ 3...θ ni)；
    调整θ 1,θ 2,θ 3...θ ni，使得似然函数值最小，似然函数为
    like(θ)＝F D(f 1(θ 1x 1),f 1(θ 2x 2),f 1(θ 3x 3)...f 1(θ nix ni)|θ 1,θ 2,θ 3...θ ni)-F D(T)；
    生成预置的多示例多标签模型，并保存最小似然函数值对应的模型参数θ 1,θ 2,θ 3...θ ni，预置的多示例多标签模型为F D(f s(θ 1x 1),f s(θ 2x 2),f s(θ 3x 3)...f s(θ nix ni)|θ 1,θ 2,θ 3...θ ni)。
  17. 根据权利要求15所述的计算机可读存储介质，其中，所述对所述原始活检图像进行纹理特征提取，并将提取到的纹理特征进行结构化特征表达，生成目标图结构时，具体实现以下步骤：
    对所述原始活检图像的纹理特征进行提取，得到多个目标纹理特征图像；
    采用预置聚类算法对所述多个目标纹理特征图像中的像素点进行分类，生成多个局部同类像素团；
    将所述多个局部同类像素团中的中心作为顶点并进行三角化，生成目标图结构。
  18. 根据权利要求17所述的计算机可读存储介质，其中，所述对所述原始活检图像的纹理特征进行提取，得到多个目标纹理特征图像时，具体实现以下步骤：
    将所述原始活检图像划分为n*n个目标区域，n为正整数；
    将每个目标区域中的一个目标像素和目标像素周围环形邻域内的8个像素点进行顺时针或逆时针的比较；
    若目标像素的像素值大于相邻像素点的像素值，则将相邻像素点赋值为1；
    若目标像素的像素值小于或等于相邻像素点的像素值，则将相邻像素点赋值为0；
    根据每个像素点的多个赋值结果生成每个像素点的8位二进制数，并转换成十进制数的数值；
    计算每个目标区域的直方图，并进行归一化处理得到统计直方图；
    将每个目标区域的统计直方图进行连接，生成原始活检图像的多个目标纹理特征图像。
  19. 根据权利要求15所述的计算机可读存储介质，其中，所述调用预置的训练模型对目标图结构进行识别，得到目标特征向量组时，具体实现以下步骤：
    调用预置的训练模型对目标图结构进行识别，生成多个候选特征；
    将多个候选特征划分为目标特征向量组x 1,x 2,K,x ni，其中，向量x ni包含ni个候选特征。
  20. 根据权利要求15所述的计算机可读存储介质，其中，所述通过所述预置的多示例多标签模型对所述目标特征向量组进行标注，得到多个目标病理标签时，具体实现以下步骤：
    根据目标特征向量组x 1,x 2,K,x ni的向量数量确定预测模型的数量；
    根据预置候选病例数据将目标特征向量组划分为多个预测向量组x 1,x 2,K,x ni，每个预测向量组对应一个预置候选病例数据；
    调用所述预置的多示例多标签模型对每个预测向量组的病理标签T s，得到多个预测值F D(T)，所述预置的多示例多标签模型为：
    F D(f s(θ 1x 1),f s(θ 2x 2),f s(θ 3x 3)...f s(θ nix ni)|θ 1,θ 2,θ 3...θ ni)；
    根据多个预测值F D(T)生成多个病理标签T 1,T 2,T 3...T s，每个病理标签对应一段病理描述文字。
PCT/CN2020/098968 2020-03-09 2020-06-29 多标签多示例的图像检测方法、装置、设备及存储介质 Ceased WO2021179484A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010156699.6 2020-03-09
CN202010156699.6A CN111462048B (zh) 2020-03-09 2020-03-09 多标签多示例的图像检测方法、装置、设备及存储介质

Publications (1)

Publication Number Publication Date
WO2021179484A1 true WO2021179484A1 (zh) 2021-09-16

Family

ID=71678357

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/098968 Ceased WO2021179484A1 (zh) 2020-03-09 2020-06-29 多标签多示例的图像检测方法、装置、设备及存储介质

Country Status (2)

Country Link
CN (1) CN111462048B (zh)
WO (1) WO2021179484A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116485792A (zh) * 2023-06-16 2023-07-25 中南大学 组织病理学亚型预测方法及成像方法
CN117392468A (zh) * 2023-12-11 2024-01-12 山东大学 基于多示例学习的癌症病理图像分类系统、介质及设备

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112465819B (zh) * 2020-12-18 2024-06-18 平安科技（深圳）有限公司 图像异常区域检测方法、装置、电子设备及存储介质
CN112712093B (zh) * 2021-01-11 2024-04-05 中国铁道科学研究院集团有限公司电子计算技术研究所 安检图像识别方法、装置、电子设备及存储介质
CN113628226B (zh) * 2021-08-31 2025-04-25 深圳平安医疗健康科技服务有限公司 基于多标签分割模型的图像数据处理方法及装置
CN113744801B (zh) * 2021-09-09 2023-05-26 首都医科大学附属北京天坛医院 肿瘤类别的确定方法、装置、系统、电子设备及存储介质
CN116012769A (zh) * 2022-09-07 2023-04-25 广东海洋大学 一种基于GLCM与K-means聚类算法的船载雷达图像溢油识别方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103324940A (zh) * 2013-05-02 2013-09-25 广东工业大学 基于多示例多标签学习的皮肤病理图片特征识别方法
CN105678309A (zh) * 2016-02-03 2016-06-15 山东工商学院 一种基于多示例包特征学习的图像多标签标注算法
CN107330263A (zh) * 2017-06-26 2017-11-07 成都知识视觉科技有限公司 一种计算机辅助乳腺浸润性导管癌组织学分级的方法
CN109800789A (zh) * 2018-12-18 2019-05-24 中国科学院深圳先进技术研究院 基于图网络的糖尿病视网膜病变分类方法及装置
US20190252073A1 (en) * 2018-02-12 2019-08-15 Ai.Skopy, Inc. System and method for diagnosing gastrointestinal neoplasm

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106504255B (zh) * 2016-11-02 2019-04-09 南京大学 一种基于多标签多示例学习的多目标图像联合分割方法
CA2948499C (en) * 2016-11-16 2020-04-21 The Governing Council Of The University Of Toronto System and method for classifying and segmenting microscopy images with deep multiple instance learning

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103324940A (zh) * 2013-05-02 2013-09-25 广东工业大学 基于多示例多标签学习的皮肤病理图片特征识别方法
CN105678309A (zh) * 2016-02-03 2016-06-15 山东工商学院 一种基于多示例包特征学习的图像多标签标注算法
CN107330263A (zh) * 2017-06-26 2017-11-07 成都知识视觉科技有限公司 一种计算机辅助乳腺浸润性导管癌组织学分级的方法
US20190252073A1 (en) * 2018-02-12 2019-08-15 Ai.Skopy, Inc. System and method for diagnosing gastrointestinal neoplasm
CN109800789A (zh) * 2018-12-18 2019-05-24 中国科学院深圳先进技术研究院 基于图网络的糖尿病视网膜病变分类方法及装置

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116485792A (zh) * 2023-06-16 2023-07-25 中南大学 组织病理学亚型预测方法及成像方法
CN116485792B (zh) * 2023-06-16 2023-09-15 中南大学 组织病理学亚型预测方法及成像方法
CN117392468A (zh) * 2023-12-11 2024-01-12 山东大学 基于多示例学习的癌症病理图像分类系统、介质及设备
CN117392468B (zh) * 2023-12-11 2024-02-13 山东大学 基于多示例学习的癌症病理图像分类系统、介质及设备

Also Published As

Publication number Publication date
CN111462048B (zh) 2024-11-22
CN111462048A (zh) 2020-07-28

Similar Documents

Publication Publication Date Title
WO2021179484A1 (zh) 多标签多示例的图像检测方法、装置、设备及存储介质
CN113506334B (zh) 基于深度学习的多模态医学图像融合方法及系统
CN111369576B (zh) 图像分割模型的训练方法、图像分割方法、装置及设备
Alshayeji et al. Early detection of diabetic foot ulcers from thermal images using the bag of features technique
CN109035234B (zh) 一种结节检测方法、装置和存储介质
CN112086197B (zh) 基于超声医学的乳腺结节检测方法及系统
US20220058821A1 (en) Medical image processing method, apparatus, and device, medium, and endoscope
CN112308888B (zh) 基于深度学习体征结构的全模态医学影像序列分组方法
CN114445334B (zh) 图像分析的方法、装置、设备及存储介质
Hsieh et al. Combining VGG16, Mask R-CNN and Inception V3 to identify the benign and malignant of breast microcalcification clusters
CN115661141A (zh) 基于cbct图像牙齿和牙槽骨分割方法及系统
Buettner et al. A systematic literature review of medical image analysis using deep learning
Thaseen et al. Breast cancer detection using deep learning model
CN116563533A (zh) 基于目标位置先验信息的医学图像分割方法及系统
Gao et al. Deep learning-based framework for segmentation of multiclass rib fractures in CT utilizing a multi-angle projection network
CN115359060A (zh) 一种肺炎ct影像的病灶实例分割方法及装置
Abdelaziz et al. Skin Cancer detection using deep learning and artificial intelligence: incorporated model of deep features fusion
Celebi et al. Guest editorial skin lesion image analysis for melanoma detection
CN110472694A (zh) 一种肺癌图像病理分类方法及装置
Luong et al. A computer-aided detection to intracranial hemorrhage by using deep learning: A case study
Sun et al. Liver tumor segmentation and subsequent risk prediction based on Deeplabv3+
CN113808130B (zh) 肿瘤图像智能分类方法、装置、设备和存储介质
CN112633405A (zh) 模型训练方法、医学影像分析方法及装置、设备、介质
KR20250015794A (ko) 영상 및 비영상 데이터를 이용한 뇌출혈 예후 통합 예측 방법, 장치 및 컴퓨터 프로그램
Koundinya et al. Dhanvantari: An intelligent diagnosis tool to classify malignant skin disease and Lung conditions using Deep Learning

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20924427

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20924427

Country of ref document: EP

Kind code of ref document: A1