CN113112475B - Traditional Chinese medicine ear five-organ region segmentation method and device based on machine learning - Google Patents

Traditional Chinese medicine ear five-organ region segmentation method and device based on machine learning Download PDF

Info

Publication number
CN113112475B
CN113112475B CN202110396463.4A CN202110396463A CN113112475B CN 113112475 B CN113112475 B CN 113112475B CN 202110396463 A CN202110396463 A CN 202110396463A CN 113112475 B CN113112475 B CN 113112475B
Authority
CN
China
Prior art keywords
image
ear
region
images
edge
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110396463.4A
Other languages
Chinese (zh)
Other versions
CN113112475A (en
Inventor
梁惠珠
冯跃
林卓胜
朱嘉健
李胜可
刘慧琳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wuyi University
Original Assignee
Wuyi University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuyi University filed Critical Wuyi University
Priority to CN202110396463.4A priority Critical patent/CN113112475B/en
Publication of CN113112475A publication Critical patent/CN113112475A/en
Application granted granted Critical
Publication of CN113112475B publication Critical patent/CN113112475B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/12—Edge-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Mathematical Physics (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Medical Informatics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Geometry (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a traditional Chinese medicine ear five-zang-organ region segmentation method and a device based on machine learning, wherein the method comprises the following steps: collecting ear images and establishing an image data set; labeling the ear image to obtain a label image; respectively expanding the ear images and the tag images; carrying out Canny operator edge detection on the expansion tag image; respectively preprocessing the extended ear image, the extended tag image and the tag edge image; generating edge images of each single region of the heart, the liver, the spleen, the lung and the kidney by utilizing the gray level images and the binary images; inputting the RGB three-channel image into a learning network to obtain a predicted image; obtaining a predicted image of each single region of the heart, the liver, the spleen, the lung and the kidney from the predicted image; respectively carrying out expansion and corrosion treatment on the predicted image to obtain a single-region rough edge prediction image; calculating a total loss function; and calculating the minimum value of the total loss function to make the learning network converge, and segmenting the ear image by using the learning network to obtain the ear five-organ region segmentation image.

Description

Traditional Chinese medicine ear five-organ region segmentation method and device based on machine learning
Technical Field
The invention relates to the field of image processing, in particular to a method and a device for segmenting ear five-organ regions in traditional Chinese medicine based on machine learning.
Background
In recent years, machine learning widely encompasses various fields of images and achieves superior performance such as image recognition, object detection, image segmentation, and the like. It is worth mentioning that the deep learning belonging to the machine learning category applied to the research of medical images has important significance for assisting the clinical diagnosis of physicians, and the tongue diagnosis image processing in the inspection of traditional Chinese medicine has achieved a lot of achievements. However, the digital auxiliary diagnosis of ear diagnosis is still rare, and especially the digitization of ear inspection does not appear in any form, wherein, image segmentation is a necessary step of clinical application, and although the ear biological information identification based on deep learning appears, the ear region segmentation based on deep learning is less, and because the boundary of each segmentation region is based on the theory of traditional Chinese medicine, the region boundary is not the inherent boundary contour of an object, and has certain ambiguity, the segmentation difficulty is brought.
Disclosure of Invention
The present invention is directed to solving at least one of the problems of the prior art. Therefore, the invention provides a traditional Chinese medicine ear five-zang-organ region segmentation method and device based on machine learning, and the traditional Chinese medicine ear five-zang-organ region segmentation method and device based on machine learning enable the corresponding region of the traditional Chinese medicine ear five-zang-organ to be automatically segmented.
In a first aspect, an embodiment of the present invention provides a method for segmenting the five ear viscera regions in the traditional Chinese medicine with the above functions. The method comprises the following steps:
collecting ear images; the acquiring ear images includes:
the existing public ear data set is fully utilized, and ear images meeting the ear diagnosis requirement of the traditional Chinese medicine are screened out to serve as original data of the data set;
the other part is that according to the ear diagnosis requirement of the traditional Chinese medicine, the original data are automatically collected, the collection equipment is a traditional Chinese medicine tongue picture collection instrument, after the left (right) side ear of the collection object is close to the collection instrument, the head rotates to an appropriate angle of about 30 degrees to the left (right), so that the five viscera region of the ear is not shielded in the collection process, the original collection image comprises the ear, the face and the edge of the collection instrument, the position of the ear is detected by the original collection image through a trained Opencv ear detector model, the condition that the ear region is not completely contained in the detection frame exists, therefore, the detection frame is taken as the center, the detection frame is cut by 1.6 times of the width and height of the detection frame, in the cut image, the complete ear region occupies about half of the area of the whole image, and the ear image is obtained.
Labeling the ear image to obtain a label image;
respectively expanding the ear image and the tag image to obtain an expanded ear image and an expanded tag image;
carrying out Canny operator edge detection on the expansion tag image to obtain a tag edge image;
respectively preprocessing the extended ear image, the extended tag image and the tag edge image to respectively obtain an RGB three-channel image, a gray image and a binary image;
generating edge images of each single region of the heart, the liver, the spleen, the lung and the kidney by using the gray level images and the binary images;
inputting the RGB three-channel image into a learning network to obtain a predicted image;
obtaining a predicted image of each single region of the heart, the liver, the spleen, the lung and the kidney from the predicted image;
respectively expanding and corroding the single region predicted images of the heart, the liver, the spleen, the lung and the kidney to obtain single region rough edge prediction images of the corresponding regions;
calculating a total loss function comprising a first loss function of the grayscale image and the predicted image and a second loss function of the single region edge image and the single region coarse edge prediction image;
and calculating the minimum value of the total loss function, converging the learning network, and segmenting the ear image by using the learning network to obtain an ear five-organ region segmentation image.
The method for segmenting the regions of the five ear zang organs in the traditional Chinese medicine based on machine learning at least has the technical effect that the regions corresponding to the five ear zang organs in the traditional Chinese medicine can be automatically segmented.
According to the traditional Chinese medicine ear five-organ region segmentation method based on machine learning, the ear image and the tag image are expanded to obtain an expanded ear image and an expanded tag image, and the method comprises the following steps:
rotating the ear image, changing the size of the ear image, horizontally turning the ear image and performing gamma conversion on the ear image to obtain the extended ear image;
and rotating the tag image in the same way as the ear image, changing the size of the tag image, horizontally turning the tag image, and enabling the tag image to only have five regions with corresponding pixel values of the heart, the liver, the spleen, the lung and the kidney, wherein the gray pixel values are respectively 50, 100, 150, 200 and 250, the regions except the five regions are background regions, and the pixel value is 0, so as to obtain the extended tag image.
According to the traditional Chinese medicine ear five-organ region segmentation method based on machine learning, the ear image is labeled to obtain a label image, and the method comprises the following steps:
labeling the ear images by using a LabelMe tool box according to a Chinese standard ear acupoint positioning schematic diagram;
and converting the labeling information of the ear images to obtain label images corresponding to each ear image, wherein six areas corresponding to the center of the Chinese standard ear acupoint positioning schematic diagram, the liver, the spleen, the lung and the kidney and the rest areas in the label images are taken as backgrounds and are respectively distinguished by different pixel values.
A data set consisting of the ear images and the corresponding label images checked by professional Chinese physicians is divided into a training set, a verification set and a testing set according to the proportion of 60%,20% and 20%, a deep neural network U-Net is used as a training model, and an Intersection over Unit (Intersection over Unit) IoU is used as a technical index. On one hand, in the training process, the intersection ratio response sequence of the five labeled regions (heart, liver, spleen, lung and kidney) in the verification set is lung, kidney, liver, spleen and heart, and on the other hand, the average intersection ratio of the five labeled regions on the test set is more than 0.38 after the training is finished. The data set that meets the criteria of both aspects is determined to be the last data set.
According to the traditional Chinese medicine ear five-organ region segmentation method based on machine learning, the method for preprocessing the extended ear image and the tag edge image respectively comprises the following steps:
and performing position shearing and normalization operation on the extended ear image to obtain the RGB three-channel image with the size of W x H.
And cutting the position of the expansion tag image corresponding to the expansion ear image to obtain the gray level image with the size of W x H, wherein the gray level image is divided into six areas of a background, a heart, a liver, a spleen, a lung and a kidney, and the gray level pixel values of the six areas are respectively 0,1, 2, 3, 4 and 5.
And cutting the position of the label edge image corresponding to the extended ear image to obtain the binary image with the size of W x H.
According to the traditional Chinese medicine ear five-organ region segmentation method based on machine learning, the gray level image and the binary image are used for generating edge images of each single region of a heart, a liver, a spleen, a lung and a kidney, and the method comprises the following steps:
let f label (x, y) represents a pixel value at an arbitrary coordinate (x, y) of the gray image, let f edge (x, y) represents a pixel value at an arbitrary coordinate (x, y) of the binary image, g i (x, y) represents the corresponding transformed pixel values at these coordinates, having
Figure BDA0003018755690000041
Where i ∈ {1,2, …,5}, x ∈ {1,2, …, W }, y ∈ {1,2, …, H }, W and H are the width and height of the grayscale image and the binary image, respectively, g 1 、g 2 、g 3 、g 4 、g 5 The single region edge images of heart, liver, spleen, lung, kidney, respectively.
The machine learning-based traditional Chinese medicine ear five-organ region segmentation method of claim 1, wherein inputting the RGB three-channel image into a learning network to obtain a predicted image comprises:
will be describedThe RGB three-channel image passes through two convolution layers with convolution kernels of 3*3 to obtain a characteristic F with 64 channels and H multiplied by W size origin ；
The feature F is measured origin Obtaining feature F by PAM module 1 。
The feature F is measured 1 Obtaining feature F by PAM module 2 。
The feature F 1 And said feature F 2 And splicing the two results obtained by multiplying the coefficients by 0.75 and 0.25 respectively on channels, and then obtaining the predicted image with the channel of 6 and the size of H multiplied by W through a convolution layer with a convolution kernel of 3*3.
According to the traditional Chinese medicine ear five-organ region segmentation method based on machine learning, the single region prediction images of the heart, the liver, the spleen, the lung and the kidney are obtained from the prediction images, and the method comprises the following steps:
taking the maximum value of the predicted image in the dimension of a channel to obtain a dimension gray image with the size of H multiplied by W and the pixel values only having six pixel values of 0,1, 2, 3, 4 and 5, wherein the regions corresponding to the six pixel values in the dimension gray image are respectively prediction segmentation regions of a background, a heart, a liver, a spleen, a lung and a kidney;
let P (x, y) denote the pixel value at any coordinate (x, y) of the predicted grayscale image, there are
Figure BDA0003018755690000051
Wherein i ∈ {1,2, …,5}, x ∈ {1,2, …, W }, y ∈ {1,2, …, H }, W and H are the width and height of the dimension grayscale image, respectively, P (x, y) = {0,1, …,5}, P ∈ is 1 、p 2 、p 3 、p 4 、p 5 The single region of heart, liver, spleen, lung, kidney, respectively, predicts the image.
According to the traditional Chinese medicine ear five-heart region segmentation method based on machine learning, the single region prediction images of the heart, the liver, the spleen, the lung and the kidney are respectively subjected to expansion and corrosion processing to obtain the single region rough edge prediction image of the corresponding region, and the method comprises the following steps:
defining a structural element B, wherein the size of the structural element B is 3*3, the origin is located at the center, and the element values are all 1;
expanding and corroding the single-region predicted images of the heart, the liver, the spleen, the lung and the kidney by using the structural element B to obtain the expanded single-region predicted image and the corroded single-region predicted image, and subtracting the corroded single-region predicted image and the single-region predicted image to obtain a corresponding rough edge prediction image of the region;
let pe i (x, y) represents the pixel value of any coordinate of the predicted image at the rough edge of the region, including
Figure BDA0003018755690000052
In the formula, symbol
Figure BDA0003018755690000053
For the expansion operation, the sign->
Figure BDA0003018755690000054
For the erosion operation, i ∈ {1,2, …,5}, x ∈ {1,2, …, W }, y ∈ {1,2, …, H }, W and H are the width and height of the image, pe and H, respectively i (x,y)∈{0,1}，，pe 1 、pe 2 、pe 3 、pe 4 、pe 5 The single region rough edge prediction images of heart, liver, spleen, lung, kidney, respectively.
According to the traditional Chinese medicine ear five-organ region segmentation method based on machine learning, a total loss function is calculated, wherein the total loss function comprises a first loss function of the gray-scale image and the predicted image and a second loss function of the single region edge image and the single region rough edge prediction image:
the first loss function is
Figure BDA0003018755690000061
Wherein x ∈ {1,2, …, W }, y ∈ {1,2, …, H }, W and H are the width and height of the grayscale image and the prediction image, respectively, C ∈ {1, …, C }, C is the number of division categories,
Figure BDA0003018755690000062
for the binary image with class c one-hot encoded, ->
Figure BDA0003018755690000065
q c (x, y) is the predicted probability that the coordinate (x, y) pixel belongs to class c;
the second loss function is
Figure BDA0003018755690000063
Wherein i ∈ {1,2, …,5}, x ∈ {1,2, …, W }, y ∈ {1,2, …, H }, W and H are the width and height of the single region edge image and the single region rough edge prediction map, respectively, and g i (x, y) is the ith area edge image, g i (x, y) is an element of {0,1}, AND is traffic, pe i (x, y) is the ith area coarse edge prediction image, pe i (x,y)∈{0,1}。
The total loss function is
Figure BDA0003018755690000064
In the formula, the parameter N is l i The number of the carbon atoms is not 0.
In a second aspect, an embodiment of the present invention further provides a device for segmenting the ear five viscera region in traditional Chinese medicine based on machine learning, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, where when the processor executes the program, the method for segmenting the ear five viscera region in traditional Chinese medicine based on machine learning according to the first aspect of the present invention is implemented.
Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Drawings
The above and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
FIG. 1 is a flowchart of a method for segmenting the five ear zang-organ regions according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a process of segmenting the five ear zang-organ regions according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of the positioning of the standard auricular point in China according to the embodiment of the present invention;
Detailed Description
Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the accompanying drawings are illustrative only for the purpose of explaining the present invention, and are not to be construed as limiting the present invention.
In the description of the present invention, it should be understood that the orientation or positional relationship referred to in the description of the orientation, such as the upper, lower, front, rear, left, right, etc., is based on the orientation or positional relationship shown in the drawings, and is only for convenience of description and simplification of description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus, should not be construed as limiting the present invention.
In the description of the present invention, the meaning of a plurality of means is one or more, the meaning of a plurality of means is two or more, and larger, smaller, larger, etc. are understood as excluding the number, and larger, smaller, inner, etc. are understood as including the number. If the first and second are described for the purpose of distinguishing technical features, they are not to be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated or implicitly indicating the precedence of the technical features indicated.
In the description of the present invention, unless otherwise explicitly limited, terms such as arrangement, installation, connection and the like should be understood in a broad sense, and those skilled in the art can reasonably determine the specific meanings of the above terms in the present invention in combination with the specific contents of the technical solutions.
Referring to fig. 1,2 and 3, a method for segmenting the five ear regions of chinese medicine based on machine learning according to a first embodiment of the present invention is described. The method comprises the following steps:
s101: collecting ear images; the acquisition of ear images includes:
the existing public ear data set is fully utilized, and ear images meeting the ear diagnosis requirements of the traditional Chinese medicine are screened out to serve as original data of the data set;
the other part is that according to the ear diagnosis requirement of the traditional Chinese medicine, the original data are automatically collected, the collection equipment is a traditional Chinese medicine tongue picture collection instrument, after the left (right) side ear of the collection object is close to the collection instrument, the head rotates to an appropriate angle of about 30 degrees to the left (right), so that the five viscera region of the ear is not shielded in the collection process, the original collection image comprises the ear, the face and the edge of the collection instrument, the position of the ear is detected by the original collection image through a trained Opencv ear detector model, the condition that the ear region is not completely contained in the detection frame exists, therefore, the detection frame is taken as the center, the detection frame is cut by 1.6 times of the width and height of the detection frame, in the cut image, the complete ear region occupies about half of the area of the whole image, and the ear image is obtained.
S102: labeling the ear image to obtain a label image;
s103: respectively expanding the ear image and the tag image to obtain an expanded ear image and an expanded tag image;
s104: carrying out Canny operator edge detection on the expansion tag image to obtain a tag edge image;
s105: respectively preprocessing the extended ear image, the extended tag image and the tag edge image to respectively obtain an RGB three-channel image, a gray image and a binary image;
s106: generating edge images of each single region of the heart, the liver, the spleen, the lung and the kidney by using the gray level images and the binary images;
s107: inputting the RGB three-channel image into a learning network to obtain a predicted image;
s108: obtaining a predicted image of each single region of the heart, the liver, the spleen, the lung and the kidney from the predicted image;
s109: respectively expanding and corroding the single region predicted images of the heart, the liver, the spleen, the lung and the kidney to obtain single region rough edge prediction images of the corresponding regions;
s110: calculating a total loss function, wherein the total loss function comprises a first loss function of the gray-scale image and the prediction image and a second loss function of the single-region edge image and the single-region rough edge prediction image;
s111: and calculating the minimum value of the total loss function to make the learning network converge, and segmenting the ear image by using the learning network to obtain an ear five-organ region segmentation image.
The method for segmenting the ear five-zang-organ areas in the traditional Chinese medicine based on the machine learning at least has the technical effect that the corresponding areas of the ear five-zang-organ areas in the traditional Chinese medicine can be automatically segmented.
Referring to fig. 1 and 2, according to a machine learning-based traditional Chinese medicine ear five-organ region segmentation method according to a first embodiment of the present invention, labeling an ear image to obtain a label image, includes: labeling the ear images by using a LabelMe tool box according to a Chinese standard ear acupoint positioning schematic diagram; and converting the labeling information of the ear images to obtain label images corresponding to each ear image, wherein six areas corresponding to the center of the Chinese standard ear acupoint positioning schematic diagram, the liver, the spleen, the lung and the kidney and the rest areas in the label images are taken as backgrounds and are respectively distinguished by different pixel values.
The ear images and the corresponding data set formed by the label images checked by professional Chinese physicians are divided into a training set, a verification set and a test set according to the proportion of 60%,20% and 20%, a deep neural network U-Net is used as a training model, and an Intersection over Unit (Intersection over Unit) IoU is used as a technical index. In one aspect, during training, the intersection specific response sequence of five labeled regions (heart, liver, spleen, lung, kidney) in the set is verified as lung, kidney, liver, spleen, heart. On the other hand, the average intersection ratio of five labeled regions on the test set after training is over 0.38. The data set that meets the criteria of both aspects is determined to be the last data set.
Referring to fig. 1 and 2, according to a method for segmenting a five-organ ear region in traditional Chinese medicine based on machine learning according to a first embodiment of the present invention, the expanding the ear image and the tag image to obtain an expanded ear image and an expanded tag image includes:
rotating the ear image, changing the size of the ear image, horizontally turning the ear image and performing gamma conversion on the ear image to obtain the extended ear image;
and rotating the tag image in the same way as the ear image, changing the size of the tag image, horizontally turning the tag image, and enabling the tag image to only have five pixel values corresponding to five areas of the heart, the liver, the spleen, the lung and the kidney, wherein the gray pixel values are respectively 50, 100, 150, 200 and 250, the areas except the five areas are background areas, and the pixel value is 0, so as to obtain the extended tag image.
Referring to fig. 1 and 2, the method for segmenting the five ear regions in traditional Chinese medicine based on machine learning according to the first embodiment of the present invention respectively preprocesses the extended ear image and the tag edge image, and includes:
and performing position shearing and normalization operation on the extended ear image to obtain the RGB three-channel image with the size of W x H.
And cutting the position of the expansion tag image corresponding to the expansion ear image to obtain the gray level image with the size of W x H, wherein the gray level image is divided into six areas of a background, a heart, a liver, a spleen, a lung and a kidney, and the gray level pixel values of the six areas are respectively 0,1, 2, 3, 4 and 5.
And cutting the positions of the label edge images corresponding to the extended ear images to obtain the binary image with the size of W x H.
Referring to fig. 1 and 2, the method for segmenting five ear regions in traditional Chinese medicine based on machine learning according to the first embodiment of the present invention, which generates edge images of each single region of heart, liver, spleen, lung and kidney by using the gray image and the binary image, includes:
let f label (x, y) represents a pixel value at an arbitrary coordinate (x, y) of the gray image, let f edge (x, y) represents a pixel value at an arbitrary coordinate (x, y) of the binary image, g i (x, y) represents the corresponding transformed pixel values at these coordinates, having
Figure BDA0003018755690000101
Where i ∈ {1,2, …,5}, x ∈ {1,2, …, W }, y ∈ {1,2, …, H }, W and H are the width and height of the grayscale image and the binary image, respectively, g 1 、g 2 、g 3 、g 4 、g 5 The single region edge images of heart, liver, spleen, lung, kidney, respectively.
Referring to fig. 1 and 2, the method for segmenting the five ear regions in traditional Chinese medicine based on machine learning according to the embodiment of the present invention generates edge images of each single region of the heart, liver, spleen, lung and kidney by using the gray image and the binary image, including:
referring to fig. 1 and 2, the method for segmenting ear regions and five viscera regions in traditional Chinese medicine based on machine learning according to the first embodiment of the invention is characterized in that the RGB three-channel images are input into a learning network to obtain a predicted image, and the method comprises the following steps:
the RGB three-channel image is subjected to convolution layer with two convolution kernels of 3*3 to obtain a characteristic F with 64 channels and H multiplied by W size origin ；
The feature F is measured origin Obtaining feature F by PAM module 1 ；
The feature F 1 Obtaining feature F by PAM module 2 ；
The feature F is measured 1 And the feature F 2 And splicing the two results obtained by multiplying the coefficients by 0.75 and 0.25 respectively on channels, and then obtaining the predicted image with the channel of 6 and the size of H multiplied by W through a convolution layer with a convolution kernel of 3*3.
Referring to fig. 1 and 2, the method for segmenting the five ear regions in traditional Chinese medicine based on machine learning according to the first embodiment of the present invention obtains, from the predicted images, predicted images of individual regions of the heart, liver, spleen, lung and kidney, including:
taking the maximum value of the predicted image in the dimension of the channel to obtain a dimension gray image with the size of H multiplied by W and the pixel value of only six pixel values of 0,1, 2, 3, 4 and 5, wherein the corresponding regions of the six pixel values in the dimension gray image are respectively prediction segmentation regions of a background, a heart, a liver, a spleen, a lung and a kidney;
let P (x, y) denote the pixel value at any coordinate (x, y) of the predicted grayscale image, as
Figure BDA0003018755690000111
Wherein i ∈ {1,2, …,5}, x ∈ {1,2, …, W }, y ∈ {1,2, …, H }, W and H are the width and height of the dimension grayscale image, respectively, P (x, y) = {0,1, …,5}, P ∈ is 1 、p 2 、p 3 、p 4 、p 5 The single region of heart, liver, spleen, lung, kidney, respectively, predicts the image.
Referring to fig. 1 and fig. 2, according to the method for segmenting the five ear regions in traditional Chinese medicine based on machine learning according to the first embodiment of the present invention, the expanding and eroding processing is performed on the predicted images of the single regions of the heart, liver, spleen, lung and kidney respectively to obtain the single-region rough edge prediction map of the corresponding region, including:
defining a structural element B, wherein the size of the structural element B is 3*3, the origin is located at the center, and the element values are all 1;
expanding and corroding the single-region predicted images of the heart, the liver, the spleen, the lung and the kidney by using the structural element B to obtain the expanded single-region predicted image and the corroded single-region predicted image, and subtracting the corroded single-region predicted image from the expanded single-region predicted image to obtain a corresponding region rough edge prediction map;
let pe i (x, y) represents the pixel value of any coordinate of the predicted image at the rough edge of the region, including
Figure BDA0003018755690000121
In the formula, symbol
Figure BDA0003018755690000122
For the expansion operation, the sign->
Figure BDA0003018755690000123
For the erosion operation, i ∈ {1,2, …,5}, x ∈ {1,2, …, W }, y ∈ {1,2, …, H }, W and H are the width and height of the image, pe, respectively i (x,y)∈{0,1}，，pe 1 、pe 2 、pe 3 、pe 4 、pe 5 The single region rough edge prediction images of heart, liver, spleen, lung, kidney, respectively.
Referring to fig. 1 and 2, a method for segmenting a five-zang ear region in traditional Chinese medicine based on machine learning according to a first embodiment of the present invention is characterized by comprising: calculating a total loss function comprising a first loss function of the grayscale image and the predicted image and a second loss function of the single region edge image and the single region coarse edge prediction map:
the first loss function is
Figure BDA0003018755690000124
Wherein x ∈ {1,2, …, W }, y ∈ {1,2, …, H }, W and H are the width and height of the grayscale image and the prediction image, respectively, C ∈ {1, …, C }, C is the number of segmentation categories,
Figure BDA0003018755690000125
for the binary image with class c one-hot encoded, ->
Figure BDA0003018755690000126
q c (x, y) is the predicted probability that the coordinate (x, y) pixel belongs to class c;
the second loss function is
Figure BDA0003018755690000127
Wherein i ∈ {1,2, …,5}, x ∈ {1,2, …, W }, y ∈ {1,2, …, H }, W and H are the width and height of the single region edge image and the single region coarse edge prediction map, respectively, g i (x, y) is the ith area edge image, g i (x, y) is an element of {0,1}, AND is traffic, pe i (x, y) is the ith area coarse edge prediction image, pe i (x,y)∈{0,1}。
The total loss function is
Figure BDA0003018755690000131
In the formula, the parameter N is l i The number of the carbon atoms is not 0.
A traditional Chinese medicine ear five-organ region segmentation device based on machine learning is characterized by comprising at least one processor and a memory connected with the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any of the above ear-five zang region segmentation in traditional Chinese medicine.
It should be recognized that the method steps in embodiments of the present invention may be embodied or carried out by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer readable memory. The method may use standard programming techniques. Each program may be implemented in a high level procedural or object oriented programming language to communicate with a computer system. However, the program(s) can be implemented in assembly or machine language, if desired. In any case, the language may be a compiled or interpreted language. Furthermore, the program can be run on a programmed application specific integrated circuit for this purpose.
Further, the operations of processes described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The processes described herein (or variations and/or combinations thereof) may be performed under the control of one or more computer systems configured with executable instructions and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) collectively executed on one or more processors, by hardware, or combinations thereof. The computer program includes a plurality of instructions executable by one or more processors.
Further, the method may be implemented in any type of computing platform operatively connected to a suitable interface, including but not limited to a personal computer, mini computer, mainframe, workstation, networked or distributed computing environment, separate or integrated computer platform, or in communication with a charged particle tool or other imaging device, and the like. Aspects of the invention may be embodied in machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, optically read and/or write storage medium, RAM, ROM, or the like, such that it may be read by a programmable computer, which when read by the storage medium or device, is operative to configure and operate the computer to perform the procedures described herein. Further, the machine-readable code, or portions thereof, may be transmitted over a wired or wireless network. The invention described herein includes these and other different types of non-transitory computer-readable storage media when such media include instructions or programs that implement the steps described above in conjunction with a microprocessor or other data processor. The invention also includes the computer itself when programmed according to the methods and techniques described herein.
A computer program can be applied to input data to perform the functions described herein to transform the input data to generate output data that is stored to non-volatile memory. The output information may also be applied to one or more output devices, such as a display. In a preferred embodiment of the invention, the transformed data represents physical and tangible objects, including particular visual depictions of physical and tangible objects produced on a display.
The embodiments of the present invention have been described in detail with reference to the accompanying drawings, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge of those skilled in the art without departing from the gist of the present invention.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an illustrative embodiment," "an example," "a specific example," or "some examples" or the like mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
While embodiments of the invention have been shown and described, it will be understood by those of ordinary skill in the art that: various changes, modifications, substitutions and alterations can be made to the embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims (9)

1. A traditional Chinese medicine ear five-organ region segmentation method based on machine learning is characterized by comprising the following steps:
collecting ear images and establishing an image data set;
labeling the ear image to obtain a label image;
respectively expanding the ear image and the tag image to obtain an expanded ear image and an expanded tag image;
carrying out Canny operator edge detection on the extended label image to obtain a label edge image;
preprocessing the extended ear image, the extended label image and the label edge image respectively to obtain an RGB three-channel image, a gray image and a binary image respectively, wherein the sizes of the RGB three-channel image, the gray image and the binary image are W multiplied by H;
generating edge images of each single region of the heart, the liver, the spleen, the lung and the kidney by using the gray level images and the binary images;
inputting the RGB three-channel image into a learning network to obtain a predicted image;
obtaining a prediction image of each single region of the heart, the liver, the spleen, the lung and the kidney from the prediction image;
respectively expanding and corroding the single region predicted images of the heart, the liver, the spleen, the lung and the kidney to obtain single region rough edge prediction images of the corresponding regions;
computing a total loss function comprising a first loss function of the grayscale image and the predicted image and a second loss function of the single region edge image and the single region coarse edge prediction map, wherein,
the first loss function is
Figure FDA0004065900810000011
Wherein X ∈ {1,2., W }, y ∈ {1,2., H }, W being widths of the grayscale image and the predictive image, respectively, H being heights of the grayscale image and the predictive image, respectively, C ∈ {1,2., X }, C being the number of segmentation classes,
Figure FDA0004065900810000012
the binary image @ -one encoded for category c>
Figure FDA0004065900810000013
q c (x, y) is the predicted probability that the coordinate (x, y) pixel belongs to class c;
the second loss function is
Figure FDA0004065900810000021
In the formula, i is equal to {1,2,., 5}, x is equal to {1,2,.., W }, y is equal to {1,2,., H }, W is the width of the single-region edge image and the single-region rough edge prediction image respectively, H is the height of the single-region edge image and the single-region rough edge prediction image respectively, and g is the height of the single-region edge image and the single-region rough edge prediction image respectively i (x, y) ∈ {0,1} is the ith area edge image, g i (x, y) is an element of {0,1}, AND is traffic, pe i (x, y) is the ith area coarse edge prediction image, pe i (x，y)∈{0，1}，
The total loss function is
Figure FDA0004065900810000022
In the formula, the parameter N is l i A number other than 0;
and calculating the minimum value of the total loss function, converging the learning network, and segmenting the ear image by using the learning network to obtain an ear five-organ region segmentation image.
2. The method for segmenting ear regions of traditional Chinese medicine based on machine learning according to claim 1, wherein labeling the ear images to obtain labeled images comprises:
labeling the ear images by using a LabelMe tool box according to a Chinese standard ear acupoint positioning schematic diagram;
and converting the labeling information of the ear images to obtain label images corresponding to each ear image, wherein six areas corresponding to the center of the Chinese standard ear acupoint positioning schematic diagram, the liver, the spleen, the lung and the kidney and the rest areas in the label images are taken as backgrounds and are respectively distinguished by different pixel values.
3. The method of machine learning-based segmentation of five ear regions in traditional Chinese medicine according to claim 1, wherein the expanding the ear image and the tag image to obtain an expanded ear image and an expanded tag image comprises:
rotating the ear image, changing the size of the ear image, horizontally turning the ear image and performing gamma conversion on the ear image to obtain the extended ear image;
and rotating the tag image in the same way as the ear image, changing the size of the tag image, horizontally turning the tag image, and enabling the tag image to only have five pixel values corresponding to five areas of the heart, the liver, the spleen, the lung and the kidney, wherein the gray pixel values are respectively 50, 100, 150, 200 and 250, the areas except the five areas are background areas, and the pixel value is 0, so as to obtain the extended tag image.
4. The method for segmenting the five ear regions in traditional Chinese medicine based on machine learning according to claim 1, wherein the pre-processing is performed on the augmented ear image and the tag edge image respectively, and includes:
performing position shearing and normalization operation on the extended ear image to obtain an RGB three-channel image with the size of WxH;
cutting the position of the expansion tag image corresponding to the expansion ear image to obtain the gray level image with the size of W multiplied by H, wherein the gray level image is divided into six areas of a background, a heart, a liver, a spleen, a lung and a kidney, and gray level pixel values of the six areas are 0,1, 2, 3, 4 and 5 respectively;
and cutting the position of the label edge image corresponding to the extended ear image to obtain the binary image with the size of W multiplied by H.
5. The method of claim 1, wherein the generating the edge image of each single region of heart, liver, spleen, lung and kidney by using the gray image and the binary image comprises:
let f label (x, y) represents a pixel value at an arbitrary coordinate (x, y) of the gray image, let f edge (x, y) represents a pixel value at an arbitrary coordinate (x, y) of the binary image, g i (x, y) represents the corresponding transformed pixel values at these coordinates, having
Figure FDA0004065900810000031
Wherein i ∈ {1,2., 5}, x ∈ {1,2., W }, y ∈ {1,2., H }, W is the width of the grayscale image and the binary image, H is the height of the grayscale image and the binary image, respectively, g ∈ {1,2., 5}, x ∈ {1,2 }, W ∈ {1,2 }, y } is the width of the grayscale image and the binary image, respectively, H is the height of the grayscale image and the binary image, respectively 1 、g 2 、g 3 、g 4 、g 5 The single region edge images of heart, liver, spleen, lung, kidney, respectively.
6. The method for segmenting the five ear regions in traditional Chinese medicine based on machine learning according to claim 1, wherein the inputting of the RGB three-channel image into a learning network to obtain a predicted image comprises:
the RGB three-channel image passes through two convolution layers with convolution kernels of 3*3 to obtain a characteristic F with 64 channels and H multiplied by W size origin ；
The feature F origin Obtaining feature F by PAM module 1 ；
The feature F 1 Obtaining feature F by PAM module 2 ；
The feature F is measured 1 And said feature F 2 Splicing the two results obtained by multiplying the coefficients by 0.75 and 0.25 respectively on the channel, and obtaining the prediction with the channel of 6 and the size of H multiplied by W through the convolution layer with the convolution kernel of 3*3And (4) an image.
7. The method according to claim 1, wherein obtaining a prediction image of each single region of heart, liver, spleen, lung and kidney from the prediction image comprises:
taking the maximum value of the predicted image in the dimension of a channel to obtain a dimension gray image with the size of H multiplied by W and the pixel values only having six pixel values of 0,1, 2, 3, 4 and 5, wherein the regions corresponding to the six pixel values in the dimension gray image are respectively prediction segmentation regions of a background, a heart, a liver, a spleen, a lung and a kidney;
let P (x, y) denote the pixel value at any coordinate (x, y) of the predicted grayscale image, there are
Figure FDA0004065900810000041
Where i ∈ {1,2, ·,5}, x ∈ {1,2,. Said, W }, y ∈ {1,2,. Said, H }, W and H are the width and height of the dimensional grayscale image, respectively, P (x, y) = {0,1,. Said, 5}, P (x, y) } 1 、p 2 、p 3 、p 4 、p 5 The single regions of heart, liver, spleen, lung, kidney, respectively, predict the image.
8. The method according to claim 7, wherein the expanding and eroding the single region prediction images of the heart, liver, spleen, lung and kidney to obtain the single region rough edge prediction map of the corresponding region comprises:
defining a structural element B, wherein the size of the structural element B is 3*3, the origin is located at the center, and the element values are all 1;
expanding and corroding the single region prediction images of the heart, the liver, the spleen, the lung and the kidney by using the structural element B to obtain the expanded single region prediction image and the corroded single region prediction image, and subtracting the expanded single region prediction image and the corroded single region prediction image to obtain the corresponding region rough edge prediction image;
let pe i (x, y) represents the pixel value of any coordinate of the predicted image at the rough edge of the region, including
Figure FDA0004065900810000051
In the formula, symbol
Figure FDA0004065900810000052
For the expansion operation, the sign->
Figure FDA0004065900810000053
For corrosion operation, i belongs to {1,2,. And 5}, x belongs to {1,2,. And W }, y belongs to {1,2,. And H }, wherein W and H are the width and height of the single-region rough edge prediction image respectively, and pe i (x，y)∈{0，1}，pe 1 、pe 2 、pe 3 、pe 4 、pe 5 Said single-region rough-edge prediction image, p, of the heart, liver, spleen, lung, kidney, respectively i (x, y) represents a pixel value of an arbitrary coordinate (x, y) of the image for the corresponding single region in the heart, liver, spleen, lung, kidney.
9. A traditional Chinese medicine ear five-organ region segmentation device based on machine learning is characterized by comprising at least one processor and a memory connected with the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.
CN202110396463.4A 2021-04-13 2021-04-13 Traditional Chinese medicine ear five-organ region segmentation method and device based on machine learning Active CN113112475B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110396463.4A CN113112475B (en) 2021-04-13 2021-04-13 Traditional Chinese medicine ear five-organ region segmentation method and device based on machine learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110396463.4A CN113112475B (en) 2021-04-13 2021-04-13 Traditional Chinese medicine ear five-organ region segmentation method and device based on machine learning

Publications (2)

Publication Number Publication Date
CN113112475A CN113112475A (en) 2021-07-13
CN113112475B true CN113112475B (en) 2023-04-18

Family

ID=76716660

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110396463.4A Active CN113112475B (en) 2021-04-13 2021-04-13 Traditional Chinese medicine ear five-organ region segmentation method and device based on machine learning

Country Status (1)

Country Link
CN (1) CN113112475B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114155240A (en) * 2021-12-13 2022-03-08 韩松洁 Ear acupoint detection method and device and electronic equipment

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020119679A1 (en) * 2018-12-14 2020-06-18 深圳先进技术研究院 Three-dimensional left atrium segmentation method and apparatus, terminal device, and storage medium
AU2020103905A4 (en) * 2020-12-04 2021-02-11 Chongqing Normal University Unsupervised cross-domain self-adaptive medical image segmentation method based on deep adversarial learning

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10699412B2 (en) * 2017-03-23 2020-06-30 Petuum Inc. Structure correcting adversarial network for chest X-rays organ segmentation
CN108596193B (en) * 2018-04-27 2021-11-02 东南大学 Method and system for building deep learning network structure aiming at human ear recognition
CN111402268B (en) * 2020-03-16 2023-05-23 苏州科技大学 Liver in medical image and focus segmentation method thereof
CN111583287A (en) * 2020-04-23 2020-08-25 浙江大学 Deep learning model training method for fine portrait picture segmentation
CN112070752B (en) * 2020-09-10 2024-08-13 杭州晟视科技有限公司 Auricle segmentation method and device for medical image and storage medium

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020119679A1 (en) * 2018-12-14 2020-06-18 深圳先进技术研究院 Three-dimensional left atrium segmentation method and apparatus, terminal device, and storage medium
AU2020103905A4 (en) * 2020-12-04 2021-02-11 Chongqing Normal University Unsupervised cross-domain self-adaptive medical image segmentation method based on deep adversarial learning

Also Published As

Publication number Publication date
CN113112475A (en) 2021-07-13

Similar Documents

Publication Publication Date Title
CN109255776B (en) Automatic identification method for cotter pin defect of power transmission line
Shen et al. An automated lung segmentation approach using bidirectional chain codes to improve nodule detection accuracy
CN110059697B (en) Automatic lung nodule segmentation method based on deep learning
US7840062B2 (en) False positive reduction in computer-assisted detection (CAD) with new 3D features
JP4739355B2 (en) Fast object detection method using statistical template matching
US8009900B2 (en) System and method for detecting an object in a high dimensional space
US20090175514A1 (en) Stratification method for overcoming unbalanced case numbers in computer-aided lung nodule false positive reduction
US20230005140A1 (en) Automated detection of tumors based on image processing
US20040109595A1 (en) Method for automated analysis of digital chest radiographs
US20080101676A1 (en) System and Method For Segmenting Chambers Of A Heart In A Three Dimensional Image
US8285013B2 (en) Method and apparatus for detecting abnormal patterns within diagnosis target image utilizing the past positions of abnormal patterns
CN111340130A (en) Urinary calculus detection and classification method based on deep learning and imaging omics
US20090252429A1 (en) System and method for displaying results of an image processing system that has multiple results to allow selection for subsequent image processing
CN111144486B (en) Heart nuclear magnetic resonance image key point detection method based on convolutional neural network
US10706534B2 (en) Method and apparatus for classifying a data point in imaging data
CN110705565A (en) Lymph node tumor region identification method and device
JP2009541838A (en) Method, system and computer program for determining a threshold in an image including image values
CN101551854B (en) A processing system of unbalanced medical image and processing method thereof
CN110246567B (en) Medical image preprocessing method
CN110490159B (en) Method, device, equipment and storage medium for identifying cells in microscopic image
JP2009163682A (en) Image discrimination device and program
CN113160185A (en) Method for guiding cervical cell segmentation by using generated boundary position
Li et al. A visual saliency-based method for automatic lung regions extraction in chest radiographs
CN116433704A (en) Cell nucleus segmentation method based on central point and related equipment
Chen et al. Image segmentation based on mathematical morphological operator

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant