CN113781455B

CN113781455B - Cervical cell image anomaly detection method, device, equipment and medium

Info

Publication number: CN113781455B
Application number: CN202111083002.8A
Authority: CN
Inventors: 韩英男; 初晓
Original assignee: Ping An Technology Shenzhen Co Ltd
Current assignee: Ping An Technology Shenzhen Co Ltd
Priority date: 2021-09-15
Filing date: 2021-09-15
Publication date: 2023-12-26
Anticipated expiration: 2041-09-15
Also published as: CN113781455A

Abstract

The invention relates to the field of medical science and technology, and provides a cervical cell image anomaly detection method, device, equipment and medium, wherein the method is implemented by collecting cervical cell images to be detected in a microscope observation area; adjusting the focal length of a microscope to obtain a region of interest of the cervical cell image; determining the objective lens multiple corresponding to the current region of interest of the microscope, and adjusting the current objective lens multiple until a cervical cell image meeting the preset resolution is obtained; preprocessing the cervical cell image to obtain an image-enhanced cervical cell image; the pretreated cervical cell image is input into a pre-trained target network for detection, a detection result of whether the cervical cell image to be detected is abnormal or not is obtained, the cervical cell is observed by using a microscope, the cervical cell image is processed in a collecting mode and a pretreatment mode, the resolution, the precision and the quality of the cervical cell image are improved, and the cervical cell image abnormality detection efficiency is also improved.

Description

Cervical cell image anomaly detection method, device, equipment and medium

Technical Field

The invention relates to the field of medical science and technology, and provides a cervical cell image anomaly detection method, device, equipment and medium.

Background

Cervical cancer has developed highly in recent years, and has become a social problem threatening the lives of women. The currently available diagnostic method for cervical cancer is cervical smear pathology, which requires a doctor to observe an electronic image converted by cervical smear scanned by a pathology scanner before diagnosis can be made. On the one hand, the execution and operation of the flow are complex, a great deal of manpower and material resources are required to be consumed, and on the other hand, the accuracy of diagnosis is easily influenced by subjective factors of doctors or visual fatigue. Therefore, techniques for automated identification of cervical cell pathology for diagnosis are becoming increasingly important.

The existing diagnosis technology for automatically identifying cervical cell pathology is mainly based on fine segmentation and feature extraction, however, because manual cooperation is still needed when cervical cell images are acquired, problems of acquisition efficiency, cell image resolution and cell image quality can be caused, great difficulty is brought to fine segmentation of cells, meanwhile, the problem that effective features cannot be extracted or invalid features are excessively introduced in the process of feature extraction can exist, and a good effect cannot be obtained.

Disclosure of Invention

The invention provides a cervical cell image anomaly detection method, a device, equipment and a medium, which mainly aim at acquiring a pathological slide containing cervical cells to be detected, automatically adjusting the focal length of a microscope to further obtain an interested region of the current pathological slide, determining cervical cells to be detected in the interested region to generate a cervical cell image meeting the preset resolution by selecting proper objective lens multiples, preprocessing the cervical cell image to obtain a cervical cell image with enhanced image brightness, namely observing the cervical cells by using the microscope, and processing the cervical cell image in a collection mode and a preprocessing mode, thereby improving the resolution, the precision, the quality and the acquisition efficiency of the cervical cell image.

In order to achieve the above object, the present invention provides a cervical cell image abnormality detection method comprising:

collecting cervical cell images to be detected in a microscope observation area;

adjusting the focal length of the microscope to obtain a region of interest of the current cervical cell image;

determining the objective lens multiple corresponding to the current region of interest of the microscope, and adjusting the current objective lens multiple until a cervical cell image meeting the preset resolution is obtained;

Preprocessing the cervical cell image to obtain an image-enhanced cervical cell image;

inputting the pretreated cervical cell image into a pre-trained target network for detection to obtain a detection result of whether the cervical cell image to be detected is abnormal or not.

Optionally, the step of determining the objective lens multiple corresponding to the current region of interest of the microscope, and adjusting the current objective lens multiple until obtaining the cervical cell image meeting the preset resolution includes:

extracting input characteristics of cervical cell images in a current region of interest;

processing the input characteristics by using an addition neural network to obtain output characteristics;

calculating the similarity between the input features and the output features by using the distance measurement, and generating a measurement result;

normalizing the measurement result to obtain a nonlinear relation between an input characteristic and an output characteristic in the measurement result;

and determining the objective lens multiple of the microscope according to the nonlinear relation between the features, and adjusting the current objective lens multiple until the cervical cell image meeting the preset resolution is obtained.

Optionally, the step of preprocessing the cervical cell image to obtain an image-enhanced cervical cell image includes:

Converting the cervical cell image meeting the preset resolution into a product of an illumination component and a reflection component;

separating the illumination component from the reflection component using a logarithmic transformation;

performing Fourier transform on the separated illumination component and reflection component respectively to obtain a frequency domain diagram;

processing the irradiation component and the reflection component in the frequency domain diagram by homomorphic filtering to obtain a low-frequency compressed irradiation component and a high-frequency enhanced reflection component;

and fusing the low-frequency compressed irradiation component and the high-frequency enhanced reflection component to obtain an image-enhanced cervical cell image.

Optionally, before the collecting the cervical cell image to be measured in the observation area of the microscope, the method further includes:

performing image amplification processing on the image of the cervical tissue to be detected;

judging the image of the cervical tissue to be detected after the image amplification treatment;

if the isolated coloring abnormal region exists, acquiring an accurate part in the cervical tissue to be detected according to the obtained coloring abnormal region.

The step of placing the pathological slide in an observation area of a microscope, automatically adjusting the focal length of the microscope to obtain the current region of interest of the pathological slide comprises the following steps:

Acquiring initial target position information in a pathological slide bearing the cervical cells;

controlling at least one of a stage and an objective lens of a microscope to move to an initial target position in a direction perpendicular to a reference mark according to initial target position information, wherein the reference mark and a target to be detected have a preset relative position;

controlling a camera device of a microscope to take a picture of a reference mark positioned in a detection area in the moving process;

calculating the definition of the shot picture, wherein the definition is a parameter for representing the definition of the outline of the cervical cells on the picture under the condition that the resolution of the image pickup device is unchanged;

and obtaining the current region of interest according to whether the definition of the shot photo accords with a preset definition range or not, and if so, obtaining the current region of interest.

Optionally, before the step of inputting the preprocessed cervical cell image into a pre-trained target network for detection, the method further includes:

labeling the preprocessed cervical cell image, and constructing a training set;

extracting cervical cell images in the training set by utilizing a feature extraction network to obtain feature images with different scales;

screening the feature graphs with different scales by using an attention mechanism to obtain a weight coefficient representing the importance degree of the feature;

Fusing the feature images with different scales according to the weight coefficients to obtain a fused feature image;

and training the neural network by using the fusion feature map to obtain a trained target network.

Optionally, the method further comprises:

optimizing a training breadth convolutional neural network by adopting a multi-scale idea based on a training set, and extracting multi-scale features of the cervical cell image by utilizing the breadth convolutional neural network;

optimizing and training a dense convolutional neural network based on a training set by adopting a cross-layer dense connection idea, and extracting features of different abstract depths of the cervical cell image by using the dense convolutional neural network;

and fusing the breadth features extracted based on the breadth convolutional neural network with the different abstract depth features extracted by the dense convolutional neural network according to the weight coefficients, and training the fully-connected neural network by the fused feature map to obtain the target network model.

In addition, to achieve the above object, the present invention provides a cervical cell image abnormality detection device comprising:

the acquisition module is used for acquiring cervical cell images to be detected in the observation area of the microscope;

the interested region acquisition module is used for adjusting the focal length of the microscope to acquire the interested region of the cervical cell image;

The objective lens adjusting module is used for determining the objective lens multiple corresponding to the current region of interest of the microscope, and adjusting the current objective lens multiple until a cervical cell image meeting the preset resolution is obtained;

the preprocessing module is used for preprocessing the cervical cell image to obtain an image-enhanced cervical cell image;

the abnormality detection module is used for inputting the preprocessed cervical cell image into a pre-trained target network for detection, and obtaining a detection result of whether the cervical cell image to be detected is abnormal or not.

Furthermore, to achieve the above object, the present invention provides a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the method according to any one of the embodiments above when the computer program is executed.

Furthermore, to achieve the above object, the present invention provides a computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method according to any of the embodiments above.

According to the cervical cell image anomaly detection method, device, equipment and medium, a pathological slide containing cervical cells to be detected is firstly obtained, the focal length of a microscope is automatically adjusted to further obtain a region of interest of the current pathological slide, cervical cells to be detected in the region of interest are determined to generate a cervical cell image meeting the preset resolution through selecting a proper objective lens multiple, and then the cervical cell image is preprocessed to obtain the cervical cell image with enhanced image brightness, namely, the cervical cells are observed by the microscope, and the cervical cell image is processed in a collecting mode and a preprocessing mode, so that the resolution, precision and quality of the obtained cervical cell image are improved, and the cervical cell image anomaly detection efficiency and detection precision are also improved.

Drawings

Fig. 1 is a schematic flow chart of a cervical cell image anomaly detection method according to an embodiment of the present invention;

fig. 2 is a schematic flow chart of another method for detecting cervical cell image abnormalities according to an embodiment of the present invention;

fig. 3 is a schematic flow chart of another method for detecting cervical cell image abnormalities according to an embodiment of the present invention;

fig. 4 is a schematic flow chart of another method for detecting cervical cell image abnormalities according to an embodiment of the present invention;

fig. 5 is a schematic structural view of a cervical cell image abnormality detection device according to an embodiment of the present invention;

fig. 6 is a schematic structural diagram of a computer device according to an embodiment of the invention.

The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.

Detailed Description

It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.

To facilitate an understanding of the present application, concepts related to the present application will be explained first.

Cervical cancer is one of the malignant tumors that pose serious harm to the life and health of women, and the incidence rate is second among the malignant tumors of women. Common cervical cytopathy includes: atypical squamous cell-ambiguities (atypicalsquamous cells of undetermined significance, ASC-US), low-grade squamous epithelial lesions (low-grade squamous intraepithelial lesion, LSIL), atypical squamous cell-non-exclusive of high-grade squamous epithelial lesions (atypical squamous cells cannot exclude high-gradesquamousintraepithelial lesion, ASC-H), high-grade squamous epithelial lesions (high-grade squamousintraepithelial lesion, HSIL), atypical glandular epithelial cells (atypical glandular cells, AGC), and the like.

The cervical fluid-based cell inspection method is the most commonly used cervical cancer screening method at present, and most of the existing intelligent auxiliary cervical cancer screening systems have lower detection precision on abnormal cells such as ASC-US and the like. The deformation degree (nuclear mass ratio increasing multiple, cell nucleus abnormal degree and the like) of part of abnormal cells in cervical abnormal cells is low, and the abnormal cells exist in the form of single small cells instead of agglomerated cells, so that the abnormal cells of the cervix are difficult to detect in the current intelligent auxiliary screening process of cervical cancer, and the detection precision of the abnormal cells of the cervix is low.

In one embodiment, a cervical cell image anomaly detection method is provided, as shown with reference to fig. 1, comprising the steps of:

s1, collecting cervical cell images to be detected in a microscope observation area.

For step S1, it is mainly reflected that the target to be measured provides corresponding cervical cells, that is, the cervical cells to be measured are carried on a pathological slide, and the pathological slide includes, but is not limited to, pathological sections or pathological smears, for example, the cervical cells to be measured are uniformly smeared on the slide to form a pathological smear containing the cervical cells to be measured, and for example, the cervical cells to be measured are sliced on a fragment to form a pathological section containing the cervical cells to be measured.

In some embodiments, when extracting cervical cells to be measured of the target to be measured, sometimes, due to the influence of gynecological diseases of the target to be measured, for example, rot, leucorrhea or peculiar smell of cervical tissues caused by cervicitis, vaginitis and other diseases exist, on one hand, the operation of doctors is not facilitated, and the observation of the doctors is influenced; on the other hand, the method is not beneficial to doctors to judge suspicious areas, and further accurately extracts cervical cells to be detected. Thus, prior to obtaining a pathology slide comprising cervical cells to be tested, the method further comprises:

collecting cervical tissues to be detected after washing by normal saline;

acquiring images of cervical tissues to be measured, which are sequentially smeared by 3% -5% acetic acid solution and compound iodine solution;

wherein, the cervical tissue to be measured after the acetic acid solution and the compound iodine solution are smeared is collected through a colposcope.

Wherein, the image of the cervical tissue to be measured is displayed in an enlarged manner on a display (computer) by collecting the image of the cervical tissue to be measured coated with acetic acid solution and compound iodine solution. For example, if there is a region of the cervix that needs biopsy, the region will exhibit characteristics such as "thick white vinegar" and "mosaic blood vessel" under the action of 3% -5% acetic acid solution; under the action of the compound iodine solution, the characteristics of bright orange color, mustard yellow color, spot-like coloring and the like are presented, the characteristics can be used as a doctor reference, and the accurate part in cervical tissues to be detected can be acquired by judging whether an isolated coloring abnormal area exists in a Gong Genglin column boundary and a columnar epithelial area in the image or not and referring to the characteristics if the isolated coloring abnormal area exists. However, the presence of these features does not determine that there is a certain lesion in the cervix, and therefore requires further judgment by the physician.

By the method, the image of the cervical tissue to be detected after the image amplification treatment is judged, so that the cervical tissue accurate part to be detected is screened, on one hand, the interference of external factors of the cervical tissue to be detected can be filtered, and an abnormal region which is possibly diseased can be accurately found, so that the accuracy of cell abnormality detection is improved; on the other hand, sample biopsy is performed by extracting cervical cells to be detected, so that sample data are enriched, labeling of the sample data is facilitated, and a training set is further quickly constructed.

S2: and adjusting the focal length of the microscope to acquire a region of interest of the current cervical cell image.

For step S2, it is mainly embodied that the microscope is directed to a region of interest (i.e. a field of view) of the pathology slide, wherein the microscope includes but is not limited to an optical microscope, an electron microscope, and the working principle of the microscope is not traced again here.

In some embodiments, the focal length of the current microscope can be adjusted by using an automatic focusing technology, so that a region of interest in the current pathological slide is obtained, for example, a video signal is utilized to judge out-of-focus of the current pathological slide according to the change of image gray scale gradient in the video signal, the out-of-focus signal is fed back to a stepping motor driving circuit, and a stepping motor moves a microscope body together with a CCD (image pickup device), so that automatic focusing is realized. The automatic focusing technology has the advantages of quick response and accuracy, and can dynamically and real-time improve the definition of the microscope image.

The region of interest outlines the region to be processed on the pathological slide in the modes of square frames, circles, ellipses, irregular polygons and the like.

In some embodiments, compared with a hospital, in the prior art, a pathological slide containing cervical cells to be detected is digitally processed by a pathological scanner, and a scanned digital image is input into a screening system to obtain a detection result of the cervical cells. In the embodiment, the image of the cervical cells to be detected is directly acquired through the microscope, so that the acquisition efficiency is greatly improved.

In some embodiments, automatically adjusting the focal length of the microscope to acquire a region of interest of the pathology slide currently includes:

s201: acquiring initial target position information in a pathological slide bearing the cervical cells;

s202: controlling at least one of a stage and an objective lens of a microscope to move to an initial target position in a direction perpendicular to a reference mark according to initial target position information, wherein the reference mark and a target to be detected have a preset relative position;

S203: controlling a camera device of a microscope to take a picture of a reference mark positioned in a detection area in the moving process;

s204: calculating the definition of the shot picture, wherein the definition is a parameter for representing the definition of the outline of the cervical cells on the picture under the condition that the resolution of the image pickup device is unchanged;

s205: and obtaining the current region of interest according to whether the definition of the shot photo accords with a preset definition range or not, and if so, obtaining the current region of interest.

Specifically, judging whether the definition of the shot photo accords with a preset definition range, if so, calculating an actual focal length for determining the measured object according to the initial target position information and the distance difference between the reference mark and the measured object, and obtaining an interested region under the current position information according to the actual focal length; otherwise, comparing the definition of the shot photo with a definition calibration curve to obtain a compensation value, and calculating an actual focal length for determining the measured object according to the distance difference between the compensation value and the measured object, wherein the definition calibration curve is generated by the definition of the reference mark photo and the corresponding position information obtained in advance.

In some embodiments, one way to obtain initial target location information is: directly setting the initial target information to a preset fixed value; another way is: and searching the position with the maximum definition value from the definition calibration curve, wherein the position information (such as Z-direction coordinates) corresponding to the maximum definition value is the initial target position information.

In some embodiments, after the initial target position information is acquired, the Z-direction coordinate information is converted into the running steps of the stepper motor in the driving device, and the driving device is controlled to drive the objective lens to move to the initial target position in the direction perpendicular to the reference mark (for example, the Z-direction).

In some embodiments, when at least one of the stage and the objective lens moves to the initial target position, the control device controls the image capturing device to capture a picture of the reference mark, so as to avoid the influence of factors such as image noise or mark drift, for example, the image capturing device can move a fixed distance to capture a picture, or capture a picture at preset time intervals.

And comparing the definition of the shot photo with a definition calibration curve to obtain a compensation value.

In some embodiments, the best definition on the definition calibration curve obtained in advance is searched, the difference between the definition of the corresponding position of the shot photo and the corresponding position on the definition calibration curve is calculated, if the difference is smaller than a set threshold, the compensation value is zero, and if the difference is larger than or equal to the set threshold, the compensation value is the difference between the initial target position information and the position information corresponding to the best definition.

By the embodiment, the visual field under the current microscope is analyzed, so that the focal length of the current microscope can be automatically adjusted, the visual field under the microscope is enlarged, and cervical cells under a pathological slide can be identified; in addition, by a focusing closed-loop control mode, the repeated focusing times are reduced, the detection speed is not influenced, and the definition and the stability of the image are ensured; meanwhile, the method is not influenced by the pollution degree, the running time, the abrasion condition and other factors of the detection pool, and real automatic focusing is realized.

S3: and determining the objective lens multiple corresponding to the current region of interest of the microscope, and adjusting the current objective lens multiple until the cervical cell image meeting the preset resolution is obtained.

Aiming at the step S3, the main embodiment is that the current interested area can be obtained after the focal length of the microscope is adjusted in the step S2, so as to form a rough resolution view; the resolution precision is also greatly affected under the condition of the corresponding multiple of different objective lenses of the microscope, namely, by adjusting the multiple of the objective lens in the region of interest, the cervical cell image meeting the requirements of high resolution or high precision can be further obtained.

The preset resolution may be a threshold set by those skilled in the art as needed.

Since the magnification of the microscope is equal to the product of the magnification of the objective lens and the magnification of the eyepiece lens, the larger the magnification speed of the objective lens is, the longer the lens is, the closer to the stage is, and the larger the magnification of the eyepiece lens is, the shorter the lens is. Therefore, the magnification of the objective lens needs to be reasonably adjusted to ensure that cervical cell images meeting the preset resolution are acquired under the microscope.

It should be noted that, by microscope, the magnification is any one of the length, width, or diameter of the cervical cells, not the magnification of the area, volume, or surface area, wherein if the cervical cells in the magnified field are aligned, the magnification is inversely proportional to the magnification, and if the cervical cells in the magnified field are randomly arranged, the magnification is inversely proportional to the square of the magnification.

In some embodiments, determining the objective lens magnification corresponding to the current region of interest of the microscope to obtain the cervical cell image satisfying the preset resolution further includes:

s301: extracting input characteristics of cervical cell images in a current region of interest;

s302: processing the input characteristics by using an addition neural network to obtain output characteristics;

s303: calculating the similarity between the input features and the output features by using the distance measurement, and generating a measurement result;

S304: normalizing the measurement result to obtain a nonlinear relation between an input characteristic and an output characteristic in the measurement result;

s305: and determining the objective lens multiple of the microscope according to the nonlinear relation between the features, and adjusting the current objective lens multiple until the cervical cell image meeting the preset resolution is obtained.

The conventional convolutional neural network is replaced by the additive neural network, and the operation speed of the target network can be remarkably increased by replacing a large number of multiplication operations in the convolutional neural network with addition operations with higher speed, so that the data processing speed of the target network is improved; meanwhile, the network computing power consumption is reduced. Since the addition operation is more computationally efficient than the multiplication operation in terms of hardware computation, and consumes less power.

The distance measurement mode includes, but is not limited to, euclidean distance, cosine similarity, hamming distance, manhattan distance, chebyshev distance, min distance, jacar index and semi-normal distance; in addition, the normalization processing mode is Batch Normalization, namely BN preprocessing, the BN preprocessing not only can effectively solve the problem that the data distribution between layers is changed before the layer characteristic extraction core is pooled by the addition neural network in the training process, but also can randomize sample data, and effectively avoids the probability that a certain sample is always selected in each batch of training.

In some embodiments, the summing neural network may include one or more summing filter layers, and may further include other layers such as an input layer, a pooling layer, an implicit layer, or an output layer, which is not limited in this embodiment. The additive neural network may include a plurality of additive filter layers, each of which may include one or more feature extraction kernels. I.e. a plurality of feature extraction kernels may be included in the additive neural network. Accordingly, the cervical cell image may be subjected to a plurality of feature extraction processes through the plurality of feature extraction checks to obtain an input feature, the output feature including a plurality of input sub-features.

In the embodiment, by automatically identifying the magnification of the current objective lens, analyzing the cervical cell image of the current region of interest of the microscope by using an additive neural network to obtain whether the image resolution corresponding to the current magnification scene meets the preset resolution or not, and if so, keeping the magnification of the current objective lens; if the current image resolution does not meet the preset resolution, for example, when the current image resolution is lower than the preset resolution, the current image resolution is indicated to not meet the preset resolution, and at the moment, the amplification factor of the current objective lens is increased, so that the amplification factor of the objective lens can be automatically adjusted, the detection precision and efficiency are improved, and the additional influence of artificial factors is avoided.

S4: and preprocessing the cervical cell image to obtain an image-enhanced cervical cell image.

Aiming at the step S4, the pretreatment is mainly embodied in order to pretreat the cervical cell image, and the pretreatment in the embodiment is mainly to adjust the influence of illumination caused by microscope imaging, overcome the problem of poor image quality caused by uneven illumination, and further effectively enhance the contrast of the image.

In some embodiments, the step of preprocessing the cervical cell image to obtain an image-enhanced cervical cell image comprises:

s401: converting the cervical cell image meeting the preset resolution into a product of an illumination component and a reflection component;

s402: separating the illumination component from the reflection component using a logarithmic transformation;

s403: performing Fourier transform on the separated illumination component and reflection component respectively to obtain a frequency domain diagram;

s404: processing the irradiation component and the reflection component in the frequency domain diagram by homomorphic filtering to obtain a low-frequency compressed irradiation component and a high-frequency enhanced reflection component;

s405: and fusing the low-frequency compressed irradiation component and the high-frequency enhanced reflection component to obtain an image-enhanced cervical cell image.

The logarithmic transformation can be used for low gray values with a narrow stretching range and high gray values with a wide compression range; it can also be used to expand dark pixel values in an image while compressing bright pixel values.

For example, for an image-enhanced cervical cell image F (x, y), the image F (x, y) is represented by the product of the illumination component i (x, y) and the reflection component r (x, y) by conversion; the following logarithmic expression will be obtained by logarithmic transformation; inf (x, y) =ini (x, y) +inr (x, y); and performing Fourier transform to obtain I { Inf (x, y) } = I { Ini (x, y) } +I { Inr (x, y) }, wherein the low-frequency components in the image are associated with illumination and the high-frequency components are associated with reflection after the Fourier transform. At this time, the homomorphic filtering mode can better control the irradiation component and the reflection component. The homomorphic filtering corresponds to a filter that affects the low and high frequencies of the fourier transform in different controllable ways. If γl <1 and γh >1 are chosen, the filter function approaches attenuation low frequency (illumination) while high frequency reflection is enhanced, the end result being simultaneous dynamic range compression and contrast enhancement, where the image filter function involved is:

where D (u, v) and D0 are the distance of the frequency center of the filter function and the cut-off frequency, respectively, and the c constant is used to control the sharpness of the gradient of the function.

Optionally, the effect of non-uniform illumination can be eliminated by homomorphic filtering without losing image details, wherein the gray scale of the image is synthesized by an illumination component and a reflection component, the reflection component reflects the image content, and the reflection component changes rapidly in space with the difference of the image details; the illumination components are generally all of a slowly varying nature in space; the spectrum of the illumination component falls in the low spatial frequency region and the spectrum of the reflection component falls in the high spatial frequency region.

By the embodiment, the homomorphic filtering can be utilized to perform preprocessing correction on the cervical cell image before the cervical cell image is input into the target network, so that the contrast of the image is improved, and the quality of the cervical cell image is improved.

S5: inputting the pretreated cervical cell image into a pre-trained target network for detection to obtain a detection result of whether the cervical cell image to be detected is abnormal or not.

Aiming at the step S5, the main embodiment is that the preprocessed cervical cell image to be detected is input into a target network, and is identified through the target network, so that the detection result of whether the cervical cell image to be detected is abnormal or not can be rapidly judged.

The target network may be a fast-RCNN deep convolutional neural network model, and when the target network is applied, the output of the target network is the prediction probability of abnormal cells with the target area as a background. The step of inputting the cervical cell image to be identified into a trained Faster-RCNN deep convolutional neural network model, and obtaining abnormal cells with different numbers and corresponding prediction probabilities through the feature extraction, the area selection network selection and the final classification.

By the cervical cell image recognition method based on the convolutional neural network, provided by the embodiment of the invention, detection results for judging whether a target is abnormal can be obtained by inputting any cervical liquid-based smear digital image into the obtained Faster-RCNN model. It should be noted that, the model training method in the embodiment of the present invention is a result of creative effort for those skilled in the art, and all the changes, adjustments or alternatives of the data enhancement method, the neural network architecture, the super parameters and the loss function in the present invention based on the embodiment of the present invention should be considered as equivalent to the present scheme.

When a detection result of an abnormal cell as a target is obtained, a detection result greater than the set confidence threshold value may be displayed by setting the confidence threshold value.

In a first embodiment: for each target cervical cell pathology slide image block, obtaining cervical cell characteristics in the target cervical cell pathology slide image blocks, so as to determine the abnormal probability of cervical cells in each image block. For example, the cervical cell characteristics are compared with the cellular characteristics of normal cervical cells to determine the probability of abnormalities in the cervical cells.

And when the abnormal probability of the cervical cell is not less than a preset abnormal probability threshold, determining that the cervical cell is an abnormal detection result.

Specifically, when the abnormality probability is not less than the preset abnormality probability threshold, the cervical cell may be considered abnormal, so that abnormal cervical cells present in each image block may be detected. By combining the detection conditions in each target cervical cell pathology slide image block, cervical abnormal cells in the cervical cell pathology slide image can be detected, and automatic detection of the cervical abnormal cells is realized.

Optionally, after determining that the cervical cell is an abnormal detection result, acquiring position information of the cervical cell in a corresponding target cervical cell pathology slide image block, so as to obtain a position of the cervical abnormal cell in the cervical cell pathology slide image.

Specifically, for each target cervical cell pathology slide image block, the abnormal probability of the cervical cells is obtained. When the abnormality probability is not smaller than a preset abnormality probability threshold, the cervical cell can be considered as an abnormality detection result, the position information of the cervical cell in the corresponding target cervical cell pathology slide image block is obtained, the position information and the abnormality probability of the cervical cell are reserved, and thus the position information and the abnormality probability of the cervical abnormal cell in each image block can be detected. The position of the cervical abnormal cells in the cervical cell pathology slide image can be determined by combining the detection condition in each target cervical cell pathology slide image block. At the time of display, the display may be performed according to the position of the cervical abnormal cells in the cervical cell pathology slide image, and the corresponding abnormal probability may also be displayed.

By the method, abnormal cells in the cervical cell image to be detected are classified, diagnosis suggestions can be given to the slice-level results, the auxiliary effect is achieved for a clinician, and the workload of the doctor is reduced. Meanwhile, the diagnosis proposal provided by the invention has higher sensitivity and specificity, and the invention can be applied to the automatic detection of other pathological digital images in the medical field, such as the detection of cervical cell images, and the like, and the invention is not limited.

In some embodiments, before inputting the preprocessed cervical cell image into the pre-trained target network for detection, the method further comprises: s6: constructing a target network for identifying cervical cell images, which specifically comprises the following steps:

s601: labeling the preprocessed cervical cell image, and constructing a training set;

s602: extracting cervical cell images in the training set by utilizing a feature extraction network to obtain feature images with different scales;

s603: screening the feature graphs with different scales by using an attention mechanism to obtain a weight coefficient representing the importance degree of the feature;

s604: fusing the feature images with different scales according to the weight coefficients to obtain a fused feature image;

S605: and training the neural network by using the fusion feature map to obtain a trained target network.

In some embodiments, labeling the preprocessed cervical cell image, and constructing the training set includes:

the cervical cell images after image enhancement can be manually marked by medical staff, and the marked cervical cell images are collected and classified to form a training set.

For example, the classes of abnormal cells or biological pathogens that need to be labeled may also be as follows:

squamous cells include: atypical squamous cell (low-grade squamous lesions, excluding high-grade squamous lesions) and squamous cell carcinoma (high-grade squamous lesions);

the glandular cells include: atypical gland cells (cervical canal cells, endometrial cells), cervical canal gland cells (prone to neoplasms), cervical canal in situ adenocarcinoma, adenocarcinoma (cervical canal adenocarcinoma, endometrial adenocarcinoma, extrauterine adenocarcinoma);

biological pathogens include: trichomonas vaginalis, fungi conforming in morphology to candida albicans (dysbacteriosis, suggesting bacterial vaginosis), bacteria conforming in morphology to actinomycetes (cytologic changes conform to herpes simplex virus infection);

and, endometrial cells.

In some embodiments, extracting cervical cell images within the training set using a feature extraction network to obtain feature maps of different scales comprises:

specifically, a multi-scale idea is adopted to optimize a training breadth convolutional neural network based on a training set, the multi-scale characteristics of the cervical cell image are extracted by utilizing the breadth convolutional neural network, and a combination center loss function and a Softmax loss function are adopted as a loss function of the training breadth convolutional neural network;

optionally, the training set is based on a cross-layer dense connection idea to optimally train the dense convolutional neural network, the dense convolutional neural network is utilized to extract features of different abstract depths of the cervical cell image, and a loss function of the training dense convolutional neural network adopts a mode of combining a central loss function and a Softmax loss function.

Optionally, the breadth features extracted based on the breadth convolutional neural network and the different abstract depth features extracted by the dense convolutional neural network are fused according to the weight coefficients, and the fused feature map trains the full-connection neural network to obtain the target network model.

Specifically, the breadth convolutional neural network is provided with a plurality of scale convolutional kernels, and classification is carried out after the length features of each scale are fused, so that the problem of inconsistent sizes of important features is solved, the extracted breadth features are more effective, and the classification accuracy is improved; meanwhile, the dense convolutional neural network is adopted to extract information of different depths, all convolutional layer outputs are fused and spliced to serve as input, the integrity of feature extraction is improved, the fully-connected neural network is built to fuse features of different depths and features of different widths, and the accuracy of cervical cell image feature classification is further improved.

In some embodiments, the screening of feature graphs of different scales by using an attention mechanism to obtain a weight coefficient representing the importance degree of the feature includes:

specifically, after filtering the different scale features using the attention mechanism, the important features are highlighted and the secondary features are suppressed, e.g., quantized with different weighting coefficients according to the degree of the highlighted important features.

In some embodiments, classifying the fused feature map to obtain a trained target network includes:

optimizing a training breadth convolutional neural network by adopting a multi-scale idea, extracting multi-scale features of the image data by utilizing the breadth convolutional neural network, and adopting a combination center loss function and a Softmax loss function as corresponding loss functions; the corresponding expression is:

Loss＝Softmaxlos+λCentorloss

wherein, the Loss function corresponding to the combined is Loss, the Softmax Loss function is Softmaxloss, the center Loss function is centrloss, and lambda represents the coefficient size;

the breadth convolutional neural network comprises a convolutional kernel with multiple scales, wherein each channel corresponds to a convolutional layer with one scale, and the convolutional kernel comprises a first connecting layer, a second connecting layer, a first maximum pooling layer, a second maximum pooling layer, a first full connecting layer, a second full connecting layer and an output layer; when the number of the convolution layers is two and the number of the input channels is three, the input image data sequentially passes through the first connecting layer and the first largest pooling layer from the first convolution layer to output a first multi-scale feature map; the first multi-scale feature map outputs a second multi-scale feature map through a second connecting layer and a second maximum pooling layer of a second layer convolution layer of the same structural channel; and the second multi-scale feature map sequentially outputs multi-scale features of the first full-connection layer and the second full-connection layer, and a classification result is output by using a Softmax classifier.

Optionally, the dense convolutional neural network includes at least four convolutional layers of the same scale convolutional kernel, a first connecting layer, a second connecting layer, a third connecting layer, a first maximum pooling layer, a first full connecting layer, a second full connecting layer and an output layer, wherein the input image sequentially passes through the first convolutional layer, the second convolutional layer, the first connecting layer, the third convolutional layer, the second connecting layer, a fourth convolutional layer and the third connecting layer; each connection layer connects the outputs of all the convolution layers in front of it, so that dense connections can make the network extract more efficient different depth features; and the third connecting layer sequentially pools the layer, the first full connecting layer and the second full connecting layer to output different depth characteristics, and a classification result is output by using a Softmax classifier.

The loss functions of the dense convolutional neural network and the fully-connected neural network adopt a mode of combining the central loss function and the Softmax loss function, which is the same as the loss function adopted by the breadth convolutional neural network, and are not repeated here.

Aiming at the problem of inconsistent size of important features in data in cervical cell images, a breadth convolutional neural network is provided for extracting features, the neural network utilizes the thought of multiple scales, the structure of the convolutional neural network is improved, the features of the cervical cell images are extracted by utilizing convolution kernels of different scales, and then the features of all scales are fused for classification, so that the problem of inconsistent size of the important features can be solved to a certain extent. In this embodiment, three convolution kernels of different sizes, 1×1, 3×3, and 5×5, respectively, are preferred. The extracted features can be more effective, so that the classification accuracy is improved.

Conventional convolutional neural networks are typically classified using features extracted from the last convolutional layer, thus ignoring some upper layer information, i.e., only one depth information, which may lose some important information. The present embodiment provides a dense convolutional neural network to extract information at different depths, where the outputs of all the preceding convolutional layers are spliced together as input as each convolutional operates, so that the resulting features can contain information at each depth.

When training breadth and dense convolutional neural networks, the network is trained by combining a center loss function and a Softmax loss function, and the intra-class distance of each class of results can be reduced by introducing the center loss function, so that the classification accuracy is improved.

A fully connected neural network is built and used for combining the features extracted by the breadth convolutional neural network and the dense convolutional neural network for use, and the extracted breadth features and the features with different depths are fully utilized. In addition, the fully connected neural network can further select the characteristics, and select more effective characteristics from the existing characteristics, so that the classification accuracy is further improved.

Extracting multi-scale breadth features of the cervical cell image by using a breadth convolutional neural network, extracting depth features of the cervical cell image by using a dense convolutional neural network, respectively receiving the multi-scale breadth features and the depth features by using an attention module, respectively extracting the dependency relationships of the multi-scale breadth features and the depth features corresponding to different pixels by using matrix multiplication, and carrying out weighted fusion on the breadth features and the depth features according to the dependency relationships among the pixels to obtain fused final features; and inputting the fused characteristics into a fully-connected neural network for training to obtain an abnormality detection model for identifying the cervical cell abnormality to be detected.

Through the embodiment, the abnormal cells of the cervical cell image are identified by combining the multiscale feature fusion network of the attention mechanism, and compared with other modes, the detection accuracy is greatly improved; in addition, the proposed attention mechanism can effectively filter the features with different scales, highlight important features and inhibit secondary features, so that feature learning is more efficient; the fusion network which is designed according with the multi-scale characteristics of clinical practice and can be designed according to the real distribution of the cervical abnormal cell size can cover abnormal cells with different sizes, and the omission of small-size cells is prevented.

In addition, the breadth features and the depth features can be used for fusing features of different layers together, so that the expression capability of the features is enriched, the detail information of cells is better extracted, the detection precision of cells like ASC-US similar to normal cells of the cervix can be improved, and the detection precision of abnormal cells of the cervix can be effectively improved.

The above cervical cell image abnormality detection method is exemplarily described below by a specific embodiment, which includes:

firstly, dividing cervical tissue to be detected to obtain cervical tissue slices, and placing the cervical tissue slices on a pathological slide which is placed in an observation area of an electron microscope; secondly, automatically adjusting the focal length of the microscope by the electron microscope according to the current observation area to obtain an interested area of the current pathological slide; thirdly, automatically identifying the magnification of the current objective lens, analyzing the cervical cell image of the current region of interest of the microscope by using an additive neural network to obtain whether the image resolution corresponding to the current magnification scene meets the preset resolution or not, and if so, keeping the magnification of the current objective lens; if the current image resolution does not meet the preset resolution, automatically identifying the magnification factor; by automatically adjusting the magnification of the objective lens, the detection precision and efficiency are improved, and the additional influence of artificial factors is avoided.

Moreover, the influence of uneven illumination can be eliminated by utilizing homomorphic filtering without losing image details, wherein the gray level of an image is synthesized by an illumination component and a reflection component, the reflection component reflects the image content, and the image content is changed rapidly in space along with the difference of the image details; the illumination components are generally all of a slowly varying nature in space; the spectrum of the illumination component falls in the low spatial frequency region and the spectrum of the reflection component falls in the high spatial frequency region. By the embodiment, the homomorphic filtering can be utilized to perform preprocessing correction on the cervical cell image before the cervical cell image is input into the target network, so that the contrast of the image is improved, and the quality of the cervical cell image is improved.

And finally, determining the abnormal probability of the cervical cells in each image to be detected by using the acquired cervical cell characteristics, thereby obtaining a detection result of whether the cervical cells are abnormal. For example, the cervical cell characteristics are compared with the cellular characteristics of normal cervical cells to determine the probability of abnormalities in the cervical cells. By classifying abnormal cells in the cervical cell image to be detected, diagnosis suggestions can be given to the sheet-level results, an auxiliary effect is provided for a clinician, and the workload of the doctor is reduced. Meanwhile, the diagnosis proposal provided by the invention has higher sensitivity and specificity.

The embodiment provides a cervical cell image anomaly detection method, which comprises the steps of firstly acquiring a pathological slide containing cervical cells to be detected, automatically adjusting the focal length of a microscope to obtain a region of interest of the current pathological slide, determining cervical cells to be detected in the region of interest to generate a cervical cell image meeting a preset resolution by selecting a proper objective lens multiple, and preprocessing the cervical cell image to obtain a cervical cell image with enhanced image brightness (the contrast of the image is improved, thereby improving the visual effect of the image), namely, observing the cervical cells by using the microscope, and processing the cervical cell image in a collection mode and a preprocessing mode, thereby improving the resolution, the precision and the quality of the acquired cervical cell image and also improving the detection efficiency and the detection precision of the cervical cell image anomaly.

In one embodiment, the present invention also provides a cervical cell image anomaly detection device 500, see fig. 5, comprising:

the acquisition module 501 is used for acquiring cervical cell images to be detected in a microscope observation area;

a region of interest acquisition module 502, configured to adjust a focal length of the microscope to acquire a region of interest of the cervical cell image;

An objective lens adjusting module 503, configured to determine an objective lens multiple corresponding to a current region of interest of the microscope, and adjust the current objective lens multiple until a cervical cell image satisfying a preset resolution is obtained;

a preprocessing module 504, configured to perform preprocessing on the cervical cell image to obtain an image-enhanced cervical cell image;

the abnormality detection module 505 is configured to input the preprocessed cervical cell image into a pre-trained target network for detection, so as to obtain a detection result of whether the cervical cell image to be detected is abnormal.

In this embodiment, the objective lens adjusting module 503 further includes:

the extraction unit is used for extracting the input characteristics of the cervical cell image in the current region of interest;

the processing unit is used for processing the input characteristics by utilizing the addition neural network to obtain output characteristics;

the measurement unit is used for calculating the similarity between the input features and the output features by using the distance measurement and generating measurement results;

the normalization processing unit is used for normalizing the measurement result to obtain a nonlinear relation between input features and output features in the measurement result;

and the objective lens adjusting unit is used for determining the objective lens multiple of the microscope according to the nonlinear relation among the characteristics, and obtaining the cervical cell image meeting the preset resolution.

In this embodiment, the preprocessing module 504 further includes:

a conversion unit for converting the cervical cell image satisfying the preset resolution into a product of an illumination component and a reflection component;

a conversion separation unit that separates the illumination component and the reflection component by logarithmic conversion;

the spectrum transformation unit is used for carrying out Fourier transformation on the separated irradiation component and the reflection component respectively to obtain a frequency domain diagram;

the filtering processing unit is used for processing the irradiation component and the reflection component in the frequency domain diagram by homomorphic filtering to obtain a low-frequency compressed irradiation component and a high-frequency enhanced reflection component;

and the fusion unit is used for obtaining the cervical cell image after image enhancement by utilizing the fusion of the irradiation component compressed at low frequency and the reflection component enhanced at high frequency.

In this embodiment, the method further includes: the target network construction module is used for constructing a target network for identifying cervical cell images; the target network construction module further includes:

the training set construction unit is used for marking the preprocessed cervical cell image and constructing a training set;

the feature extraction unit is used for extracting cervical cell images in the training set by utilizing a feature extraction network to obtain feature images with different scales;

The weight calculation unit screens the feature graphs with different scales by using an attention mechanism to obtain weight coefficients representing the importance degree of the features;

the feature fusion unit is used for fusing the feature images with different scales according to the weight coefficients to obtain a fused feature image;

and the target network construction unit trains the neural network by utilizing the fusion feature map to obtain a trained target network.

In this embodiment, the target network construction unit further includes:

the first network construction subunit adopts a multi-scale idea to optimize and train the breadth convolutional neural network based on the training set, and extracts multi-scale characteristics of the cervical cell image by using the breadth convolutional neural network;

the second network construction subunit adopts a cross-layer dense connection idea to optimally train the dense convolutional neural network based on the training set, and utilizes the dense convolutional neural network to extract the characteristics of the cervical cell images at different abstract depths;

and the target network construction subunit is used for fusing the breadth features extracted on the breadth convolutional neural network with the different abstract depth features extracted on the dense convolutional neural network according to the weight coefficients, and training the full-connection neural network by the fused feature map to obtain a target network model.

In this embodiment, before the acquisition module 501, the method further includes:

In this embodiment, the region of interest acquisition module 502 further includes:

acquiring initial target position information in a pathological slide;

The embodiment provides a cervical cell image anomaly detection device, the device obtains a pathological slide containing cervical cells to be detected firstly, automatically adjusts the focal length of a microscope to further obtain a region of interest of the current pathological slide, determines cervical cells to be detected in the region of interest to generate a cervical cell image meeting a preset resolution through selecting a proper objective multiple, and pre-processes the cervical cell image to obtain a cervical cell image with enhanced image brightness, namely, the cervical cell is observed by using the microscope, and the cervical cell image is processed in a collecting mode and a pre-processing mode, so that the resolution, the precision and the quality of the obtained cervical cell image are improved, and the cervical cell image anomaly detection efficiency and the detection precision are also improved.

It should be noted that the cervical cell image anomaly detection device system substantially includes a plurality of modules for executing the cervical cell image anomaly detection method according to any one of the above embodiments, and specific functions and technical effects thereof are described with reference to the above embodiments and are not repeated herein.

In an embodiment, referring to fig. 6, the present embodiment further provides a computer device 600, comprising a memory 601, a processor 602 and a computer program stored on the memory and executable on the processor, said processor 602 implementing the steps of the method according to any of the embodiments above when said computer program is executed.

In an embodiment, there is also provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the method of any of the embodiments above.

The embodiment of the application can acquire and process the related data based on the artificial intelligence technology. Among these, artificial intelligence (Artificial Intelligence, AI) is the theory, method, technique and application system that uses a digital computer or a digital computer-controlled machine to simulate, extend and extend human intelligence, sense the environment, acquire knowledge and use knowledge to obtain optimal results.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, apparatus, article, or method. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, apparatus, article or method that comprises the element.

The foregoing embodiment numbers of the present invention are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments. From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) as described above, comprising instructions for causing a terminal device (which may be a mobile phone, a computer, a server, or a network device, etc.) to perform the method according to the embodiments of the present invention.

The foregoing description is only of the preferred embodiments of the present invention, and is not intended to limit the scope of the invention, but rather is intended to cover any equivalents of the structures or equivalent processes disclosed herein or in the alternative, which may be employed directly or indirectly in other related arts.

Claims

1. A cervical cell image anomaly detection method, the method comprising:

determining the objective lens multiple corresponding to the current region of interest of the microscope, and adjusting the current objective lens multiple until a cervical cell image meeting the preset resolution is obtained; extracting input characteristics of cervical cell images in a current region of interest; processing the input characteristics by using an addition neural network to obtain output characteristics; calculating the similarity between the input features and the output features by using the distance measurement, and generating a measurement result; normalizing the measurement result to obtain a nonlinear relation between an input characteristic and an output characteristic in the measurement result; determining the objective lens multiple of the microscope according to the nonlinear relation between the features, and adjusting the current objective lens multiple until a cervical cell image meeting the preset resolution is obtained;

2. The cervical cell image anomaly detection method according to claim 1, wherein the step of preprocessing the cervical cell image to obtain an image-enhanced cervical cell image comprises:

3. The cervical cell image abnormality detection method according to any one of claims 1 to 2, characterized by further comprising, before the acquisition of the cervical cell image to be measured in the microscopic observation area:

performing image amplification treatment on the image of the cervical tissue to be detected;

4. The cervical cell image abnormality detection method according to any one of claims 1 to 2, characterized in that the step of adjusting the focal length of the microscope to acquire a region of interest of the current cervical cell image includes:

5. The cervical cell image anomaly detection method according to claim 1, wherein before the step of inputting the preprocessed cervical cell image into a pre-trained target network for detection, further comprising:

Labeling the preprocessed cervical cell image, and constructing a training set;

6. The cervical cell image anomaly detection method of claim 5, further comprising:

7. A cervical cell image anomaly detection device, the device comprising:

the objective lens adjusting module is used for determining the objective lens multiple corresponding to the current region of interest of the microscope, and adjusting the current objective lens multiple until a cervical cell image meeting the preset resolution is obtained; extracting input characteristics of cervical cell images in a current region of interest; processing the input characteristics by using an addition neural network to obtain output characteristics; calculating the similarity between the input features and the output features by using the distance measurement, and generating a measurement result; normalizing the measurement result to obtain a nonlinear relation between an input characteristic and an output characteristic in the measurement result; determining the objective lens multiple of the microscope according to the nonlinear relation between the features, and adjusting the current objective lens multiple until a cervical cell image meeting the preset resolution is obtained;

8. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the method according to any one of claims 1 to 6 when the computer program is executed.

9. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 6.