CN113781455A

CN113781455A - Cervical cell image abnormality detection method, device, equipment and medium

Info

Publication number: CN113781455A
Application number: CN202111083002.8A
Authority: CN
Inventors: 韩英男; 初晓
Original assignee: Ping An Technology Shenzhen Co Ltd
Current assignee: Ping An Technology Shenzhen Co Ltd
Priority date: 2021-09-15
Filing date: 2021-09-15
Publication date: 2021-12-10
Anticipated expiration: 2041-09-15
Also published as: CN113781455B

Abstract

The invention relates to the field of medical science and technology, and provides a method, a device, equipment and a medium for detecting the abnormality of a cervical cell image, wherein the method comprises the steps of collecting the cervical cell image to be detected in a microscope observation area; adjusting the focal length of the microscope to acquire the region of interest of the current cervical cell image; determining the objective lens multiple corresponding to the current region of interest of the microscope, and adjusting the current objective lens multiple until a cervical cell image meeting the preset resolution is obtained; preprocessing the cervical cell image to obtain an image-enhanced cervical cell image; inputting the preprocessed cervical cell image into a pre-trained target network for detection to obtain a detection result of whether the cervical cell image to be detected is abnormal or not, observing the cervical cell by using a microscope, and processing the cervical cell image in an acquisition mode and a preprocessing mode, so that the resolution, the precision and the quality of the cervical cell image are improved, and the abnormal detection efficiency of the cervical cell image is also improved.

Description

Cervical cell image abnormality detection method, device, equipment and medium

Technical Field

The invention relates to the field of medical science and technology, and provides a method, a device, equipment and a medium for detecting cervical cell image abnormality.

Background

Cervical cancer has become a social problem threatening the life of women in recent years due to its high incidence. The currently effective cervical cancer diagnosis method is cervical smear pathological examination, and the method needs a doctor to make a diagnosis after observing an electronic image converted from cervical smear scanned by a pathological scanner. On one hand, the execution and operation of the flow referring to the above process are complex, and a large amount of manpower and material resources are consumed, and on the other hand, the accuracy of diagnosis is easily affected by subjective factors of doctors or visual fatigue. Therefore, techniques for automatically identifying cervical cell pathologies for diagnosis are becoming increasingly important.

The existing diagnosis technology for automatically identifying cervical cell pathology generally takes fine segmentation and feature extraction as main parts, however, because the cervical cell image still needs manual cooperation when being collected, the problems of collection efficiency, cell image resolution and cell image quality can be caused, great difficulty is brought to the fine segmentation of cells, meanwhile, the problem that effective features cannot be extracted or invalid features are introduced too much can exist in the feature extraction process, and a good effect cannot be obtained.

Disclosure of Invention

The invention provides a method, a device, equipment and a medium for detecting cervical cell image abnormity, which mainly aim to obtain a pathological slide containing cervical cells to be detected, automatically adjust the focal length of a microscope to further obtain an interested area of the current pathological slide, determine the cervical cells to be detected in the interested area by selecting a proper objective lens multiple to generate a cervical cell image meeting a preset resolution, and then preprocess the cervical cell image to obtain a cervical cell image with enhanced image brightness, namely, the cervical cells are observed by using the microscope, and the cervical cell image is processed by an acquisition mode and a preprocessing mode, so that the resolution, the precision, the quality and the acquisition efficiency of the cervical cell image are improved.

In order to achieve the above object, the present invention provides a method for detecting an image abnormality of cervical cells, the method comprising:

collecting a cervical cell image to be detected in a microscope observation area;

adjusting the focal length of the microscope to acquire the region of interest of the current cervical cell image;

determining the objective lens multiple corresponding to the current region of interest of the microscope, and adjusting the current objective lens multiple until a cervical cell image meeting a preset resolution is obtained;

preprocessing the cervical cell image to obtain an image-enhanced cervical cell image;

inputting the preprocessed cervical cell image into a pre-trained target network for detection to obtain a detection result of whether the cervical cell image to be detected is abnormal or not.

Optionally, the step of determining an objective lens multiple corresponding to the current region of interest of the microscope, and adjusting the current objective lens multiple until obtaining a cervical cell image satisfying a preset resolution includes:

extracting input characteristics of cervical cell images in the current region of interest;

processing the input features by using an additive neural network to obtain output features;

calculating the similarity between the input features and the output features by using the distance measurement to generate a measurement result;

normalizing the measurement result to obtain a nonlinear relation between the input characteristic and the output characteristic in the measurement result;

and determining the objective lens multiple of the microscope according to the nonlinear relation among the characteristics, and adjusting the current objective lens multiple until a cervical cell image meeting the preset resolution is obtained.

Optionally, the step of preprocessing the cervical cell image to obtain an image-enhanced cervical cell image includes:

converting the cervical cell image satisfying the preset resolution into a product of an illumination component and a reflection component;

separating the illumination component from the reflection component using a logarithmic transformation;

respectively carrying out Fourier transform on the separated illumination component and reflection component to obtain a frequency domain diagram;

processing the irradiation component and the reflection component in the frequency domain graph by using homomorphic filtering to obtain a low-frequency compressed irradiation component and a high-frequency enhanced reflection component;

and fusing the low-frequency compressed illumination component and the high-frequency enhanced reflection component to obtain the cervical cell image after image enhancement.

Optionally, before acquiring the cervical cell image to be measured in the observation area of the microscope, the method further includes:

carrying out image amplification processing on the image of the cervical tissue to be detected;

judging the image of the cervical tissue to be detected after the image amplification treatment;

and if an isolated abnormal coloring region exists, acquiring an accurate part in the cervical tissue to be detected according to the obtained abnormal coloring region.

The step of placing the pathological slide in the observation area of a microscope and automatically adjusting the focal length of the microscope to acquire the current region of interest of the pathological slide comprises the following steps:

acquiring initial target position information in a pathological slide bearing the cervical cells;

controlling at least one of a stage and an objective lens of the microscope to move to an initial target position in a direction perpendicular to a reference mark according to the initial target position information, wherein the reference mark and the target to be measured have a preset relative position;

controlling a camera device of the microscope to take a picture of the reference mark positioned in the detection area in the moving process;

calculating the definition of the shot picture, wherein the definition is a parameter which represents the definition of the cervical cell outline on the picture under the condition that the resolution of the camera device is not changed;

and obtaining the current region of interest according to whether the definition of the shot picture meets the preset cleaning degree range or not and if the definition of the shot picture meets the preset cleaning degree range.

Optionally, before the step of inputting the preprocessed cervical cell image into a pre-trained target network for detection, the method further includes:

marking the preprocessed cervical cell image to construct a training set;

extracting cervical cell images in the training set by using a feature extraction network to obtain feature maps with different scales;

screening the feature graphs of different scales by using an attention mechanism to obtain a weight coefficient representing the importance degree of the features;

fusing the feature maps of different scales according to the weight coefficient to obtain a fused feature map;

and training a neural network by using the fusion characteristic diagram to obtain a trained target network.

Optionally, the method further includes:

optimizing a training breadth convolution neural network by adopting a multi-scale thought based on a training set, and extracting multi-scale features of the cervical cell image by utilizing the breadth convolution neural network;

optimizing and training a dense convolutional neural network by adopting a cross-layer dense connection idea based on a training set, and extracting features of different abstract depths of the cervical cell image by utilizing the dense convolutional neural network;

and fusing the breadth features extracted based on the breadth convolutional neural network and the different abstract depth features extracted based on the dense convolutional neural network according to the weight coefficients, and training the fully-connected neural network by the fused fusion feature map to obtain a target network model.

In addition, to achieve the above object, the present invention provides a cervical cell image abnormality detection apparatus, including:

the acquisition module is used for acquiring cervical cell images to be detected in a microscope observation area;

the interested region acquisition module is used for adjusting the focal length of the microscope to acquire the interested region of the current cervical cell image;

the objective lens adjusting module is used for determining an objective lens multiple corresponding to the current region of interest of the microscope and adjusting the current objective lens multiple until a cervical cell image meeting a preset resolution is obtained;

the preprocessing module is used for preprocessing the cervical cell image to obtain an image-enhanced cervical cell image;

and the abnormality detection module is used for inputting the preprocessed cervical cell image into a pre-trained target network for detection to obtain a detection result of whether the cervical cell image to be detected is abnormal or not.

Furthermore, to achieve the above object, the present invention further provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of the method according to any one of the above embodiments.

Furthermore, to achieve the above object, the present invention further provides a computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the steps of the method according to any one of the above embodiments.

The method comprises the steps of firstly obtaining a pathological slide containing cervical cells to be detected, automatically adjusting the focal length of a microscope to further obtain an interested area of the current pathological slide, determining the cervical cells to be detected in the interested area by selecting a proper objective lens multiple to generate a cervical cell image meeting a preset resolution, and then preprocessing the cervical cell image to obtain a cervical cell image with enhanced image brightness, namely, observing the cervical cells by using the microscope and processing the cervical cell image by using an acquisition mode and a preprocessing mode of the cervical cell image, so that the resolution, the precision and the quality of the obtained cervical cell image are improved, and the abnormal detection efficiency and the detection precision of the cervical cell image are also improved.

Drawings

Fig. 1 is a schematic flow chart of a method for detecting an abnormality in an image of cervical cells according to an embodiment of the present invention;

fig. 2 is a schematic flow chart of a method for detecting an abnormality in an image of cervical cells according to an embodiment of the present invention;

fig. 3 is a schematic flow chart of a method for detecting an abnormality in an image of cervical cells according to an embodiment of the present invention;

fig. 4 is a schematic flow chart of a method for detecting an abnormality in an image of cervical cells according to an embodiment of the present invention;

fig. 5 is a schematic structural diagram of an abnormal cervical cell image detection apparatus provided in an embodiment of the present invention;

fig. 6 is a schematic structural diagram of a computer device provided in an embodiment of the present invention.

The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.

Detailed Description

It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

To facilitate understanding of the present application, the concepts related to the present application will be explained first.

Cervical cancer is one of the malignancies that poses a serious health hazard to women, with the incidence second among women's malignancies. Common cervical cytopathies include: atypical squamous epithelial cells-undefined (ASC-US), low-grade squamous epithelial lesions (LSIL), atypical squamous epithelial cells-nonexclusive high-grade squamous epithelial lesions (ASC-H), high-grade squamous epithelial lesions (HSIL), atypical glandular epithelial cells (AGC), and so forth.

The cervical liquid-based cell inspection method is the most common cervical cancer screening method at present, and most of the current cervical cancer intelligent auxiliary screening systems have low detection precision on ASC-US and other abnormal cells. The deformation degree (nuclear-to-cytoplasmic ratio increase multiple, nuclear heteromorphism degree and the like) of part of abnormal cells in the abnormal cervical cells is low, and some abnormal cervical cells exist in the form of single small cells instead of clustered cells, so that the abnormal cervical cells are difficult to detect in the current intelligent auxiliary cervical cancer screening process, and the detection precision of the abnormal cervical cells is low.

In one embodiment, a method for detecting an image abnormality of a cervical cell is provided, which is shown in fig. 1 and includes the following steps:

and S1, acquiring an image of the cervical cells to be detected in the observation area of the microscope.

In step S1, it is mainly embodied that the target to be tested provides corresponding cervical cells, that is, the cervical cells to be tested are carried on a pathological slide, which includes but is not limited to a pathological section or a pathological smear, for example, the cervical cells to be tested are uniformly smeared on the slide to form a pathological smear containing the cervical cells to be tested, or for example, the cervical cells to be tested are cut into slices on the broken slide to form a pathological slice containing the cervical cells to be tested.

In some embodiments, when extracting cervical cells to be detected of a target to be detected, sometimes due to the influence of gynecological diseases of the target to be detected, for example, cervical tissues have rottenness, leucorrhea or abnormal odor due to cervicitis, vaginitis and other diseases, on one hand, the operation of a doctor is not facilitated, and the observation of the doctor is influenced; on the other hand, the method is not beneficial to the doctor to judge the suspicious lesion area so as to accurately extract the cervical cells to be detected. Therefore, before obtaining the pathology slide containing the cervical cells to be tested, the method further comprises:

collecting cervical tissues to be detected after being washed by normal saline;

acquiring an image of cervical tissue to be detected which is coated with 3% -5% acetic acid solution and compound iodine solution in sequence;

wherein, the cervical tissues to be detected after being smeared with the acetic acid solution and the compound iodine solution are collected through a colposcope.

Wherein, the image of the cervical tissue to be detected is magnified and displayed on a display (computer) by collecting the image of the cervical tissue to be detected coated with the acetic acid solution and the compound iodine solution. For example, if there is a region of the cervix needing biopsy, the cervix can present characteristics of 'thick vinegar white' and 'blood vessel embedding' under the action of 3% -5% acetic acid solution; under the action of the compound iodine solution, the characteristics of bright orange, mustard yellow, spot coloring and the like can be presented, the characteristics can be used as reference of a doctor, whether isolated abnormal coloring areas exist in cervical squamous column boundaries and columnar epithelial areas in the image or not is judged, and if the isolated abnormal coloring areas exist in the cervical squamous column boundaries and the columnar epithelial areas, the accurate parts in the cervical tissue to be detected can be acquired by the abnormal coloring areas by referring to the characteristics. However, the presence of these features does not determine that the cervix will be diseased and therefore requires further judgment by the physician.

By the method, the image of the cervical tissue to be detected after the image amplification processing is judged, so that the accurate part to be detected of the cervical tissue is discriminated, on one hand, the interference of external factors of the cervical tissue to be detected can be filtered, the abnormal area which is possibly subjected to pathological changes can be accurately found, and the accuracy of cell abnormality detection is improved; on the other hand, sample data is enriched by extracting cervical cells to be detected for sample biopsy, and the sample data is favorably marked, so that a training set is quickly constructed.

S2: and adjusting the focal distance of the microscope to acquire the region of interest of the current cervical cell image.

With respect to step S2, it is mainly embodied that the microscope including but not limited to an optical microscope and an electron microscope is applied to the region of interest (i.e., the field of view) of the pathological slide, and the working principle of the microscope is not described herein.

In some embodiments, the focus of the current microscope can be adjusted by using an automatic focusing technology, so as to obtain the region of interest in the current pathological slide, for example, the defocusing of the current pathological slide is judged according to the image gray gradient change in the video signal by using a video signal, the defocusing signal is fed back to a stepping motor driving circuit, and the stepping motor moves the microscope body together with a CCD (image pickup device), so that the automatic focusing is realized. The automatic focusing technology has the advantages of quick response and accuracy, and can dynamically improve the definition of a microscope image in real time.

The region of interest outlines the region to be processed on the pathological slide in a mode of a square frame, a circle, an ellipse, an irregular polygon and the like.

In some embodiments, compared to a hospital, in the prior art, a pathological slide containing cervical cells to be detected is digitally processed by a pathological scanner, and a scanned digital image is input into a screening system, so as to obtain a detection result of the cervical cells. In the embodiment, the image of the cervical cell to be detected is directly acquired through the microscope, so that the acquisition efficiency is greatly improved.

In some embodiments, automatically adjusting the focal length of the microscope to acquire the region of interest of the current pathological slide comprises:

s201: acquiring initial target position information in a pathological slide bearing the cervical cells;

s202: controlling at least one of a stage and an objective lens of the microscope to move to an initial target position in a direction perpendicular to a reference mark according to the initial target position information, wherein the reference mark and the target to be measured have a preset relative position;

s203: controlling a camera device of the microscope to take a picture of the reference mark positioned in the detection area in the moving process;

s204: calculating the definition of the shot picture, wherein the definition is a parameter which represents the definition of the cervical cell outline on the picture under the condition that the resolution of the camera device is not changed;

s205: and obtaining the current region of interest according to whether the definition of the shot picture meets the preset cleaning degree range or not and if the definition of the shot picture meets the preset cleaning degree range.

Specifically, whether the definition of a shot picture meets a preset definition range is judged, if yes, an actual focal length used for determining the measured object is calculated according to the initial target position information and the distance difference between the reference mark and the measured object, and an interested area under the current position information is obtained according to the actual focal length; otherwise, comparing the definition of the shot picture with a definition calibration curve to obtain a compensation value, and calculating the actual focal length of the measured object according to the distance difference between the compensation value and the measured object, wherein the definition calibration curve is a curve generated by the definition of the reference identification picture and the corresponding position information thereof which are acquired in advance.

In some embodiments, one way to obtain initial target location information is to: directly setting the initial target information as a preset fixed value; the other mode is as follows: and finding out the position with the maximum definition value from the definition calibration curve, wherein the position information (such as Z-direction coordinates) corresponding to the maximum definition value is the initial target position information.

In some embodiments, after the initial target position information is acquired, the coordinate information in the Z direction is converted into the number of operation steps of a stepping motor in the driving device, and the objective lens is driven to move to the initial target position in a direction perpendicular to the reference mark (for example, in the Z direction) by controlling the driving device.

In some embodiments, the control device controls the camera device to take a picture of the reference mark when at least one of the stage and the objective lens is moved to the initial target position, so as to avoid the influence of image noise or mark drift, for example, the camera device may take a picture once when moving a fixed distance or at preset time intervals.

And comparing the definition of the shot picture with the definition calibration curve to obtain a compensation value.

In some embodiments, the optimum definition on the definition calibration curve obtained in advance is searched, the difference between the definition of the corresponding position of the shot picture and the corresponding position on the definition calibration curve is calculated, if the difference is smaller than a set threshold, the compensation value is zero, and if the difference is greater than or equal to the set threshold, the compensation value is the difference between the initial target position information and the position information corresponding to the optimum definition.

By the embodiment, the visual field under the current microscope is analyzed, the focal length of the current microscope can be automatically adjusted, the visual field under the microscope is expanded, and cervical cells under a pathological slide can be identified; in addition, through a focusing closed-loop control mode, the times of repeated focusing are reduced, the detection speed is not influenced, and the definition and the stability of an image are ensured; meanwhile, the method is not influenced by factors such as pollution degree, running time and abrasion condition of the detection pool, and real automatic focusing is realized.

S3: and determining the objective lens multiple corresponding to the current interested area of the microscope, and adjusting the current objective lens multiple until obtaining a cervical cell image meeting the preset resolution.

For step S3, it is mainly embodied that the current region of interest can be obtained after adjusting the focal length of the microscope in step S2, so as to form a coarse resolution field of view; because the microscope has a great influence on the resolution precision under the condition of the multiple corresponding to different objective lenses, namely, the cervical cell image meeting the requirements of high resolution or high precision can be further obtained by adjusting the multiple of the objective lens in the region of interest.

Wherein the preset resolution can be a threshold value set by a person skilled in the art according to needs.

It should be noted that, since the magnification of the microscope is equal to the product of the magnification of the objective lens and the magnification of the eyepiece, the higher the magnification speed of the objective lens is, the longer the lens is, and the closer the objective lens is to the stage, the higher the magnification of the eyepiece is, and the shorter the lens is. Therefore, the times of the objective lenses need to be reasonably adjusted to ensure that cervical cell images meeting the preset resolution are obtained under a microscope.

It should be noted that the microscopic magnification is any one of the length, width, or diameter of the cervical cells, not the area, volume, or surface area, and is inversely proportional to the magnification if the cervical cells in the magnified field are aligned in a line, or inversely proportional to the square of the magnification if the cervical cells in the magnified field are randomly arranged.

In some embodiments, determining a multiple of an objective lens corresponding to the current region of interest of the microscope to obtain a cervical cell image satisfying a preset resolution, further includes:

s301: extracting input characteristics of cervical cell images in the current region of interest;

s302: processing the input features by using an additive neural network to obtain output features;

s303: calculating the similarity between the input features and the output features by using the distance measurement to generate a measurement result;

s304: normalizing the measurement result to obtain a nonlinear relation between the input characteristic and the output characteristic in the measurement result;

s305: and determining the objective lens multiple of the microscope according to the nonlinear relation among the characteristics, and adjusting the current objective lens multiple until a cervical cell image meeting the preset resolution is obtained.

The addition neural network is used for replacing the traditional convolution neural network, and a large amount of multiplication operations in the convolution neural network are replaced by addition operations with higher speed, so that the operation speed of the target network can be obviously increased, and the data processing speed of the target network is further improved; meanwhile, the function of reducing the power consumption of network computation is achieved. Since the addition operation is more computationally efficient and consumes less power than the multiplication operation due to the hardware computations.

Wherein, the distance measurement mode includes, but is not limited to, Euclidean distance, cosine similarity, Hamming distance, Manhattan distance, Chebyshev distance, Min distance, Jacobian exponent and half-positive vector distance; in addition, the Normalization processing mode is Batch Normalization, namely BN preprocessing, the BN preprocessing can effectively solve the problem that data distribution among layers is changed before the neural network pooling layer feature extraction kernel is added in the training process, sample data can be randomized, and the probability that a certain sample is always selected in each Batch of training is effectively avoided.

In some embodiments, the additive neural network may include one or more additive filter layers, and may further include other layers such as an input layer, a pooling layer, an implicit layer, or an output layer, which is not limited in this embodiment. A plurality of additive filter layers may be included in the additive neural network, and each additive filter layer may include one or more feature extraction kernels. I.e. a plurality of feature extraction cores may be included in the additive neural network. Accordingly, the cervical cell image may be subjected to a plurality of feature extraction processes by the plurality of feature extraction checks to obtain an input feature, the output feature including a plurality of input sub-features.

In the embodiment, the amplification factor of the current objective lens is automatically identified, the cervical cell image of the current interest area of the microscope is analyzed by using an additive neural network, whether the corresponding image resolution meets the preset resolution under the current amplification factor condition is obtained, and if so, the amplification factor of the current objective lens is maintained; if the current image resolution does not meet the preset resolution, according to the condition that the current image resolution does not meet the preset resolution, for example, when the current image resolution is lower than the preset resolution, the condition that the current image resolution does not meet the preset resolution is shown, at the moment, because the magnification factor of the current objective is increased, the magnification factor of the objective can be automatically adjusted through the mode, the detection precision and efficiency are improved, and the additional influence of manual factors is avoided.

S4: and preprocessing the cervical cell image to obtain an image-enhanced cervical cell image.

For step S4, it is mainly embodied to preprocess the cervical cell image, and the preprocessing in this embodiment is mainly to adjust the influence of light on the microscope imaging, so as to overcome the problem of poor image quality caused by uneven lighting, and further effectively enhance the contrast of the image.

In some embodiments, the step of preprocessing the cervical cell image to obtain an image-enhanced cervical cell image comprises:

s401: converting the cervical cell image satisfying the preset resolution into a product of an illumination component and a reflection component;

s402: separating the illumination component from the reflection component using a logarithmic transformation;

s403: respectively carrying out Fourier transform on the separated illumination component and reflection component to obtain a frequency domain diagram;

s404: processing the irradiation component and the reflection component in the frequency domain graph by using homomorphic filtering to obtain a low-frequency compressed irradiation component and a high-frequency enhanced reflection component;

s405: and fusing the low-frequency compressed illumination component and the high-frequency enhanced reflection component to obtain the cervical cell image after image enhancement.

Wherein, logarithmic transformation can be used for low gray values with a narrow stretching range and high gray values with a wide compressing range; it can also be used to expand dark pixel values in an image while compressing light pixel values.

For example, for an enhanced cervical cell image F (x, y), the image F (x, y) is represented by the product of the illumination component i (x, y) and the reflection component r (x, y) by conversion; the following logarithmic expressions will be obtained by logarithmic conversion; inf (x, y) ═ Ini (x, y) + Inr (x, y); and performing Fourier transform to obtain I { Inf (x, y) } ═ I { Ini (x, y) } + I { Inr (x, y) }, wherein after the Fourier transform, low-frequency components in the image are related to illumination, and high-frequency components are related to reflection. In this case, the illumination component and the reflection component can be better controlled and better controlled by adopting a homomorphic filtering mode. The homomorphic filter is a filter corresponding to homomorphic filtering, and different controllable methods can be used for influencing the low frequency and the high frequency of Fourier transform. If γ L <1 and γ H >1 are chosen, the filter function tends to attenuate low frequencies (illumination) while enhancing high frequency reflections, the end result being simultaneous compression of the dynamic range and contrast enhancement, where the image filter function involved is:

where D (u, v), D0 are the distance of the center of frequency of the filter function and the cutoff frequency, respectively, and the c constant is used to control the sharpness of the function slope.

Optionally, homomorphic filtering may be used to eliminate the effect of uneven illumination without loss of image detail, where the grey scale of the image is synthesized from the illumination component and the reflection component, and the reflection component reflects the image content and changes rapidly spatially with the image detail; the illumination components are typically of a slowly varying nature spatially; the frequency spectrum of the illumination component falls in a spatial low frequency region, and the frequency spectrum of the reflection component falls in a spatial high frequency region.

Through the embodiment, before the cervical cell image is input into the target network, the cervical cell image can be preprocessed and corrected by using homomorphic filtering, so that the contrast of the image is improved, and the quality of the cervical cell image is improved.

S5: inputting the preprocessed cervical cell image into a pre-trained target network for detection to obtain a detection result of whether the cervical cell image to be detected is abnormal or not.

As for step S5, it is mainly embodied that, in this embodiment, the preprocessed cervical cell image to be detected is input into the target network, and is identified by the target network, so that the detection result of whether the cervical cell image to be detected is abnormal can be quickly determined.

It should be noted that the target network may also be a fast-RCNN deep convolutional neural network model, and when applied, the output of the target network is the prediction probability of abnormal cells with the target region as the background. Inputting the cervical cell image to be identified into a trained fast-RCNN deep convolution neural network model, and obtaining abnormal cells with different numbers and corresponding prediction probabilities through the feature extraction, the region selection network selection and the final classification.

By the method for recognizing the cervical cell image based on the convolutional neural network, a detection result for judging whether a target is abnormal can be obtained by inputting any digital image of the cervical fluid-based smear to the obtained Faster-RCNN model. It should be noted that the model training mode in the embodiment of the present invention is a creative work achievement for those skilled in the art, and all the changes, adjustments or replacement schemes of the data enhancement mode, the neural network architecture, the hyper-parameter and the loss function in the present invention based on the embodiment of the present invention should be regarded as being equivalent to the present scheme.

When a detection result targeting abnormal cells is obtained, a detection result larger than a set confidence threshold may be displayed by setting the confidence threshold.

In a first embodiment: and acquiring the characteristics of the cervical cells in each target cervical cell pathology slide image block so as to determine the abnormal probability of the cervical cells in each image block. For example, the cervical cell characteristics are compared to the cellular characteristics of normal cervical cells to determine the probability of abnormality of the cervical cells.

And when the abnormal probability of the cervical cell is not less than a preset abnormal probability threshold value, determining the cervical cell as an abnormal detection result.

Specifically, when the abnormality probability is not less than the preset abnormality probability threshold, the cervical cell may be considered to be abnormal, so that the abnormal cervical cell existing in each image block may be detected. And by combining the detection condition in each target cervical cell pathological slide image block, the abnormal cervical cells in the cervical cell pathological slide image can be detected, and the automatic detection of the abnormal cervical cells is realized.

Optionally, after determining that the cervical cell is an abnormal detection result, acquiring position information of the cervical cell in the corresponding target cervical cell pathological slide image block to obtain a position of the abnormal cervical cell in the cervical cell pathological slide image.

Specifically, for each target cervical cell pathology slide image block, the abnormality probability of the cervical cells is acquired. When the abnormal probability is not less than the preset abnormal probability threshold, the cervical cell is considered as an abnormal detection result, the position information of the cervical cell in the corresponding target cervical cell pathological slide image block is obtained, and the position information and the abnormal probability of the cervical cell are reserved, so that the position information and the abnormal probability of the abnormal cervical cell in each image block can be detected. In combination with the detected conditions in each target cervical cytopathic slide image block, the location of abnormal cervical cells in the cervical cytopathic slide image can be determined. When displaying, the abnormal cervical cells can be displayed according to the positions of the abnormal cervical cells in the cervical cell pathology slide image, and the corresponding abnormal probability can also be displayed.

Through the method, the abnormal cells in the cervical cell image to be detected are classified, diagnosis suggestions can be given to the slice-level results, a clinician is assisted, and the workload of the clinician is reduced. Meanwhile, the diagnosis suggestion provided by the invention has higher sensitivity and specificity, and it should be noted that the invention can be equally applied to the automatic detection of other pathological digital images in the medical field, such as the detection of cervical cell images and the like, and the invention does not limit the invention.

In some embodiments, before inputting the preprocessed cervical cell image into the pre-trained target network for detection, the method further includes: s6: constructing a target network for identifying the cervical cell image, which specifically comprises the following steps:

s601: marking the preprocessed cervical cell image to construct a training set;

s602: extracting cervical cell images in the training set by using a feature extraction network to obtain feature maps with different scales;

s603: screening the feature graphs of different scales by using an attention mechanism to obtain a weight coefficient representing the importance degree of the features;

s604: fusing the feature maps of different scales according to the weight coefficient to obtain a fused feature map;

s605: and training a neural network by using the fusion characteristic diagram to obtain a trained target network.

In some embodiments, the preprocessed cervical cell image is labeled, and the step of constructing the training set includes:

the cervical cell images after image enhancement can be manually marked by medical staff, and the marked cervical cell images are collected and classified to form a training set.

For example, the categories of abnormal cells or biological pathogens that need to be labeled can also be as follows:

squamous cells include: atypical squamous epithelial cells (low-grade squamous epithelial lesions, except high-grade squamous epithelial lesions) and squamous cell carcinoma (high-grade squamous epithelial lesions);

the glandular cells include: atypical gland cells (cervical canal cells, endometrial cells), cervical canal gland cells (prone to tumorigenesis), cervical canal in situ adenocarcinoma, adenocarcinoma (cervical canal adenocarcinoma, endometrial adenocarcinoma, extrauterine adenocarcinoma);

biological pathogens include: trichomonas vaginalis, fungi with a morphology consistent with candida albicans (dysbacteriosis, prompting bacterial vaginosis), bacteria with a morphology consistent with actinomycetes (cytological changes are consistent with herpes simplex virus infection);

and, endometrial cells.

In some embodiments, extracting images of cervical cells in the training set using a feature extraction network to obtain feature maps of different scales includes:

specifically, a multi-scale thought is adopted to optimize a training breadth convolutional neural network based on a training set, multi-scale features of the cervical cell image are extracted by using the breadth convolutional neural network, and a combined center loss function and a Softmax loss function are adopted as a loss function of the training breadth convolutional neural network;

optionally, a training dense convolutional neural network is optimized based on a training set by adopting a cross-layer dense connection idea, features of different abstract depths of the cervical cell image are extracted by using the dense convolutional neural network, and a loss function of the training dense convolutional neural network is in a mode of combining a central loss function and a Softmax loss function.

Optionally, the breadth features extracted based on the breadth convolutional neural network and the different abstract depth features extracted based on the dense convolutional neural network are fused according to the weight coefficients, and the fused feature graph trains the fully-connected neural network to obtain the target network model.

Specifically, the breadth convolution neural network has multiple scales of convolution kernels, and classification is performed after the length features of all scales are fused, so that the problem of inconsistent important features is solved, the extracted breadth features are more effective, and the classification accuracy is improved; meanwhile, information of different depths is extracted by adopting the intensive convolutional neural network, all convolutional layers are output, fused and spliced to serve as input, the integrity of feature extraction is improved, the fully-connected neural network is built to fuse features of different depths and features of wide ranges, and the accuracy of cervical cell image feature classification is further improved.

In some embodiments, screening feature maps of different scales by using an attention mechanism to obtain a weight coefficient representing the importance degree of a feature includes:

specifically, after filtering the features of different scales by using an attention mechanism, the important features are highlighted, and the secondary features are suppressed, for example, quantized by using different weight coefficients according to the highlighted degree of the important features.

In some embodiments, classifying the fused feature map to obtain a trained target network includes:

optimizing and training an breadth convolutional neural network by adopting a multi-scale thought, extracting multi-scale characteristics of the image data by utilizing the breadth convolutional neural network, wherein a corresponding loss function adopts a combined central loss function and a Softmax loss function; the corresponding expression is as follows:

Loss＝Softmaxlos+λCentorloss

in the formula, the corresponding Loss function after combination is Loss, the Softmax Loss function is Softmaxloss, the central Loss function is Centorlos, and the lambda represents the coefficient size;

the extensive convolutional neural network comprises a convolutional kernel with multiple scales, each channel corresponds to a convolutional layer with one scale, and the convolutional layer comprises a first connecting layer, a second connecting layer, a first maximum pooling layer, a second maximum pooling layer, a first full connecting layer, a second full connecting layer and an output layer; when the convolution layers are two layers and the number of input channels is three, the input image data sequentially passes through a first connecting layer and a first maximum pooling layer from the first layer of convolution layers to output a first multi-scale feature map; the first multi-scale feature map outputs a second multi-scale feature map through a second connection layer of a second layer of convolutional layers and a second maximum pooling layer of the same structural channel; and the second multi-scale feature map outputs multi-scale features in the first full connection layer and the second full connection layer in sequence, and a classification result is output by using a Softmax classifier.

Optionally, the dense convolutional neural network includes at least four convolutional layers of convolutional kernels of the same scale, a first connection layer, a second connection layer, a third connection layer, a first maximum pooling layer, a first full connection layer, a second full connection layer, and an output layer, where an input image sequentially passes through the first convolutional layer, the second convolutional layer, the first connection layer, the third convolutional layer, the second connection layer, a fourth convolutional layer, and the third connection layer; each connecting layer connects the output of all the convolution layers in front of the connecting layer, so that dense connection can enable the network to extract more effective features with different depths; the third connecting layer sequentially pools the layer, the first full connecting layer and the second full connecting layer to output different depth features, and a classification result is output by using a Softmax classifier.

It should be noted that the loss functions of the dense convolutional neural network and the fully-connected neural network adopt a manner of combining the central loss function and the Softmax loss function, which is the same as the loss function adopted by the above-mentioned extensive convolutional neural network, and details are not repeated here.

Aiming at the problem that important features in data in cervical cell images are different in size, an extensive convolutional neural network is provided for extracting the features, the neural network utilizes the multi-scale thought, the structure of the convolutional neural network is improved, convolution kernels of different scales are utilized for extracting the features of the cervical cell images, and then the features of all scales are fused for classification, so that the problem that the important features are different in size can be solved to a certain extent. In this embodiment, three convolution kernels of different sizes, 1 × 1, 3 × 3 and 5 × 5, respectively, are preferred. The features extracted in this way can be more effective, so that the classification accuracy is improved.

The conventional convolutional neural network is generally classified by using features extracted from the last convolutional layer, so that some upper layer information is ignored, that is, only one depth of information is used, and some important information may be lost. The present embodiment provides a dense convolutional neural network to extract information of different depths, and when each convolution is operated, the outputs of all previous convolutional layers are spliced together as input, so that the obtained features can contain information of various depths.

When the breadth and dense convolutional neural network is trained, the network is trained by combining a central loss function and a Softmax loss function, and the intra-class distance of each class of results can be reduced by introducing the central loss function, so that the classification accuracy is improved.

A fully-connected neural network is built and used for combining the features extracted by the wide convolutional neural network and the dense convolutional neural network for use, and the extracted wide features and the features of different depths are fully utilized. In addition, the fully-connected neural network can further select the features, so that more effective features can be selected from the existing features, and the classification accuracy is further improved.

Extracting multi-scale breadth features of a cervical cell image by using a breadth convolution neural network, extracting depth features of the cervical cell image by using a dense convolution neural network, simultaneously respectively receiving the multi-scale breadth features and the depth features by using an attention module, respectively extracting the dependency relationships of the multi-scale breadth features and the depth features in different pixels by using matrix multiplication, and performing weighted fusion on the breadth features and the depth features according to the dependency relationships among the pixels to obtain fused final features; inputting the fused features into a fully-connected neural network for training to obtain an abnormality detection model for identifying the abnormality of the cervical cells to be detected.

By the embodiment, the abnormal cells of the cervical cell image are identified by combining the multi-scale feature fusion network of the attention mechanism, so that the detection accuracy is greatly improved compared with other modes; in addition, the attention mechanism can effectively filter the features with different scales, highlight important features and inhibit secondary features, so that the feature learning is more efficient; the fusion network which is designed according with the actual distribution of the sizes of the abnormal cervical cells and accords with the clinical actual multi-scale characteristics can cover the abnormal cells with different sizes, and the missing detection of the small-size cells is prevented.

In addition, the breadth characteristic and the depth characteristic can fuse the characteristics of different levels together, and the expression capability of the characteristics is enriched, so that the detailed information of the cells is better extracted, the detection precision of cells such as ASC-US similar to normal cells of the cervix can be improved, and the detection precision of abnormal cells of the cervix is effectively improved.

The above-mentioned method for detecting an abnormality in a cervical cell image is exemplarily described below by a specific embodiment, and the specific method for detecting an abnormality in a cervical cell image includes:

firstly, segmenting cervical tissue to be detected to obtain cervical tissue slices, and placing the cervical tissue slices on a pathological slide which is placed in an observation area of an electron microscope; secondly, the electron microscope automatically adjusts the focal length of the microscope according to the current observation area to obtain the area of interest of the current pathological slide; thirdly, analyzing the cervical cell image of the current interested area of the microscope by utilizing an addition neural network through automatically identifying the magnification of the current objective lens to obtain whether the corresponding image resolution under the current magnification situation meets the preset resolution, and if so, keeping the magnification of the current objective lens; if the current image resolution does not meet the preset resolution, automatically identifying the magnification factor according to the condition that the current image resolution does not meet the preset resolution; by automatically adjusting the magnification of the objective lens, the detection precision and efficiency are improved, and additional influence of artificial factors is avoided.

Moreover, the use of homomorphic filtering eliminates the effects of non-uniform illumination without loss of image detail, wherein the gray level of the image is synthesized from an illumination component and a reflection component, the reflection component reflects the image content and rapidly changes spatially with the image detail; the illumination components are typically of a slowly varying nature spatially; the frequency spectrum of the illumination component falls in a spatial low frequency region, and the frequency spectrum of the reflection component falls in a spatial high frequency region. Through the embodiment, before the cervical cell image is input into the target network, the cervical cell image can be preprocessed and corrected by using homomorphic filtering, so that the contrast of the image is improved, and the quality of the cervical cell image is improved.

And finally, determining the abnormal probability of the cervical cells in each image to be detected according to the acquired cervical cell characteristics, so as to obtain a detection result of whether the cervical cells are abnormal or not. For example, the cervical cell characteristics are compared to the cellular characteristics of normal cervical cells to determine the probability of abnormality of the cervical cells. By classifying abnormal cells in the cervical cell image to be detected, diagnosis suggestions can be given to the slice-level results, an auxiliary effect is achieved for clinicians, and the workload of the clinicians is reduced. Meanwhile, the diagnosis suggestion provided by the invention has higher sensitivity and specificity.

The embodiment provides a cervical cell image anomaly detection method, which includes the steps of firstly obtaining a pathological slide containing cervical cells to be detected, automatically adjusting the focal length of a microscope to further obtain an interested area of the current pathological slide, determining cervical cells to be detected in the interested area by selecting a proper objective lens multiple to generate a cervical cell image meeting a preset resolution, and then preprocessing the cervical cell image to obtain a cervical cell image with enhanced image brightness (the contrast of the image is improved, so that the visual effect of the image is improved).

In one embodiment, the present invention further provides a cervical cell image abnormality detection apparatus 500, referring to fig. 5, including:

the acquisition module 501 is used for acquiring cervical cell images to be detected in a microscope observation area;

a region-of-interest acquiring module 502, configured to adjust a focal length of the microscope to acquire a region of interest of the current cervical cell image;

the objective lens adjusting module 503 is configured to determine an objective lens multiple corresponding to the current region of interest of the microscope, and adjust the current objective lens multiple until a cervical cell image meeting a preset resolution is obtained;

a preprocessing module 504, configured to preprocess the cervical cell image to obtain an image-enhanced cervical cell image;

and an anomaly detection module 505, configured to input the preprocessed cervical cell image into a pre-trained target network for detection, so as to obtain a detection result of whether the cervical cell image to be detected is abnormal.

In this embodiment, the objective lens adjusting module 503 further includes:

the extraction unit is used for extracting the input characteristics of the cervical cell image in the current region of interest;

the processing unit is used for processing the input characteristics by utilizing an additive neural network to obtain output characteristics;

the measuring unit calculates the similarity between the input features and the output features by using distance measurement to generate a measuring result;

the normalization processing unit is used for normalizing the measurement result to obtain a nonlinear relation between the input characteristic and the output characteristic in the measurement result;

and the objective lens adjusting unit is used for determining the objective lens multiple of the microscope according to the nonlinear relation among the characteristics to obtain the cervical cell image meeting the preset resolution.

In this embodiment, the preprocessing module 504 further includes:

a conversion unit for converting the cervical cell image satisfying the preset resolution into a product of an illumination component and a reflection component;

a transformation separation unit that separates the illumination component and the reflection component by logarithmic transformation;

the frequency spectrum transformation unit is used for respectively carrying out Fourier transformation on the separated illumination component and reflection component to obtain a frequency domain diagram;

the filtering processing unit is used for processing the irradiation component and the reflection component in the frequency domain graph by homomorphic filtering to obtain a low-frequency compressed irradiation component and a high-frequency enhanced reflection component;

and the fusion unit is used for obtaining the cervical cell image after image enhancement by fusing the low-frequency compressed irradiation component and the high-frequency enhanced reflection component.

In this embodiment, the method further includes: the target network construction module is used for constructing a target network for identifying the cervical cell image; the target network building module further comprises:

the training set construction unit is used for marking the preprocessed cervical cell image to construct a training set;

the characteristic extraction unit is used for extracting cervical cell images in the training set by utilizing a characteristic extraction network to obtain characteristic graphs of different scales;

the weight calculation unit is used for screening the feature graphs of different scales by using an attention mechanism to obtain a weight coefficient representing the importance degree of the features;

the characteristic fusion unit is used for fusing the characteristic graphs of different scales according to the weight coefficient to obtain a fusion characteristic graph;

and the target network construction unit is used for training the neural network by using the fusion characteristic diagram to obtain a trained target network.

In this embodiment, the target network construction unit further includes:

the first network construction subunit optimizes a training breadth convolutional neural network by adopting a multi-scale thought based on a training set, and extracts multi-scale features of the cervical cell image by utilizing the breadth convolutional neural network;

a second network construction subunit, which optimizes and trains the dense convolutional neural network by adopting a cross-layer dense connection idea based on the training set, and extracts the features of different abstract depths of the cervical cell image by utilizing the dense convolutional neural network;

and a target network construction subunit, fusing the breadth features extracted based on the breadth convolutional neural network and the different abstract depth features extracted based on the dense convolutional neural network according to weight coefficients, and training the fully-connected neural network by the fused fusion feature graph to obtain a target network model.

In this embodiment, before the acquiring module 501, the method further includes:

In this embodiment, the region of interest obtaining module 502 further includes:

acquiring initial target position information in a pathological slide;

The embodiment provides a cervical cell image anomaly detection device, the device is through acquireing the pathology slide that contains the cervical cell that awaits measuring earlier, automatically regulated microscope's focus and then obtain the region of interest of current pathology slide, the cervical cell image that satisfies preset resolution ratio is generated to the cervical cell that awaits measuring in the region of interest through selecting suitable objective multiple determination, carry out the preliminary treatment to the cervical cell image again, obtain the cervical cell image of image brightness reinforcing, promptly, utilize microscope observation cervical cell, and from its collection mode, the preliminary treatment mode is handled the cervical cell image, the resolution ratio of the cervical cell image that has improved the acquisition, precision and quality, cervical cell image anomaly detection efficiency and detection precision have also been improved.

It should be understood that the above-mentioned cervical cell image abnormality detection apparatus system is substantially provided with a plurality of modules for executing the cervical cell image abnormality detection method in any of the above-mentioned embodiments, and specific functions and technical effects refer to the above-mentioned embodiments, and are not described herein again.

In an embodiment, referring to fig. 6, the embodiment further provides a computer device 600, which includes a memory 601, a processor 602, and a computer program stored on the memory and executable on the processor, and when the processor 602 executes the computer program, the steps of the method according to any one of the above embodiments are implemented.

In an embodiment, a computer-readable storage medium is also provided, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method according to any of the above embodiments.

The embodiment of the application can acquire and process related data based on an artificial intelligence technology. Among them, Artificial Intelligence (AI) is a theory, method, technique and application system that simulates, extends and expands human Intelligence using a digital computer or a machine controlled by a digital computer, senses the environment, acquires knowledge and uses the knowledge to obtain the best result.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, apparatus, article, or method. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, apparatus, article, or method that includes the element.

The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments. Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) as described above and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.

The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.

Claims

1. A method for detecting image abnormalities of cervical cells, the method comprising:

2. The method for detecting cervical cell image abnormality according to claim 1, wherein the step of determining the objective lens multiple corresponding to the current region of interest of the microscope, and adjusting the current objective lens multiple until obtaining the cervical cell image satisfying a preset resolution includes:

3. The method for detecting cervical cell image abnormality according to claim 1, wherein the step of preprocessing the cervical cell image to obtain an image-enhanced cervical cell image includes:

4. The method for detecting abnormality of cervical cell image according to any one of claims 1 to 3, wherein said acquiring the cervical cell image to be measured in the observation area of the microscope further comprises:

5. The cervical cell image abnormality detection method according to any one of claims 1 to 3, wherein the step of adjusting the focal length of the microscope to acquire the region of interest of the current cervical cell image includes:

6. The method for detecting cervical cell image abnormality according to claim 1, wherein before the step of inputting the preprocessed cervical cell image into a pre-trained target network for detection, the method further comprises:

marking the preprocessed cervical cell image to construct a training set;

7. The method for detecting an image abnormality of a cervical cell according to claim 6, further comprising:

8. An apparatus for detecting image abnormalities of cervical cells, the apparatus comprising:

9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the steps of the method of any of claims 1 to 7 are implemented by the processor when executing the computer program.

10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 7.