CN114283406A

CN114283406A - Cell image recognition method, device, equipment, medium and computer program product

Info

Publication number: CN114283406A
Application number: CN202111074023.3A
Authority: CN
Inventors: 林一; 曲志勇; 李悦翔; 马锴; 郑冶枫
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2021-09-14
Filing date: 2021-09-14
Publication date: 2022-04-05

Abstract

The application discloses a cell image identification method, a cell image identification device, a cell image identification equipment, a cell image identification medium and a computer program product, and relates to the field of deep learning. The method comprises the following steps: acquiring a sample data set; inputting the sample data set into a cell nucleus recognition model, and outputting to obtain a segmentation prediction result of a first cell image and a coloring prediction result of a second cell image; obtaining a segmentation loss value based on the cell nucleus information and the segmentation prediction result of the first cell image; obtaining a staining loss value based on the second cell image and the staining prediction result; and training the cell nucleus recognition model based on the segmentation loss value and the coloration loss value to obtain a target recognition model, wherein the target recognition model is used for recognizing the cell nucleus in the cell image. The cell image is input into the cell nucleus recognition model, the cell nucleus recognition model is trained on the basis of the segmentation loss value and the coloring loss value to obtain the target recognition model, and the model precision and the recognition efficiency of cell recognition are improved.

Description

Cell image recognition method, device, equipment, medium and computer program product

Technical Field

Embodiments of the present disclosure relate to the field of deep learning, and in particular, to a method, an apparatus, a device, a medium, and a computer program product for cell image recognition.

Background

The histological examination is a pathological tissue screening means, belongs to one of clinical case examinations, and can help diagnosis of many diseases, especially some suspicious malignant pathological diseases, such as cervical cancer or pancreatic cancer.

In the related technology, the cell nucleus segmentation is a key step that the histopathology image can be automatically analyzed in the histological examination, the cell nucleus segmentation usually adopts a method of performing threshold segmentation on the cell nucleus, and the cell image is subjected to related preprocessing through morphological operation to improve the quality of the cell image, so that a cell nucleus segmentation result is obtained.

However, the result of the segmentation of the cell nucleus obtained by only morphological operations often has a large difference from the real state of the cell nucleus, and the segmentation accuracy is low, which affects the diagnosis result, so that there is a possibility of misdiagnosis and missed diagnosis.

Disclosure of Invention

The embodiment of the application provides a cell image identification method, a cell image identification device, cell image identification equipment, a cell image identification medium and a cell image identification computer program product, which can improve the accuracy of a cell nucleus identification result. The technical scheme is as follows:

in one aspect, a cell image recognition method is provided, the method including:

acquiring a sample data set, wherein the sample data set comprises a first cell image and a second cell image, and the first cell image is marked with cell nucleus information;

inputting the sample data set into a cell nucleus recognition model, and outputting a segmentation prediction result of the first cell image and a coloring prediction result of the second cell image;

obtaining a segmentation loss value based on the cell nucleus information of the first cell image and the segmentation prediction result;

deriving a staining loss value based on the second cell image and the staining prediction result;

and training the cell nucleus recognition model based on the segmentation loss value and the coloration loss value to obtain a target recognition model, wherein the target recognition model is used for recognizing the cell nucleus in the cell image.

In another aspect, there is provided a cell image recognition apparatus, the apparatus including:

the system comprises a sample acquisition module, a cell identification module and a cell identification module, wherein the sample acquisition module is used for acquiring a sample data set, the sample data set comprises a first cell image and a second cell image, and the first cell image is marked with cell nucleus information;

the output module is used for inputting the sample data set into a cell nucleus recognition model and outputting a segmentation prediction result of the first cell image and a coloring prediction result of the second cell image;

a loss obtaining module, configured to obtain a segmentation loss value based on the cell nucleus information of the first cell image and the segmentation prediction result;

the loss acquisition module is further used for obtaining a staining loss value based on the second cell image and the staining prediction result;

and the training module is used for training the cell nucleus recognition model based on the segmentation loss value and the coloration loss value to obtain a target recognition model, and the target recognition model is used for recognizing the cell nucleus in the cell image.

In another aspect, there is provided a computer device comprising a processor and a memory, the memory having stored therein at least one instruction, at least one program, set of codes, or set of instructions, which is loaded and executed by the processor to implement the method of cell image recognition as described in any of the embodiments of the present application.

In another aspect, there is provided a computer readable storage medium having stored therein at least one instruction, at least one program, set of codes, or set of instructions, which is loaded and executed by a processor to implement a cell image recognition method as described in any one of the embodiments of the present application.

In another aspect, a computer program product or computer program is provided, the computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to cause the computer device to execute the cell image recognition method described in any of the above embodiments.

The beneficial effects brought by the technical scheme provided by the embodiment of the application at least comprise:

the method comprises the steps of inputting a first cell image and a second cell image in a sample data set into a cell nucleus recognition model to carry out segmentation prediction training and coloring prediction training respectively, obtaining a segmentation prediction result corresponding to the first cell image with a label and a coloring prediction result corresponding to the second cell image without the label, calculating a segmentation loss value and a coloring loss value corresponding to each segmentation prediction result and the coloring prediction result, training the cell nucleus recognition model based on the segmentation loss value and the coloring loss value, improving the model precision of the cell nucleus recognition model, and improving the accuracy and the recognition efficiency of cell nucleus recognition.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

FIG. 1 is a flow chart of an overall implementation provided by an exemplary embodiment of the present application;

FIG. 2 is a schematic diagram of an implementation environment of a cell image recognition method according to an exemplary embodiment of the present disclosure;

FIG. 3 is a flow chart of a method for cell image recognition provided by an exemplary embodiment of the present application;

FIG. 4 is a flow chart of a method of cell image identification provided by another exemplary embodiment of the present application;

FIG. 5 is a flow chart of a method of cell image identification provided by another exemplary embodiment of the present application;

FIG. 6 is a schematic diagram of obtaining a grayscale image according to an embodiment of the present application;

FIG. 7 is a schematic diagram of a cell image identification process provided by an exemplary embodiment of the present application;

fig. 8 is a schematic structural diagram of a cell image recognition apparatus according to an exemplary embodiment of the present application;

fig. 9 is a schematic structural diagram of a cell image recognition apparatus according to another exemplary embodiment of the present application;

fig. 10 is a schematic structural diagram of a server according to an exemplary embodiment of the present application.

Detailed Description

To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

First, a brief description is given of terms referred to in the embodiments of the present application:

artificial Intelligence (AI): the method is a theory, method, technology and application system for simulating, extending and expanding human intelligence by using a digital computer or a machine controlled by the digital computer, sensing the environment, acquiring knowledge and obtaining the best result by using the knowledge. In other words, artificial intelligence is a comprehensive technique of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a manner similar to human intelligence. Artificial intelligence is the research of the design principle and the realization method of various intelligent machines, so that the machines have the functions of perception, reasoning and decision making.

The artificial intelligence technology is a comprehensive subject and relates to the field of extensive technology, namely the technology of a hardware level and the technology of a software level. The artificial intelligence infrastructure generally includes technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and the like.

Machine Learning (ML): the method is a multi-field cross discipline and relates to a plurality of disciplines such as probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and the like. The special research on how a computer simulates or realizes the learning behavior of human beings so as to acquire new knowledge or skills and reorganize the existing knowledge structure to continuously improve the performance of the computer. Machine learning is the core of artificial intelligence, is the fundamental approach for computers to have intelligence, and is applied to all fields of artificial intelligence. Machine learning and deep learning generally include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and teaching learning.

Computer Vision technology (Computer Vision, CV): the method is a science for researching how to make a machine see, and particularly refers to that a camera and a computer are used for replacing human eyes to perform machine vision such as identification, tracking and measurement on a target, and further graphics processing is performed, so that the computer processing becomes an image more suitable for human eyes to observe or is transmitted to an instrument to detect. As a scientific discipline, computer vision research-related theories and techniques attempt to build artificial intelligence systems that can capture information from images or multidimensional data. The computer vision technology generally includes image processing, image Recognition, image semantic understanding, image retrieval, Optical Character Recognition (OCR), video processing, video semantic understanding, video content/behavior Recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, synchronous positioning, map construction, and other technologies, and also includes common biometric technologies such as face Recognition and fingerprint Recognition.

Hematoxylin-eosin Staining (HE Staining): the HE staining method is the most basic and widely used technical method in histology, embryology, pathology teaching and scientific research. The hematoxylin staining solution is alkaline, so that chromatin in cell nuclei and nucleic acid in cytoplasm are bluish in color; eosin is an acid dye that can redden components of the cytoplasm and extracellular matrix.

Multi-Organ nuclear data set (Multi-Organ nuclear Segmentation, monseg): a published nuclear segmentation dataset consisting of a 40-fold magnified HE staining image downloaded from a Cancer Genome Atlas (TCGA) archive was obtained by accurate labeling of histopathological images of different tumor organs of multiple patients in multiple hospitals.

Gray image coloring: a method for changing black and white image into color image by using neural network. In the embodiment of the present application, particularly, black and white Magnetic Resonance Imaging (MRI), X-ray images and Computed Tomography (CT) images are colored in the medical aspect, so that the images are converted into color images, the image characteristics can be more fully displayed, and a doctor can find a disease in time conveniently.

Histopathology plays a crucial role in the diagnosis, prognosis and treatment decision of cancer, where nuclear segmentation is a key step in which histopathological images can be automatically analyzed. In the related art, a method of performing threshold segmentation on cells is usually adopted for cell segmentation, and a series of preprocessing methods such as blurring and contrast enhancement are performed on a cell image through some morphological operations to assist in improving the quality of the cell image, but the obtained cell nucleus segmentation result is often greatly different from the true form of the cell nucleus, so that the precision of threshold segmentation is poor. In the training of nucleus segmentation of the cell image by using deep learning, the probability of nucleus and non-nucleus areas is obtained by performing semantic segmentation on the cell image, and the precision of nucleus segmentation is greatly improved.

At present, most of the commonly adopted nucleus segmentation methods based on deep learning need to complete training by means of pixel-level labels, the quality of the pixel-level labels directly determines the quality of a nucleus segmentation model, and pixel-level label labeling of nuclei only by means of manpower is difficult, so that a large amount of manpower and material resources are consumed. In some cell nucleus segmentation models based on weak labels, due to the incompleteness of the labels, only the weak labels obtained through point annotation or doodle annotation are trained, and the results obtained by the obtained training models in the actual cell nucleus segmentation have certain difference with the real form of the cell nucleus.

The embodiment of the application provides a cell image identification method, a first cell image and a second cell image in a sample data set are input into a cell nucleus identification model to be respectively subjected to segmentation prediction training and coloring prediction training, a segmentation prediction result corresponding to the first cell image with a label and a coloring prediction result corresponding to the second cell image without the label are obtained, a segmentation loss value and a coloring loss value which correspond to each other are calculated, the cell nucleus identification model is trained on the basis of the segmentation loss value and the coloring loss value, the model precision of the cell nucleus identification model is improved, and therefore the accuracy and the identification efficiency of cell nucleus identification are improved.

An embodiment of the present application provides a cell image identification method, schematically, fig. 1 is a flowchart of an overall implementation scheme provided in an embodiment of the present application, as shown in fig. 1.

First, a sample data set 100 is acquired, where the sample data set 100 includes a first cell image 101 and a second cell image 102, both the first cell image 101 and the second cell image 102 include a pathological image of HE staining, the first cell image 101 includes annotation information for annotating cell nucleus information corresponding to the pathological image of HE staining, and the second cell image 102 includes or does not include the annotation information.

The grayscale image 110 corresponding to the sample data set 100 is obtained by performing grayscale processing on the sample data set 100, the grayscale image 110 is input into the cell nucleus recognition model 120, and the intermediate features 121 corresponding to the grayscale image 110 are obtained by performing feature extraction on the grayscale image 110 in the cell nucleus recognition model 120. The cell nucleus recognition model 120 further comprises a segmentation submodel 122 and a coloring submodel 123, the intermediate features 121 are respectively input into the segmentation submodel 122 and the coloring submodel 123 to obtain segmentation prediction results 124 and coloring prediction results 125 corresponding to the intermediate features 121, loss values 130 are obtained according to the segmentation prediction results 124 and the coloring prediction results 125, segmentation loss values 131 are obtained according to the segmentation prediction results 124 and cell nucleus information of the first cell image, coloring loss values 132 are obtained according to the second cell image 102 and the coloring prediction results 125, the cell nucleus recognition model 120 is trained on the basis of the segmentation loss values 131 and the coloring loss values 132, and finally the target recognition model 140 is obtained.

Next, an implementation environment related to the embodiment of the present application is described, and please refer to fig. 2 schematically, in which a terminal 210 and a server 220 are related, and the terminal 210 and the server 220 are connected through a communication network 230.

The terminal 210 is used to transmit the histopathology image to the server 220. Illustratively, the terminal 210 is a terminal applied by a doctor, and the doctor performs auxiliary diagnosis through the cell nucleus recognition model in the process of diagnosing diseases through the histopathological image, so as to improve the accuracy of diagnosis; alternatively, the terminal 210 is a terminal applied by a user, such as: the patient himself, or the relatives of the patient, etc., the user sends the histopathology image to the server 220, thereby obtaining a reference diagnosis result; or, the terminal 210 is a terminal connected to the histopathology image scanning device, the histopathology image scanning device scans the histopathology image and transmits the histopathology image to the terminal 210, and the terminal 210 receives the histopathology image and forwards the histopathology image to the server 220 for auxiliary diagnosis.

After the server 220 trains the cell nucleus recognition model 221 in the manner shown in fig. 1 to obtain the trained target recognition model 222, the server receives the histopathology image uploaded by the terminal 210, and performs cell nucleus recognition on the histopathology image through the target recognition model 222 to obtain a cell nucleus recognition result. The server 220 feeds back the cell nucleus recognition result to the terminal 210.

The terminal may be a mobile phone, a tablet computer, a desktop computer, a portable notebook computer, and other terminal devices in various forms, which is not limited in this application.

It should be noted that the server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a Network service, cloud communication, a middleware service, a domain name service, a security service, a Content Delivery Network (CDN), a big data and artificial intelligence platform, and the like.

The Cloud technology (Cloud technology) is a hosting technology for unifying series resources such as hardware, software, network and the like in a wide area network or a local area network to realize calculation, storage, processing and sharing of data. The cloud technology is based on the general names of network technology, information technology, integration technology, management platform technology, application technology and the like applied in the cloud computing business model, can form a resource pool, is used as required, and is flexible and convenient. Cloud computing technology will become an important support. Background services of the technical network system require a large amount of computing and storage resources, such as video websites, picture-like websites and more web portals. With the high development and application of the internet industry, each article may have its own identification mark and needs to be transmitted to a background system for logic processing, data in different levels are processed separately, and various industrial data need strong system background support and can only be realized through cloud computing.

In some embodiments, the servers described above may also be implemented as nodes in a blockchain system. The Blockchain (Blockchain) is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism, an encryption algorithm and the like. The block chain, which is essentially a decentralized database, is a string of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, which is used for verifying the validity (anti-counterfeiting) of the information and generating a next block. The blockchain may include a blockchain underlying platform, a platform product services layer, and an application services layer.

In conjunction with the above noun introduction, the application scenario involved in the embodiment of the present application is illustrated:

first, a doctor performs a diagnosis-assisting scenario through a cell nucleus recognition model.

That is, the doctor sends the tissue pathology image to the server through the terminal, and the server carries out the cell nucleus discernment through the target identification model that trains well to the tissue pathology image, obtains the cell nucleus discernment result that corresponds with the tissue pathology image to show cell nucleus discernment result feedback to the terminal that the doctor used, thereby the doctor carries out the auxiliary diagnosis analysis to the tissue pathology image through the cell nucleus discernment result, and reachs final diagnosis result.

Second, the user makes a diagnosis through the target recognition model.

The method comprises the steps that a user (a patient or relatives and friends of the patient) sends a tissue pathology image to a server, the server comprises a pathology diagnosis result database, the server conducts cell nucleus recognition on the tissue pathology image through a trained target recognition model to obtain a cell nucleus recognition result corresponding to the tissue pathology image, the cell nucleus recognition result is compared with the pathology diagnosis result database in the server, a corresponding preliminary diagnosis result is obtained, the preliminary diagnosis result is fed back to a terminal applied by the user to be displayed, the user firstly conducts preliminary understanding on an abnormal life state according to the preliminary diagnosis result, and then a detailed diagnosis result is obtained through doctor diagnosis.

With reference to the above noun introduction and application scenario, the cell image recognition method provided in the present application is described, for example, the method is applied to a server, as shown in fig. 3, and the method includes:

step 301, a sample data set is obtained.

The sample data set comprises a first cell image and a second cell image, and the first cell image is marked with cell nucleus information. The second cell image is marked or not marked with cell nucleus information.

The first cell image is used for representing a type of image marked with cell nucleus information, namely, the sample data set comprises at least one first cell image; similarly, the second cell image is used to represent a type of image without being labeled with cell nucleus information, that is, the sample data set includes at least one second cell image.

In some embodiments, the obtaining of the sample data set comprises at least one of:

1. acquiring a sample data set (such as a MoNuSeg data set) from a cell nucleus image set disclosed in a medical event;

2. acquiring a sample data set from a cell nucleus image set used in public discussion in a medical lecture or a cell nucleus image set related to a public medical paper for reasonable reference;

3. obtaining a sample data set from a nuclear image which is disclosed in a medical forum and agrees to authorize the use;

4. the image of the cell nucleus granted the right to use is obtained from the histopathological image library of the hospital.

It should be noted that the above approach to acquire the sample data set is only an illustrative example, and the approach to acquire the sample data set is not particularly limited in the embodiment of the present application.

In some embodiments, the nuclear information is labeled based on medical knowledge and diagnostic experience after medical personnel observe the cell image. After the medical staff marks the cell nucleus information of the cell image, the cell nucleus information is sent to the server, and the server acquires the cell nucleus information corresponding to the cell image.

In an embodiment of the present application, the labeled cell nucleus information includes at least one of the following information: information of cell nucleus boundaries; or, cell nucleus distribution characteristic information; or information on the size of cell nuclei, etc., and is not limited herein.

In some embodiments, the first cellular image and the second cellular image may be histopathological images of the same organ or the same body part; alternatively, histopathological images of different organs or different body parts are also possible.

In some embodiments, during the acquisition of the pathological tissue image, a sample is first collected from the living body and smeared on a slide to obtain a smear, so that the smear is subjected to the acquisition of the pathological tissue image through an image reading device (such as an optical microscope, an electron microscope and the like). Taking an organism as an example of a human body, the collected pathological tissue sample comprises at least one of human body fluid natural cast-off cell tissues, mucous membrane cell tissues, human tissue cells obtained by fine needle puncture or ultrasonic guide puncture and the like.

Step 302, inputting the sample data set into the cell nucleus recognition model, and outputting to obtain the segmentation prediction result of the first cell image and the coloring prediction result of the second cell image.

In the embodiment of the present application, the cell nucleus identification model is to identify and analyze each pixel point of the cell image in the sample data set, and the identification function of the cell nucleus identification model may include at least one of the following forms:

1. the method is used for distinguishing the cell nucleus part from the non-cell nucleus part (such as cytoplasm, extracellular matrix and the like) in the cell image, namely judging whether each pixel unit in the currently input cell image is a cell nucleus or not; the pixel unit is a pixel point, or the pixel unit is a pixel array formed by a plurality of pixel points;

2. the cell nucleus distribution characteristic acquisition module is used for acquiring the distribution characteristic of cell nuclei of cell images in an input sample data set, namely the number of the cell nuclei displayed in a unit area image area;

3. for calculating the average size of the nuclei in the input cell image, i.e. the average area of a single nucleus within the image area.

It should be noted that the above description of the function of the cell nucleus recognition model is only an illustrative example, and the specific function of the cell nucleus recognition model is not limited in any way in the embodiments of the present application.

In the embodiment of the present application, the segmentation prediction result of the first cell image is a determination result about cell nucleus boundary information obtained by performing region segmentation on a cell nucleus in the first cell image, the coloring prediction result of the second cell image is a determination result about cell nucleus boundary information obtained by performing grayscale image coloring on a cell nucleus in the second cell image, a color image of the second cell image is obtained, and boundary information about the cell nucleus in the second cell image is obtained according to a coloring result corresponding to the color image.

Step 303, obtaining a segmentation loss value based on the cell nucleus information of the first cell image and the segmentation prediction result.

In some embodiments, the nuclear information of the first cell image and the segmentation loss value of the segmentation prediction result are obtained based on a cross entropy loss function.

Alternatively, the cross entropy loss function can refer to formula one:

the formula I is as follows: l is_ce＝-[y_L·logy_sL+(1-y_L)·log(1-y_sL)]

Wherein L is_ceTo divide the loss value, y_LNuclear information, y, being the first cell image_sLThe result is predicted for the segmentation of the first cell image. As can be seen from formula one, when the segmentation prediction result of the first cell image is closer to the information of the cell nucleus thereof, the smaller the segmentation loss value is, the more accurate the segmentation prediction result of the cell nucleus is.

In the embodiment of the present application, the segmentation loss value is used to describe an error between the segmentation prediction result of the first cell image and the corresponding cell nucleus information, such as: edge errors, size errors, distribution errors, etc.

A staining loss value is obtained based on the second cell image and the staining prediction result, step 304.

In some embodiments, the staining loss value is obtained based on a residual square of the second cell image and the staining prediction.

Alternatively, the formula for calculating the color loss value may refer to formula two:

the formula II is as follows: l is_MSE＝(y_cU-r_U)

Wherein L is_MSEIs the loss of coloration value, y_cURefers to the result of the prediction of staining of the second cell image, r_UIs the second cell image. As can be seen from equation two, when the square value of the residual error between the second cell image and its coloring prediction result is smaller, the smaller the coloring loss value is, the more accurate the coloring prediction result is.

In the embodiment of the present application, the staining loss value is used to describe the error between the staining prediction result of the second cell image and the corresponding cell image, such as: color shading error, color distribution error, color edge error, and the like.

And 305, training the cell nucleus recognition model based on the segmentation loss value and the coloring loss value to obtain a target recognition model.

The target recognition model is used for recognizing cell nucleuses in the cell images.

In some embodiments, a weighted result of the segmentation loss value and the shading loss value is determined to obtain a target loss value, that is, a product of the segmentation loss value and the first weighting parameter is determined to obtain a first weighted part; determining the product of the tinting loss value and the second weighting parameter to obtain a second weighted part; determining the sum of the first weighted part and the second weighted part as a target loss value, wherein the first weight parameter and the second weight parameter are preset parameters; and adjusting the model parameters of the cell nucleus identification model based on the target loss value.

Alternatively, the formula for calculating the target loss value may refer to formula three:

the formula III is as follows: l is alpha. L_ce+β·L_MSE

And when the numerical value of the weighting parameter is higher, the weighting of the loss value is higher, the influence on the target loss value is larger, and the cell nucleus identification model is influenced by the segmentation loss value and the coloring loss value simultaneously.

Adjusting model parameters of the cell nucleus identification model through the target loss value, and optionally adjusting and obtaining model parameters corresponding to the coloring prediction result; or adjusting and obtaining model parameters corresponding to the segmentation prediction result; alternatively, the model parameters corresponding to both the coloring prediction result and the segmentation prediction result are adjusted, which is not limited herein.

In some embodiments, the object recognition model is used for recognizing cell nuclei in the cell image, and the recognition content includes at least one of the following contents:

1. identifying cell nucleuses in the cell image, namely identifying image areas corresponding to the cell nucleuses in the cell image;

2. identifying the boundary of the cell nucleus in the cell image, namely acquiring the coordinate value of a pixel point corresponding to the cell nucleus in the cell image, connecting the coordinate values of the pixel points corresponding to the cell nucleus, and outlining to obtain the outline range of the cell nucleus;

3. and identifying the size of the cell nucleus in the cell image, namely acquiring the size area of the cell nucleus in the cell image.

It should be noted that the identification content of the cell nucleus in the cell image by the target identification model is only an illustrative example, and the specific content of the cell nucleus identification in the embodiment of the present application is not limited at all.

In summary, the embodiment of the present application provides a cell image recognition method, which includes inputting a first cell image and a second cell image in a sample data set into a cell nucleus recognition model, and performing segmentation prediction training and coloring prediction training respectively to obtain a segmentation prediction result corresponding to the first cell image with a label and a coloring prediction result corresponding to the second cell image without a label, calculating a segmentation loss value and a coloring loss value corresponding to each segmentation prediction result, and training the cell nucleus recognition model based on the segmentation loss value and the coloring loss value to improve model accuracy of the cell nucleus recognition model, thereby improving accuracy and recognition efficiency of cell nucleus recognition.

In an alternative embodiment, the cell nucleus recognition model includes a segmentation submodel and a coloring submodel, and further includes an encoder, when the sample data set is input into the cell nucleus recognition model, the segmentation submodel obtains a segmentation prediction result, and the coloring prediction result obtains a coloring prediction result through the coloring submodel, for example, referring to fig. 4, which shows a flowchart of a cell image recognition method provided by an exemplary embodiment of the present application, the method includes the following steps:

step 401, obtaining a sample data set.

The sample data set comprises a first cell image and a second cell image, and the first cell image is marked with cell nucleus information.

In some embodiments, the nuclear information is based on medical knowledge and diagnostic experience after medical personnel view the images of the cells. After the medical staff marks the cell nucleus information of the cell image, the cell nucleus information is sent to the server, and the server acquires the cell nucleus information.

And 402, downsampling the cell image in the sample data set through an encoder to obtain the intermediate features of the cell image.

In some embodiments, the cell nucleus recognition model is a known network model, such as a U-network (relational network for biological Image Segmentation), an FCN (full relational network for Semantic Segmentation), a DenseNet (strained Connected relational network), and the like, and the cell nucleus recognition model includes at least one two-dimensional network model, or at least one three-dimensional network model, which is not limited herein. In the embodiment of the application, the U-net network is adopted to train the cell nucleus recognition model.

The U-net network is a U-shaped structure network, and performs semantic segmentation using a full convolution network, in this embodiment, an encoder is present in the U-net network model-based cell nucleus identification model, and is configured to perform downsampling on a cell image in a sample data set, and optionally, the encoder is configured to perform at least two layers of downsampling on the cell image. In this embodiment, four layers of downsampling are performed in the downsampling process, after each downsampling process is completed, the number of feature channels of the input cell image is increased, so that the size of the feature image corresponding to the cell image is reduced, the feature extraction is performed on the input cell image, along with gradual deepening of each downsampling, the feature extraction is performed on the input cell image from shallow to deep, the extracted features include a cell nucleus distribution feature, a cell nucleus edge contour feature, a cell nucleus size area and the like, and therefore limitation is not made here. In the down-sampling process, a shallow structure (such as a first layer of down-sampling) in the down-sampling structure can extract some simple features, such as boundary features of cell nuclei, color features of cell nuclei and the like, of the input cell image, and a deep structure (such as a fourth layer of down-sampling) in the down-sampling structure can extract some abstract features of the input cell image, so as to finally obtain intermediate features corresponding to the input cell image.

The steps 4031 to 4032 are training procedures of the coloring submodel and the segmentation submodel, which are in a parallel relationship, that is, after the sample data set is input into the cell nucleus recognition model, the coloring submodel and the segmentation submodel in the cell nucleus recognition model perform coloring processing and segmentation processing on the cell image at the same time, so as to obtain a coloring prediction result and a segmentation prediction result corresponding to the sample data set.

Step 4031, the sample data set is input into the cell nucleus recognition model, and the sample data set is colored through the coloring sub-model, so that a coloring prediction result of the sample data set is obtained.

Optionally, the staining sub-model performs a staining process on the cell image in the sample data set, where the staining process includes at least one of the following processing modes:

1. coloring the cell nucleus in the input cell image, namely, the coloring prediction result of the output sample data set comprises the coloring result of the cell nucleus region;

2. performing coloring processing on regions except cell nuclei in the input cell image, namely, the coloring prediction result of the output sample data set comprises the coloring result of a non-cell nucleus region;

3. and performing different coloring processing on the cell nucleus region and the non-cell nucleus region in the input cell image, namely, including coloring results of different regions of the cell image in the output coloring prediction result of the sample data set.

It should be noted that the processing manner of the coloring treatment is only an illustrative example, and the specific processing manner of the coloring treatment is not limited in this embodiment.

In some embodiments, the intermediate features are color channel up-sampled by a coloring decoder in the coloring submodel to obtain coloring features; and performing coloring prediction on the coloring characteristics to obtain a coloring prediction result of the sample data set.

In this embodiment of the application, the shading sub-model includes a shading decoder, and the shading decoder is configured to decode an intermediate feature corresponding to a cell image output by the encoder, that is, the shading sub-model performs a shading process on a cell nucleus edge on the intermediate feature corresponding to the input cell image (the cell image input to the cell nucleus identification model is a grayscale image), so as to generate a color image corresponding to a color channel, where the shading decoder performs down-sampling on the intermediate feature four times, each down-sampling aims to restore a feature vector obtained in a previous-layer down-sampling process, finally obtains a shading feature map having the same size as the input cell image through four-layer down-sampling, and obtains a shading prediction result based on the shading feature map, where the shading prediction result is, illustratively, the feature map y is a feature map y with one three-channel (i.e., a color channel)_c∈Y_c＝[0,1]^N×N×3Wherein, y_cIs a characteristic diagram, Y_cFor colored prediction results, N is the channel.

Wherein, the three-channel characteristic map is a characteristic map for describing edge information of cell nuclei in the input cell image (namely, the cell nucleus area is a coloring part); or, a feature map for describing a non-cell nucleus region in the input cell image (i.e., the non-cell nucleus region is a colored portion); alternatively, the feature map of each tissue distribution region in the input cell image (i.e., different tissues are labeled with different colors in the cell image) is not limited herein.

Step 4032, the sample data set is segmented through the segmentation submodel, and a segmentation prediction result of the sample data set is obtained.

Illustratively, the segmentation submodel performs segmentation processing on the input cell image in a manner including at least one of:

1. the segmentation submodel segments the cell nucleus area in the input cell image, namely, segments the cell nucleus based on the edge of the cell nucleus to make the boundary of the cell nucleus clear and definite;

2. the segmentation submodel segments the non-cell nucleus region in the input cell image, namely, segments the non-cell nucleus region based on the distribution of the non-cell nucleus region in the cell image.

It should be noted that the above description of the division processing method is only an illustrative example, and the specific processing method of the division processing is not limited in any way in the embodiments of the present application.

In some embodiments, the nucleus edge features are obtained by up-sampling the nucleus identification channel for the intermediate features by a segmentation decoder in the segmentation submodel; and performing segmentation prediction on the cell nucleus edge characteristics to obtain a segmentation prediction result of the sample data set.

In this embodiment, the segmentation submodel includes a segmentation decoder, the segmentation decoder is configured to decode the intermediate features output by the encoder, and finally obtain a segmentation prediction result corresponding to the input cell image, that is, the segmentation decoder performs an upsampling process on the intermediate features corresponding to the input cell image, where the segmentation decoder includes four layers of upsampling, and it is noted that each layer of upsampling of the segmentation decoder corresponds to each layer of downsampling layer of the encoder, so in each layer of upsampling, the feature vector in the segmentation decoder is spliced with the feature vector with the same size in the corresponding encoder in the same layer to obtain the feature vector for next layer of upsampling, and after four layers of upsampling, a cell nucleus edge feature map with the same size as the input cell image is obtained, based on the segmentation feature map, and obtaining a segmentation prediction result of the sample data set. Wherein. Probability map y for dividing prediction result into two channels_s∈Y_s＝[0,1]^N×N×2(i.e. a probabilistic result describing whether the region is a nucleus or not,exemplified by a nuclear region of 1 and a non-nuclear region of 0), wherein y is_sIs a probability map, Y_sFor partitioning the prediction result, N is a channel.

Illustratively, a two-channel probability map may be used to describe the probability of being a nuclear region in the input image of the cell (i.e., the region is a nuclear region or the region does not belong to a nuclear region); alternatively, the probability of a non-cell nucleus region in the input cell image may be described, which is not limited herein.

Step 4041 obtains a prediction result of segmentation of the first cell image from the prediction results of segmentation of the sample data set.

The segmentation prediction result of the sample data set comprises the segmentation prediction result of the first cell image and the segmentation prediction result of the second cell image, and the segmentation prediction result of the first cell image is obtained.

Step 4042, a prediction of staining of the second cell image is obtained from the prediction of staining of the sample dataset.

The coloring prediction result of the sample data set comprises a coloring prediction result of the first cell image and a segmentation prediction result of the second cell image, and the coloring prediction result of the second cell image is obtained.

Step 405, a segmentation loss value is obtained based on the cell nucleus information of the first cell image and the segmentation prediction result.

The description of the segmentation loss value in step 405 is discussed in detail in step 303, and is not repeated here.

At step 406, a staining loss value is obtained based on the second cell image and the staining prediction result.

The description of the color loss value in step 406 is discussed in detail in step 304, and is not repeated here.

And step 407, training the cell nucleus recognition model based on the segmentation loss value and the coloration loss value to obtain a target recognition model.

In some embodiments, the segmentation loss value is used for training the segmentation sub-model, including adjusting model parameters in the segmentation sub-model, so as to improve the precision of the segmentation prediction result corresponding to the segmentation sub-model; the coloring loss value is used for training not only the chromaticness sub-model, but also the encoder, when the coloring loss value is used for training the chromaticness sub-model, model parameters in the chromaticness sub-model can be adjusted, and the accuracy of a coloring prediction result is improved; the encoder is trained through the coloring loss value, the accuracy of feature extraction of the cell image in the lower sampling process of the encoder can be improved, namely, the feature extraction in the encoder is trained, the accuracy of feature extraction in the encoder is improved, and more accurate image features in the input cell image can be extracted in the lower sampling process of the encoder.

In an alternative embodiment, the sample data set input into the cell nucleus recognition model is a grayscale image of a cell image, and therefore, the cell image needs to be grayscale-processed, please refer to fig. 5, which shows a flowchart of a cell image recognition method provided by an exemplary embodiment of the present application, and the method includes the following steps:

step 501, extracting target color features of cell images in the sample data set.

In some optional embodiments, the process of extracting the target color feature is to perform color channel decomposition on the cell image to obtain at least two color channels; a target color channel corresponding to the target color feature is determined from the at least two color channels.

In the embodiment of the present application, the cell image in the sample data set is an HE stained pathological tissue image, in which the alkaline stain hematoxylin stains the chromatin in the cell nucleus and the nuclear count in the cytoplasm to bluish, the acidic stain eosin stains the cytoplasm and the components in the extracellular matrix to red, that is, in the HE stained image, the cell nucleus region is bluish, the non-cell nucleus region is red, and the color feature of the hematoxylin component in the HE stained image is extracted as the target color feature.

Step 502, based on the target color feature in the cell image, performing gray processing on the cell image to obtain a gray image.

In some alternative embodiments, the grayscale processing includes preserving target color channels corresponding to the target color features based on the target color features in the cell image; and carrying out gray processing on the cell image based on the target color channel to obtain the gray image.

In this embodiment of the present application, the HE-stained pathological tissue image is a color image, that is, there is a color channel (RGB channel), and a staining separation technique is used to map a color space corresponding to the color channel to a hematoxylin component space, and optionally, refer to formula four:

the formula four is as follows:

the pseudo-inverse matrix of formula IV is used in the staining separation process, wherein the RGB value of hematoxylin corresponding to hematoxylin component is [0.644, 0.717, 0.267%]，HE_i，jIs the hematoxylin and eosin channel value, RGB, of pixel (i, j)_i，jThe red, green and blue channel values of pixel (i, j) and + represents the pseudo-inverse of the matrix.

That is, two color features, namely, a hematoxylin component color feature and an eosin component color feature, exist in the HE stain component, and after the staining separation, two corresponding color channels, namely an H channel (hematoxylin component channel) and an E channel (eosin component channel), are obtained, which respectively represent the respective staining intensities of the hematoxylin component and the eosin component. The color features of the hematoxylin component in the cell image are retained, so that a cell image corresponding to an H channel is obtained, wherein the color contrast of the cell nucleus is enhanced when the color features of the cell nucleus are intersected with the corresponding color features of the cell nucleus in the original cell image, and the color contrast of the non-cell nucleus area is reduced compared with the color features of the corresponding non-cell nucleus area in the original cell image.

And performing gray processing on the cell image based on the cell image corresponding to the H channel, wherein the specific mode of the gray processing comprises at least one of the following modes:

1. performing gray processing by using a component method, namely, taking the brightness of red, green and blue components in the cell image (color image at this time) corresponding to the H channel as the gray values of three gray images, selecting any one gray value, and acquiring the gray image based on the gray value;

2. performing gray processing by adopting a maximum value method, namely taking the maximum value of the brightness of the red, green and blue components in the cell image (the color image at this time) corresponding to the H channel as the gray value of a gray image, and acquiring the gray image based on the gray value;

3. performing gray processing by using an average value method, namely, taking the average value of the brightness of the red, green and blue components in the cell image (the color image at this time) corresponding to the H channel as the gray value of a gray image, and acquiring the gray image based on the gray value;

4. carrying out gray scale processing by adopting a weighted average method, carrying out weighted average on the cell images (color images at this time) corresponding to the H channel by using different weight values according to the importance and other indexes of the red, green and blue components in the cell images to obtain a gray scale value after weighted average, and acquiring the gray scale image based on the gray scale value.

It should be noted that the above description of the gray scale processing method is only an illustrative example, and the specific gray scale processing method in the embodiment of the present application is not limited at all.

Referring to fig. 6, which is a schematic diagram illustrating obtaining a gray scale image according to an embodiment of the present application, as shown in fig. 6, a cell image 601 obtained by HE staining extracts a hematoxylin component in the cell image through a staining separation technique, and performs gray scale processing on the cell image to obtain a gray scale image 602 of the HE stained cell image corresponding to the hematoxylin component, as can be seen from fig. 6, the contrast of a cell nucleus area of the gray scale image obtained by extracting the hematoxylin component is more obvious, that is, a boundary of the cell nucleus is clearer.

Step 503, a sample data set is obtained, wherein the sample data set comprises a first cell image and a second cell image.

Wherein the first cell image is marked with cell nucleus information.

The description of the sample data set in step 503 is described in detail in step 301, and is not described herein again.

Step 504, inputting the gray image into the cell nucleus recognition model, and outputting the segmentation prediction result of the first cell image and the coloring prediction result of the second cell image.

The description of the segmentation prediction result and the coloring prediction result in step 504 is described in detail in step 302, and is not described herein again.

And 505, obtaining a segmentation loss value based on the cell nucleus information of the first cell image and the segmentation prediction result.

The description of the segmentation loss value in step 505 is already described in detail in step 303, and is not repeated here.

A staining loss value is obtained based on the second cell image and the staining prediction result, step 506.

The description of the coloring loss value in step 506 is described in detail in step 304, and is not repeated here.

And 507, training the cell nucleus recognition model based on the segmentation loss value and the coloring loss value to obtain a target recognition model.

The description of the target recognition model in step 507 is already described in detail in step 305, and is not repeated here.

In summary, the embodiment of the present application provides a cell image recognition method, which includes inputting a first cell image and a second cell image in a sample data set into a cell nucleus recognition model, and performing segmentation prediction training and coloring prediction training respectively to obtain a segmentation prediction result corresponding to the first cell image with a label and a coloring prediction result corresponding to the second cell image without a label, calculating a segmentation loss value and a coloring loss value corresponding to each segmentation prediction result, and training the cell nucleus recognition model based on the segmentation loss value and the coloring loss value to improve model accuracy of the cell nucleus recognition model, thereby improving recognition rate and recognition efficiency of cell nucleus segmentation.

In an alternative embodiment, as shown in fig. 7, which shows a schematic diagram of a cell image recognition process provided in an exemplary embodiment of the present application, as shown in fig. 7, a sample data set 701 is input into a cell nucleus recognition model 702, where the sample data set 701 includes information Y including cell nucleus_L＝(0，1)^N×NThe first cell image of (2) corresponds to a grayscale image X_LGray scale image X corresponding to the second cell image_UIn the cell nucleus recognition model 702, an encoder performs downsampling on an input cell image to obtain an intermediate feature 703, the cell nucleus recognition model further comprises a segmentation submodel and a coloring submodel, a segmentation branch decoder in the segmentation submodel 704 performs upsampling on the intermediate feature 703 to obtain a segmentation prediction result 706, a coloring branch decoder in the coloring submodel 705 performs upsampling on the intermediate feature 703 to obtain a coloring prediction result 707, segmentation loss is obtained based on cell nucleus information of a first cell image and the segmentation prediction result 706, coloring loss is obtained based on a second cell image and the coloring prediction result 707, the cell nucleus recognition model 702 is trained based on the segmentation loss and the coloring loss, and finally the target recognition model is obtained.

When the target recognition model is tested, firstly, hematoxylin components are extracted from the HE stained cell image, the gray level processing is carried out on the hematoxylin components to obtain a corresponding gray level image, the gray level image is input into the target recognition model, the segmentation prediction result and the coloring prediction result of the gray level image are obtained through the target recognition model, the segmentation prediction result is adjusted based on the coloring prediction result, and the cell nucleus segmentation result corresponding to the final HE stained cell image is obtained.

Optionally, the manner of adjusting the segmentation prediction result based on the coloring prediction result includes at least one of the following manners:

1. covering the three-channel characteristic image corresponding to the coloring prediction result on the probability image corresponding to the segmentation prediction result as a color mask, and modifying the label in the probability image based on the color mask so as to obtain a final cell nucleus segmentation result;

2. comparing the three-channel feature map corresponding to the coloring prediction result with the probability map corresponding to the segmentation prediction result, and adjusting the probability map based on the cell nucleus edge features in the feature map so as to obtain a final next mint segmentation result;

3. and marking the label in the probability graph corresponding to the segmentation prediction result in the three-channel feature graph corresponding to the coloring prediction result, and acquiring the part of the label overlapped with the coloring feature as the finally obtained cell nucleus segmentation result.

It should be noted that the above-mentioned manner of adjusting the partition prediction result for the coloring prediction result is only an exemplary example, and the present application is not limited thereto.

According to the method and the device, supervised segmentation training can be effectively carried out by using a small amount of data of cell nucleus information (such as pixel-level segmentation labels), a large amount of unlabeled data is used for coloring training to assist in segmentation tasks, and relatively accurate cell nucleus identification in the histopathology image of HE staining is realized. The specific use flow is as follows: collecting a large number of HE-stained histopathology images at an off-line end, carrying out complete cell nucleus labeling on a few histopathology images, then carrying out hematoxylin component extraction on all histopathology images, and training a cell nucleus segmentation and staining model; at a server side, firstly, the hematoxylin component extraction is carried out on the input histopathology image to obtain a gray level image, then, a segmentation sub-model and a coloring sub-model are called, and a corresponding cell nucleus recognition result image is output.

The scheme of the application can also be applied to automatic analysis of histopathology images, and the characteristics of average size, density, arrangement and the like of cell nucleuses can be obtained through subsequent calculation of cell nucleus recognition results, so that clinical diagnosis and treatment of cancers of different types, cancer grading, patient risk grading and the like can be realized. In addition, the method can also be applied to other identification tasks with similar target colors in the same category and larger target color difference in different categories.

In summary, the embodiment of the present application provides a cell image recognition method, which includes inputting a first cell image and a second cell image in a sample data set into a cell nucleus recognition model, and performing segmentation prediction training and coloring prediction training respectively to obtain a segmentation prediction result corresponding to the first cell image with a label and a coloring prediction result corresponding to the second cell image without a label, calculating a segmentation loss value and a coloring loss value corresponding to each segmentation prediction result, and training the cell nucleus recognition model based on the segmentation loss value and the coloring loss value, so as to improve model accuracy of the cell nucleus recognition model and reduce labeling burden of a pathologist.

Fig. 8 is a schematic structural diagram of a cell image recognition apparatus according to an exemplary embodiment of the present application, where as shown in fig. 8, the apparatus includes:

a sample obtaining module 810, configured to obtain a sample data set, where the sample data set includes a first cell image and a second cell image, and the first cell image is marked with cell nucleus information;

an output module 820, configured to input the sample data set into a cell nucleus identification model, and output a segmentation prediction result of the first cell image and a coloring prediction result of the second cell image;

a loss obtaining module 830, configured to obtain a segmentation loss value based on the cell nucleus information of the first cell image and the segmentation prediction result;

the loss obtaining module 830 is further configured to obtain a staining loss value based on the second cell image and the staining prediction result;

the training module 840 is configured to train the cell nucleus recognition model based on the segmentation loss value and the staining loss value to obtain a target recognition model, where the target recognition model is used to recognize a cell nucleus in a cell image.

In an alternative embodiment, the training module 840 comprises:

a determining unit 841, configured to determine a weighting result of the segmentation loss value and the shading loss value to obtain a target loss value;

an adjusting unit 842, configured to adjust the model parameters of the cell nucleus identification model based on the target loss value.

In an alternative embodiment, the determining unit 841 is further configured to determine a product of the segmentation loss value and a first weighting parameter, to obtain a first weighted portion; determining a product of the shading loss value and a second weighting parameter to obtain a second weighted part; determining the sum of the first weighted part and the second weighted part as the target loss value, wherein the first weight parameter and the second weight parameter are preset parameters.

In an alternative embodiment, the cell nucleus recognition model comprises a segmentation sub-model and a staining sub-model;

the output module 820 includes:

the coloring unit 821 is configured to input the sample data set into the cell nucleus identification model, and perform coloring processing on the sample data set through the coloring sub model to obtain a coloring prediction result of the sample data set.

A dividing unit 822, configured to perform a dividing process on the sample data set through the sub-dividing model to obtain a prediction result of the sample data set.

An obtaining unit 823 is configured to obtain a segmentation prediction result of the first cell image from a segmentation prediction result of the sample data set, and obtain a coloring prediction result of the second cell image from a coloring prediction result of the sample data set.

In an optional embodiment, the cell nucleus recognition model further comprises an encoder;

the rendering unit 821 is further configured to perform downsampling on the cell image in the sample data set through the encoder to obtain an intermediate feature of the cell image.

The shading unit 821 is further configured to perform color channel upsampling on the intermediate feature through a shading decoder in the shading submodel to obtain a shading feature; and performing coloring prediction on the coloring characteristics to obtain a coloring prediction result of the sample data set.

In an optional embodiment, the segmentation unit 822 is further configured to perform cell nucleus identification channel upsampling on the intermediate features by a segmentation decoder in the segmentation sub-model to obtain cell nucleus edge features; and performing segmentation prediction on the cell nucleus edge characteristics to obtain a segmentation prediction result of the sample data set.

In an optional embodiment, the apparatus further comprises:

an extracting module 801, configured to extract a target color feature of a cell image in the sample data set;

a processing module 802, configured to perform gray processing on the cell image based on a target color feature in the cell image to obtain a gray image;

the output module 820 is further configured to input the grayscale image into the cell nucleus identification model.

In an optional embodiment, the extraction module 801 is further configured to perform color channel decomposition on the cell image to obtain at least two color channels; determining a target color channel of the at least two color channels corresponding to the target color feature.

The extraction module 801 is further configured to reserve a target color channel corresponding to a target color feature based on the target color feature in the cell image; and carrying out gray level processing on the cell image based on the target color channel to obtain the gray level image.

In an optional embodiment, the loss obtaining module 830 is further configured to obtain a segmentation loss value of the cell nucleus information of the first cell image and the segmentation prediction result based on a cross entropy loss function.

The loss obtaining module 830 is further configured to obtain the staining loss value based on a square of a residual between the second cell image and the staining prediction result.

In summary, the cell image recognition apparatus provided in this embodiment inputs the first cell image and the second cell image in the sample data set into the cell nucleus recognition model to perform segmentation prediction training and coloring prediction training, respectively, to obtain the segmentation prediction result corresponding to the first cell image with the label and the coloring prediction result corresponding to the second cell image without the label, and calculates the segmentation loss value and the coloring loss value corresponding to each, and trains the cell nucleus recognition model based on the segmentation loss value and the coloring loss value, thereby improving the model accuracy of the cell nucleus recognition model, and improving the recognition efficiency of the cell nucleus recognition.

It should be noted that: the cell image recognition device provided in the above embodiment is only exemplified by the division of the above functional modules, and in practical applications, the above functions may be distributed by different functional modules according to needs, that is, the internal structure of the device may be divided into different functional modules to complete all or part of the above described functions. In addition, the cell image recognition device provided by the above embodiment and the cell image recognition method embodiment belong to the same concept, and specific implementation processes thereof are described in the method embodiment and are not described herein again.

Fig. 10 shows a schematic structural diagram of a server provided in an exemplary embodiment of the present application. Specifically, the method comprises the following steps:

the server 1000 includes a Central Processing Unit (CPU) 1001, a system Memory 1004 including a Random Access Memory (RAM) 1002 and a Read Only Memory (ROM) 1003, and a system bus 1005 connecting the system Memory 1004 and the Central Processing Unit 1001. The server 1000 also includes a mass storage device 1006 for storing an operating system 1013, application programs 1014, and other program modules 1015.

The mass storage device 1006 is connected to the central processing unit 1001 through a mass storage controller (not shown) connected to the system bus 1005. The mass storage device 1006 and its associated computer-readable media provide non-volatile storage for the server 1000. That is, the mass storage device 1006 may include a computer-readable medium (not shown) such as a hard disk or Compact disk Read Only Memory (CD-ROM) drive.

Without loss of generality, computer readable media may comprise computer storage media and communication media. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes RAM, ROM, Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), flash Memory or other solid state Memory technology, CD-ROM, Digital Versatile Disks (DVD), or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage, or other magnetic storage devices. Of course, those skilled in the art will appreciate that computer storage media is not limited to the foregoing. The system memory 1004 and mass storage device 1006 described above may be collectively referred to as memory.

According to various embodiments of the present application, the server 1000 may also operate as a remote computer connected to a network through a network, such as the Internet. That is, the server 1000 may be connected to the network 1012 through a network interface unit 1011 connected to the system bus 1005, or the network interface unit 1011 may be used to connect to another type of network or a remote computer system (not shown).

The memory further includes one or more programs, and the one or more programs are stored in the memory and configured to be executed by the CPU.

Embodiments of the present application further provide a computer device, which includes a processor and a memory, where at least one instruction, at least one program, a set of codes, or a set of instructions is stored in the memory, and the at least one instruction, the at least one program, the set of codes, or the set of instructions is loaded and executed by the processor to implement the cell image recognition method provided by the above-mentioned method embodiments.

Embodiments of the present application further provide a computer-readable storage medium, on which at least one instruction, at least one program, a code set, or a set of instructions is stored, and the at least one instruction, the at least one program, the code set, or the set of instructions is loaded and executed by a processor to implement the cell image recognition method provided by the above-mentioned method embodiments.

Embodiments of the present application also provide a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to cause the computer device to execute the cell image recognition method described in any of the above embodiments.

Optionally, the computer-readable storage medium may include: a Read Only Memory (ROM), a Random Access Memory (RAM), a Solid State Drive (SSD), or an optical disc. The Random Access Memory may include a resistive Random Access Memory (ReRAM) and a Dynamic Random Access Memory (DRAM). The above-mentioned serial numbers of the embodiments of the present application are merely for description and do not represent the merits of the embodiments.

It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.

The above description is only exemplary of the present application and should not be taken as limiting, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A method of cell image recognition, the method comprising:

2. The method of claim 1, wherein training the cell nucleus recognition model based on the segmentation loss values and the staining loss values comprises:

determining a weighted result of the segmentation loss value and the coloring loss value to obtain a target loss value;

and adjusting model parameters of the cell nucleus identification model based on the target loss value.

3. The method of claim 2, wherein determining a weighted result of the segmentation loss value and the shading loss value to obtain a target loss value comprises:

determining the product of the segmentation loss value and a first weighting parameter to obtain a first weighted part;

determining a product of the shading loss value and a second weighting parameter to obtain a second weighted part;

determining the sum of the first weighted part and the second weighted part as the target loss value, wherein the first weight parameter and the second weight parameter are preset parameters.

4. The method of any one of claims 1 to 3, wherein the cell nucleus recognition model comprises a segmentation sub-model and a staining sub-model;

the inputting the sample data set into a cell nucleus recognition model, and outputting a segmentation prediction result of the first cell image and a coloring prediction result of the second cell image, including:

inputting the sample data set into the cell nucleus recognition model, and performing coloring processing on the sample data set through the coloring sub-model to obtain a coloring prediction result of the sample data set;

carrying out segmentation processing on the sample data set through the segmentation sub-model to obtain a segmentation prediction result of the sample data set;

and acquiring a segmentation prediction result of the first cell image from the segmentation prediction results of the sample data set, and acquiring a coloring prediction result of the second cell image from the coloring prediction results of the sample data set.

5. The method of claim 4, wherein the cell nucleus recognition model further comprises an encoder;

before the sample data set is colored by the coloring sub-model and a coloring prediction result of the sample data set is obtained, the method further comprises the following steps:

and downsampling the cell image in the sample data set through the encoder to obtain the intermediate features of the cell image.

6. The method of claim 5, wherein said rendering said sample data set by said render submodel to obtain rendered predictors for said sample data set comprises:

performing color channel up-sampling on the intermediate features through a coloring decoder in the coloring sub-model to obtain coloring features;

and performing coloring prediction on the coloring characteristics to obtain a coloring prediction result of the sample data set.

7. The method of claim 5, wherein the performing a segmentation process on the sample data set by the segmentation submodel to obtain a prediction result of the segmentation of the sample data set comprises:

performing cell nucleus identification channel up-sampling on the intermediate features through a segmentation decoder in the segmentation submodel to obtain cell nucleus edge features;

and performing segmentation prediction on the cell nucleus edge characteristics to obtain a segmentation prediction result of the sample data set.

8. The method of any one of claims 1 to 3, wherein said inputting said sample data set into a cell nucleus recognition model further comprises:

extracting target color features of cell images in the sample data set;

performing gray processing on the cell image based on the target color feature in the cell image to obtain a gray image;

the inputting the sample data set into a cell nucleus recognition model comprises:

and inputting the gray scale image into the cell nucleus identification model.

9. The method of claim 8, wherein said extracting target color features of images of cells in said sample data set comprises:

performing color channel decomposition on the cell image to obtain at least two color channels;

determining a target color channel of the at least two color channels corresponding to the target color feature.

10. The method of claim 9, wherein performing gray-scale processing on the cell image based on the target color feature in the cell image to obtain a gray-scale image comprises:

based on a target color feature in the cell image, reserving a target color channel corresponding to the target color feature;

and carrying out gray level processing on the cell image based on the target color channel to obtain the gray level image.

11. The method of any one of claims 1 to 3, wherein the deriving a segmentation loss value based on the cell nucleus information of the first cell image and the segmentation prediction result comprises:

and obtaining the cell nucleus information of the first cell image and the segmentation loss value of the segmentation prediction result based on a cross entropy loss function.

12. The method of any one of claims 1 to 3, wherein said deriving a staining loss value based on said second cellular image and said staining prediction comprises:

obtaining the staining loss value based on a residual square of the second cell image and the staining prediction.

13. A cell image recognition apparatus, characterized in that the apparatus comprises:

14. A computer device comprising a processor and a memory, the memory having stored therein at least one instruction, at least one program, a set of codes, or a set of instructions, which is loaded and executed by the processor to implement the method of cell image recognition according to any one of claims 1 to 12.

15. A computer readable storage medium having stored therein at least one instruction, at least one program, a set of codes, or a set of instructions, which is loaded and executed by a processor to implement the method of cell image recognition according to any one of claims 1 to 12.

16. A computer program product comprising a computer program or instructions which, when executed by a processor, carries out the method of cell image recognition according to any one of claims 1 to 12.