TW201832181A - Image Analysis Systems and Related Methods - Google Patents

Abstract

Embodiments disclosed herein are directed to systems and methods for determining a presence and an amount of an analyte in a biological sample. The systems and methods for determining the presence of an analyte utilize a plurality of images of a sample slide including multiple fields-of-view having multiple focal planes therein. The systems and methods utilize algorithms configured to balance the color and grayscale intensity of the plurality of images and based thereon determine if the plurality of images contain the analyte therein.

Description

Image analysis system and related methods

The present invention relates to an image analysis system and related methods.

Microscopy is used to diagnose a variety of diseases, blood disorders, and the like. Some microscopy techniques require specialized microscopes or other equipment to achieve adequate resolution for proper diagnosis.

Microscopy can be used to detect analytes (such as malaria) using smears such as thick blood smears. Typically, the microscope includes an oil immersion lens with a relatively shallow depth of field to achieve the resolution required to detect parasitic protozoa that cause malaria. The lens typically exhibits a depth of field of only a few microns (about 1 micron or less than 1 micron). Typically, the entire thickness of the smear is imaged to critically diagnose the condition indicated by the presence of the analyte. However, the thickness of the smear is greater than a few microns, which can cause diagnostic problems, depending on the focal plane of the image. To ensure that the entire smear is analyzed, the distance between the sample and the lens can be reduced or increased to capture multiple focal planes in each field of view (FoV) of the smear.

A typical microscope includes a conventional focusing system configured to increase or decrease the distance between the lens and the sample in micrometer displacement. However, such conventional focusing systems can be expensive and complicated, making conventional focusing systems unsuitable for the most common areas of malaria, such as in poor areas. Typical diagnostic measures include the use of a technician to scan a slide on a microscope to visually determine the presence or absence of an analyte. However, factors that limit the sensitivity and consistency of the human microscopist include interpersonal and human intrinsic variability, neglect, eye fatigue, sleepiness, and lack of training. Lack of training is especially important in the context of less resource allocation, where high-quality microscope operators may be in short supply compared to the burden of diseases such as malaria. Furthermore, the skilled person may not be able to determine or quantify a particularly low concentration of analyte (eg, hypoparasemia) in the sample slide.

Therefore, developers and users of microscopes continue to seek improvements in the microscopes and diagnostic techniques used to determine the presence of analytes.

Embodiments disclosed herein relate to systems and methods for diagnosing, identifying, and quantifying biological analytes in biological samples. In one embodiment, a system for determining the presence of an analyte in blood is disclosed. The system includes at least one memory storage medium configured to store a plurality of images of a sample slide. The plurality of images includes: a plurality of fields of view, each comprising a unique x and y coordinate of the sample slide; and a plurality of focal planes, each having a unique z coordinate of the sample slide. The system includes at least one processor operatively coupled to the at least one memory storage medium. The at least one processor is configured to determine a white balance transform and apply the white balance transform to each of the plurality of images to effectively generate a plurality of color corrected images. The at least one processor is configured to determine an adaptive grayscale transform and apply the adaptive grayscale transform to each of the plurality of images to provide adaptive gray for each of the plurality of images Degree intensity image. The at least one processor is configured to detect and identify one or more candidate objects in the plurality of color corrected images and adaptive grayscale intensity images. The at least one processor is configured to extract and score the one or more candidate objects based at least in part on one or more characteristics of the one or more candidate objects, filtering the one based at least in part on a score Or a plurality of candidate objects, and outputting one or more color corrected image patches and one or more adaptive grayscale intensity image patches for each filtered candidate object. The at least one processor is configured to extract one or more feature vectors from the color corrected image patch and the adaptive grayscale intensity image patch and output the one or more feature vectors. The at least one processor is configured to classify each feature vector as corresponding to an artifact or an analyte. The at least one processor is configured to determine whether the feature vector classified as an analyte is above or below a threshold level associated with a positive diagnosis.

In one embodiment, a method of determining the presence of an analyte in blood is disclosed. The method includes receiving a plurality of images of a sample slide. The plurality of images includes: a plurality of fields of view, each comprising a unique x and y coordinate of the sample slide; and a plurality of focal planes, each having a unique z coordinate of the sample slide. The method includes applying a white balance transform to each of the plurality of images to effectively generate a plurality of color corrected images. The method includes applying an adaptive grayscale transform to each of the plurality of images to provide an adaptive grayscale intensity image for each of the plurality of images. The method includes detecting and identifying one or more candidate objects in the plurality of color corrected images and the adaptive grayscale intensity image. The method includes filtering the one or more candidate objects based at least in part on a score based at least in part on one or more characteristics of the one or more candidate objects, and outputting one for each filtered candidate object Or multiple color corrected image patches and one or more adaptive grayscale intensity image patches. The method includes extracting one or more feature vectors from the color corrected image patch and the adaptive grayscale intensity image patch and outputting the one or more feature vectors. The method includes classifying each feature vector to correspond to an artifact or an analyte. The method includes determining whether the feature vector classified as an analyte is above or below a threshold level associated with a positive diagnosis.

In one embodiment, a system for determining the presence of a malaria parasite in blood is disclosed. The system includes a microscope configured to capture a plurality of images of a blood slide. Each of the plurality of images includes: a plurality of fields of view, each comprising a unique x and y coordinate of the blood slide; and a plurality of focal planes, each having a unique z coordinate of the blood slide. The system includes at least one memory storage medium configured to store a plurality of images of the blood slide. The system includes at least one processor operatively coupled to the at least one memory storage medium. The at least one processor is configured to determine a white balance transform and apply the white balance transform to each of the plurality of images to effectively generate a plurality of color corrected images. The at least one processor is configured to determine an adaptive grayscale transform and apply the adaptive grayscale transform to each of the plurality of images to provide adaptive gray for each of the plurality of images Degree intensity image. The at least one processor is configured to detect and identify one or more candidate objects in the plurality of color corrected images and adaptive grayscale intensity images. The at least one processor is configured to extract and score one or more characteristics of the one or more candidate objects, filtering the one or more candidate objects based at least in part on the score. The at least one processor is configured to extract color corrected image patches and adaptive grayscale intensity image patches for the one or more filtered candidate objects and output one or for each filtered candidate object Multiple feature vectors. The at least one processor is configured to classify each feature vector as an artifact or an analyte. The at least one processor is configured to determine whether the feature vector classified as an analyte is above or below a threshold level associated with a positive diagnosis.

In one embodiment, a system for determining the presence of an analyte in blood is disclosed. The system includes at least one memory storage medium configured to store a plurality of images of a sample slide, the plurality of images comprising: a plurality of fields of view, each field of view including a unique one of the sample slides x and y coordinates; and a plurality of focal planes, each focal plane having a unique z coordinate of the sample slide. The system includes at least one processor operatively coupled to the at least one memory storage medium. The at least one processor of the system is configured to determine a white balance transform and apply the white balance transform to each of the plurality of images to effectively generate a plurality of color corrected images. The at least one processor of the system is configured to: determine an adaptive grayscale transform and apply the adaptive grayscale transform to each of the plurality of images to target each of the plurality of images One provides an adaptive grayscale intensity image. The at least one processor of the system is configured to detect and identify one or more candidate objects in the color corrected image and the adaptive grayscale intensity image. The at least one processor of the system is configured to perform an adaptive thresholding operation on the adaptive grayscale intensity image and output one or more candidate objects based thereon. The at least one processor of the system is configured to cluster the one or more detected candidate objects into a cluster comprising one or more adjacent candidate objects/clusters, and the association indicates one or more phases A cluster of neighboring candidate objects is a cluster of detected candidate objects of a single candidate object and outputs a location of the cluster of one or more adjacent candidate objects, the location including the one or more adjacent candidate objects One or more images are patched. The at least one processor of the system is configured to locate the focal plane with each of the individual candidate objects having the best focus. The at least one processor of the system is configured to determine an attribute of each of the focal planes in the focal plane that has the best focus for each individual candidate object. The at least one processor of the system is configured to filter each individual candidate object based at least in part on one or more determined attributes. The at least one processor of the system is configured to extract and output one or more image patches, each image patch comprising at least one filtered single candidate object of the one or more candidate objects.

In one embodiment, a method for determining the presence of an analyte in blood is disclosed. The method includes receiving a plurality of images of a sample slide, the plurality of images comprising: a plurality of fields of view, each field of view including a unique x and y coordinate of the sample slide; and a plurality of focal planes, Each focal plane has a unique z coordinate of the sample slide. The method includes applying a white balance transform to each of the plurality of images to effectively generate a plurality of color corrected images. The method includes applying an adaptive grayscale transform to each of the plurality of images to provide an adaptive grayscale intensity image for each of the plurality of images. The method includes detecting and identifying one or more candidate objects in the plurality of color corrected images and the adaptive grayscale intensity image. The detecting and identifying one or more candidate objects of the method includes performing an adaptive thresholding operation on the adaptive grayscale intensity image and outputting one or more candidate objects based thereon. The detecting and identifying one or more candidate objects of the method includes clustering the one or more detected candidate objects into a cluster comprising one or more candidate objects/clusters, and the association indicating one or more phases An adjacent candidate object is a cluster of detected candidate objects of a single candidate object and outputs a location of a cluster of the one or more adjacent candidate objects, the location comprising a cluster comprising the one or more adjacent candidate objects One or more images are patched. The detecting and identifying one or more candidate objects of the method includes identifying the focal plane with each of the individual candidate objects having the best focus. The detecting and identifying one or more candidate objects of the method includes determining attributes of each of the individual candidate objects in the focal plane having the best focus for each individual candidate object. The detecting and identifying the one or more candidate objects of the method includes filtering each individual candidate object based at least in part on the one or more determined attributes. The detecting and identifying the one or more candidate objects of the method comprises: extracting and outputting one or more image patches, each image patch comprising at least one filtered single candidate of the one or more candidate objects object.

Features from any of the disclosed embodiments can be used in combination with each other without limitation. Other features and advantages of the present invention will become apparent from the following detailed description and drawings.

The above summary is illustrative only and is not intended to be in any way limiting. In addition to the above described aspects, embodiments and features, further aspects, embodiments, and features of the present invention will become apparent from the Detailed Description.

[0001] All of the subject matter of the priority application (multiple applications) is incorporated herein by reference to the extent that the subject matter is not inconsistent with the present application.

Embodiments disclosed herein relate to image analysis systems and methods of using the image analysis systems. The images disclosed herein include images in any computer readable format such as png, jpeg, gif, tiff, bmp, or any other suitable file type. The image analysis system and related methods herein are capable of solving and analyzing the entire vertical thickness of a sample smear (eg, a thick blood smear) on a slide (eg, substantially parallel to the optical axis or z-axis on the microscope) And an image on the lateral portion (for example, based on the dimensions of the x-axis and the y-axis). The systems and methods herein can identify objects that are actually the same object at different focal planes (z-levels), but these objects appear different due to different depths of focus or have different x-y coordinates due to camera shake. As explained in more detail below, the blood smear can use multiple fields of view (FoV) that define the discrete lateral (sub)parts of the blood smear and dispersion (vertically stacked) that is defined over the entire thickness of the blood smear. Analysis of multiple focal planes of the plane. The image analysis system herein can accurately identify the presence of parasites (multiple parasites) or other analytes in a sample, and in some embodiments their type or stage. The systems and methods disclosed herein can provide one or more of automatic diagnosis and quantification of one or more analytes in a biological sample at an operational level equivalent to or superior to a trained microscope operator. As used herein, the term "analyte" is not intended to be limited to a particular chemical species, but is intended to extend at least to parasites (eg, malaria, etc.), blood components, or other items in the sample being analyzed. One or more. The systems and methods disclosed herein provide a comprehensive machine learning framework for detecting analytes using computer vision and machine learning techniques including support vector machine (SVM) and convolutional neural network (CNN).

The image analysis system and related methods herein include a plurality of modules (eg, programs or algorithms) configured to perform different functions even at low concentrations (eg, low parasitemia) The presence of an infection or condition in the sample is also accurately determined without manual observation. The plurality of modules may include a pre-processing module, a candidate detection module, a feature extraction module, a classification module, and a diagnosis module. Although described herein as a separate "module" for clarity, each of the "modules" may be one or more algorithms or based on the one or more algorithms stored in at least one storage device. And a machine readable program executable by a processor operatively coupled thereto. The plurality of modules may include discrete programming modules and sub-modules stored in a memory storage medium of at least one controller (eg, a computer) or stored in one or more processors thereof, in the one or more Each of the processors has a program configured to perform the functions of the associated module. The terms "module" and "sub-module" are used to distinguish between elements and sub-elements of an algorithm or system, and may be used interchangeably depending on the context. For example, a sub-module may also be referred to as a module, such as when a sub-module is discussed without involving the parent module of the sub-module.

Typically, each module is configured to cause a controller or processor to perform the functions described below. While a high level overview of the functionality is generally described below for ease of understanding, the specific aspects of each module are disclosed in greater detail below.

The image pre-processing module can generate an adaptive white-balanced color image and an adaptive gray-scale intensity image of the plurality of images, including a plurality of fields of view of the sample slide and a plurality of focal planes (eg, multiple focal lengths) Each of the planes is substantially perpendicular to the optical axis). The candidate detection module can identify one or more candidate objects based on one or more attributes (eg, intensity, color type, focus level, or other attributes) of the candidate object in the image, identify and based on the same attribute Excluding one or more artifacts (eg, non-analyte objects such as non-parasitic objects (including white blood cells) in the sample), and color-corrected image patching and adaptation with each candidate object can be extracted Grayscale intensity image patching. The feature extraction module can identify and output one or more data sets of one or more candidate objects in a particular image (eg, a particular FoV and one or more vectors of its focal plane). The feature extraction module can establish the recognition based on a manual function that includes one or more of the best focus scores of the candidate objects, and a focus score across the focal plane in the field of view. Standard deviation (or other discrete measurements), or redshift score. The feature extraction module can additionally or alternatively identify and output one or more images based on one or more automated features, including computer samples of positive samples, negative samples, or both. Attributes (for example, one or more vectors learned through convolutional neural networks). The classification module can be configured to determine whether the extracted feature has a high probability score based at least in part on weights learned from known positive and negative samples (eg, including the presence, type, stage, or type of parasite) (indicating The analyte or artifact is present) and an estimate of the concentration of the analyte (eg, parasite) in the sample is determined.

The following mathematical notation will be used in the equations used throughout the algorithms disclosed in this disclosure. Italic lowercase or uppercase letters represent a value (for example, ). Bold italic lowercase letters indicate column vectors (for example, ). Bold italic uppercase letters indicate a matrix (for example, ). The superscript T indicates matrix transposition (for example, ). Image plane coordinates are called And the coordinates in the vertical direction, that is, the coordinates parallel to the optical axis are called .

The image analysis system of the present disclosure receives as input an image of a biological sample taken from a high resolution image capture device (eg, a high resolution microscope) and produces one or more analytes as an output ( For example, the presence, type, and count of diagnostic information about the state of the biological sample, such as a pathogen such as a parasite or a naturally occurring component such as a blood component.

In one embodiment, the biological sample comprises a microscope slide of the sample (eg, a blood smear), and the image analysis system herein analyzes one or more captured sample slide images to determine the presence or absence of the sample therein. One or more analytes (eg, malaria parasites). The image analysis system herein analyzes sample slides for identifying the presence, count, and type of analyte. While the systems and methods disclosed herein are not limited to use with blood smears, blood smears will be used throughout the disclosure as examples to illustrate the concepts, and it should be understood that the present disclosure is applicable to other biological samples without limitation.

In one embodiment, the blood smear is stained with a Giemsa stain prior to pathological diagnosis of one or more of the analytes (eg, malaria). Giemsa dye is a combination of methylene blue, eosin Y and azure B; it dyes red blood cells (red blood cells, hereinafter referred to as "RBC") into pink, and white blood cell nuclei (white blood cells, hereinafter referred to as "WBC") ) dyed into deep magenta. The nucleus of the malaria parasite will also be dyed red, but not as dark as the white blood cell nucleus. The cytoplasm of the malaria parasite is lightly stained to medium blue. The systems and methods disclosed herein are not limited to detecting malaria, but malaria will be used throughout the disclosure as an example of an embodiment to illustrate the concept, and it should be understood that the present disclosure is also applicable to other analytes without limitation. In addition, other dyes and staining methods that complement the analytes detected can be used. For example, suitable dyes can include Field Dye, Jaswant Singh Bhattacharya (JSB) Dye, Leishman Dye, and the like.

In one embodiment, the systems and methods herein can be used to detect and quantify the amount of analyte in a sample based, at least in part, on the shape, color, or size of the analyte. In some embodiments, the analyte can have more than one conformation or appearance. The systems and methods herein can be configured to detect or quantify one or more conformations, types, or categories of analytes. As an example of an embodiment, the human malaria parasite belongs to five different Plasmodium species: Plasmodium falciparum, Plasmodium vivax, Plasmodium ovale, Plasmodium vivax and Plasmodium berghei. Individuals of each of these categories undergo a series of complex stages throughout their life cycle. At each stage, the parasites exhibit different physical appearances, and the systems and methods herein can detect and identify parasites from each of five different species.

Figure 1 is a schematic diagram of the malaria life cycle practice of the National Institute of Allergy and Infectious Diseases. The right side of Figure 1 shows the stages that occur in the life cycle of malaria parasites in mosquitoes. The left side of the figure shows the various stages in the infected human body. In mosquitoes, malaria parasites begin with gametophytes (both female and male). The gametophyte is propagated to form a gamete that eventually develops and multiplies into sporozoites. The sporozoites migrate to the salivary glands of the mosquito. When mosquitoes bite humans, sporozoites enter the bloodstream and travel to the liver and infect liver cells (hepatocytes, liver cells). The sporozoites multiply into merozoites, rupturing the liver cells of the infected host and returning them to the bloodstream. Individual merozoites infect red blood cells and develop into a ring form, which is an immature trophozoite. The ring form develops into a relatively mature trophozoite and eventually becomes a schizon. Each schizont will divide into multiple merozoites, each of which seeks its own red blood cells for infection. In this way, the asexual part of the reproductive cycle repeats itself, as indicated by the human red blood cell cycle shown at the upper left of Figure 1. Some merozoites can develop into gametophytes, which, if infused by mosquitoes that are biting, will continue the life cycle of the parasite.

Different species have different life cycle durations and have a unique physical appearance (even under the same life cycle stage). Since treatment regimens vary among malaria species, it is important to distinguish them when performing histopathological diagnosis of malaria. The systems and methods of the present disclosure can be automatically distinguished between different malaria stages or species (or analytes).

2A and 2B are schematic views of a circular parasite. Cylindrical parasites are commonly found in peripheral blood. The physical appearance of the ring parasite varies greatly. The circular parasite usually has one (Fig. 2A) or two (Fig. 2B) chromatin spots 201 containing the parasite's nucleoplasm. The chromatin spot 201 was dyed in red under the conditions of the Giemsa dye as described above. The circular parasite also employs a slender cytoplasm 202 that is lightly stained to a medium blue color under the conditions of Giemsa staining as described above. Chroma dots 201 are typically about 1 micron in diameter and the entire ring is about 3 microns in diameter. The systems and methods herein can be used to identify or quantify analytes of about 200 nanometers or greater, such as from about 200 nanometers to about 100 microns, from about 500 nanometers to about 10 microns, from about 1 micron to about 5 microns, or less than about 50 microns. Analyte. In one embodiment, in order to obtain a high quality image of such a small object, a microscope with a high resolution lens is used. For example, a suitable high resolution microscope can include an objective lens having a numerical aperture greater than or equal to about 1.2 oil immersion 100 times. The microscope can be equipped with a digital image capture device such as a camera. The depth-of-field of the high-magnification optical system herein can be about 0.35 microns or less (eg, 0.3 microns, 0.2 microns, 0.1 microns, or a range between any of the foregoing values), while blood coating The film can be several times thicker than this. In various embodiments, multiple focal planes are captured for each FoV to capture a focused image of the parasite that can be positioned vertically at any location between the bottom and top of the blood smear. The number of focal planes captured by each FoV is specified as .

2C is a schematic diagram of a plurality of images 301, in accordance with an embodiment. The plurality of images 301 are arranged in a plurality of rows and in a plurality of columns. The rows and columns of the image collectively define a blood smear or other sample slide. For example, a blood smear can be captured substantially entirely by multiple images arranged in a collection of y rows, x columns, and z focal planes. The number of captured FoVs is specified as . The lateral extent of each FOV (eg, the x and y ranges) is defined by one or more of the magnification of the lens or the image sensor size of the imaging device. A given size of blood smear may require multiple FoVs to provide suitable image resolution for the purposes of this document. Each FoV can have multiple focal plane images corresponding thereto. For example, a FoV corresponding to the x, y coordinates in a plurality of images may include a z-focal plane image corresponding to the number of focal planes at which the image is captured at the corresponding FoV. That is, a particular image corresponding to FoV can be specified by a unique x and y coordinate, which can be specified by a unique z coordinate in FoV. Each image (eg, a particular FoV and focal plane) may contain multiple image patches therein. Image patching is a lateral segmentation of the FoV (at a particular focal plane) with a lateral segment of the FoV having one or more candidate objects therein and defining smaller segments of the blood slide. The systems and methods disclosed herein utilize multiple images including nxy FoV and nz focal planes to identify and quantify analytes in a sample.

In some embodiments, the size of the micro-captured FoV herein can be on the order of 10,000 square microns or more (eg, 10,000 square microns to about 20,000 square microns). In some embodiments, the size of the Fov captured by the microscope herein can be less than about 10,000 square microns (eg, from 1000 square microns to about 10,000 square microns). A FoV of about 10,000 square microns corresponds to about 3 x 10 ^-4 microliters of blood in a thick blood smear sample. The number of parasites in a FoV with 100 parasite protozoa/microliter blood smears in malaria patients will be Poisson-distributed, with an average of 3 x ^10-2 parasites per FoV.

In some embodiments, 300 FoVs or more can be captured to achieve sufficient statistics for reliable detection and enumeration of parasites under low parasitemia. For example, about 300 to 2000 FoVs can be captured or about 500 to 1000 FoVs can be captured. In some embodiments, 300 FoV or less can be captured to achieve sufficient statistics for reliable detection and enumeration of parasites under low parasitemia. For example, about 10 to 300 FoVs can be captured or about 50 to 200 FoVs can be captured. The lowest detectable protozoan level for a particular analyte is called the limit of detection (LoD). In general, the more the number of FoVs captured, the lower the LoD.

The preceding paragraphs provide an overview of the features of the images as input to the image analysis system disclosed herein.

FIG. 3A is a schematic illustration of multiple modules of a system 300 for automatically detecting and quantifying one or more analytes in a sample, in accordance with one embodiment. The modules may be algorithms that are collectively configured to determine the presence of a parasite in a sample or a controller that includes the algorithm (eg, electronically stored therein). 3B and 3C are schematic illustrations of a plurality of images 301 and input images 311 of the modules, respectively, input into the modules of system 300.

Referring to FIG. 3A, one or more modules include an image pre-processing module 310, a candidate object detection module 320, a feature extraction module 330, an object classifier module 340, and a diagnostic module 350. As indicated above, the modules and sub-modules herein may refer to one that is stored in at least one memory storage device (eg, a computer hard drive) and that is executable by at least one processor operatively coupled thereto. Or multiple algorithms and machine readable programs. The modules and sub-modules described herein may similarly refer to methods of automatically detecting and quantifying one or more analytes in a sample.

Input 301 input into the system can include one or more FoV images of the sample slide. presence FoV, each of which includes Each focal plane, including the red, green, and blue channel images (as shown in Figure 3B).

In the embodiment illustrated in FIG. 3A, system 300 can receive as input a plurality of images 301 in image pre-processing module 310. The plurality of images 301 can include multiple FoVs and multiple focal planes for each FoV. The image pre-processing module 310 can output a plurality of output images 311 including a color corrected image and an adaptive grayscale intensity image. A plurality of color corrected images and adaptive grayscale intensity images may be received as input at candidate object detection module 320 and feature extraction module 330. The candidate object detection module 320 receives the color corrected image and the adaptive grayscale intensity image, and outputs the candidate object and all of the candidates. The color corrected R, G, B image of the focal plane is patched 321 . The feature extraction module 330 receives the color corrected R, G, B image patches 321 (based on a plurality of color corrected images and adaptive grayscale intensity images in the output image 311) as inputs. The feature extraction module 330 extracts and outputs the feature vector 331 of the candidate object in the color corrected R, G, B image patch 321 and the adaptive grayscale intensity image patch. A feature vector is a multidimensional vector that represents the numerical characteristics of an object. In other aspects, the feature vector is a vector that represents one or more variables that include one or more characteristics (eg, color, size, location, etc.) that describe the object. The object classifier 340 receives the feature vector 331 as input and output classification object data 341 corresponding to each candidate object category as an analyte or artifact. At the diagnostic module 350, the classified item data is received as an input, and the diagnostic module 350 determines and provides a diagnosis of the sample. The diagnostic module can output the relative concentrations of the diagnostic 351 and the analyte (eg, parasite protozoa). Each of the image analysis system modules 310, 320, 330, 340, and 350 is described in detail below.

A. Image preprocessing module

Microscope slides that are tissue-stained (eg, with Giemsa dye) typically exhibit a color change between the inside of the slide (within the slide) and between the slides from different samples (between slides). This color change can be caused by the difference in the pH of the dye and the duration of the dyeing process. Uncorrected, these color differences may reduce the operational performance of an image analysis system whose purpose is to detect and classify objects of interest in the image.

White balance techniques can be used to normalize colors in an image. The white balance technique can calculate a linear color transform as follows. The average color of the brightest pixels in the image is calculated and expressed as a red-green-blue column vector: Where R, G, and B are pixel values of the red channel, the green channel, and the blue channel, respectively. The sum is the brightest pixel and N is the number of pixels included in the sum.

The diagonal transformation matrix A is calculated as follows:

Pixel Color corrected value It is obtained by the linear transformation defined by Equation 1: among them, It is selected such that the color corrected pixel values are within the range [0, k]; k is typically chosen to be 1 or 255. From this point on, in the present disclosure, to simplify the annotation, the prime number And R ' , G ', B ' will be discarded Substitution with R , G , B means that the color corrected value is predetermined.

As noted above, in some embodiments, at least 300 orders of magnitude of FoV can be captured for each blood sample. Not all of these images will contain white parts, so white balance of each individual FoV image will result in color distortion. To solve this problem, it is feasible to determine white balance transformation by separately acquiring one or more images on the white portion of the microscope slide. However, this introduces additional scanning steps to the workflow.

The systems and methods herein avoid color distortion introduced by forcing each FoV to white balance according to its brightest pixels. The systems and methods herein are also free of the need to additionally scan clear areas in the slide as an additional step.

The image pre-processing module 310 of FIG. 3A can be configured to determine a white balance transformation of the sample by accumulating the brightest pixels across the plurality of FoVs. FIG. 4 shows a block diagram of an image pre-processing module 310. In one embodiment, the subset 401 of the total set of input FoV images 301 is randomly selected at the sub-module 400. The number of FoVs in a subset of the FoV image 401 is large enough to include a clear region in the set of pixels close to one. Using the weighted sum of the pixel values of the color corrected red, green, and blue channels defined by the equation in Equation 2, a subset of the FoV image 401 is converted to standard by the sub-module 410. Grayscale intensity image 411:

among them, The standard grayscale intensity value for the pixel.

The grayscale intensity value is used; the sampled red, green, and blue values of the brightest pixel 451 in the subset 411 are selected by the sub-module 450 and stored in the data memory (eg, the storage medium of the memory) )in. Sub-module 460 calculates white balance transform 461 based on the stored values of red, green, and blue from each of the samples of brightest pixel 451. The white balance transform parameter 461 can be saved in the data store. Sub-module 470 applies a white balance transform to input image 301 to produce a color corrected FoV image 471. The white balance transform algorithm and its associated parameters are described in detail herein.

The image preprocessing module allows for a general affine matrix of the transformation matrix in Equation 1.

In one embodiment, the affine matrix A is a rotation matrix (also denoted A ).

As described above, the vector is the average color of the samples of the brightest pixel 451. These pixels are shown in the red, green, and blue pixel value spaces in FIG. White is made from white vector Said. White balance transform by making vectors Around a vector perpendicular to white And average color vector The axis vector n of both is rotated to the rotation of the vector to define. Figure 5 is a vector in the color value space of the red, green, and blue axes. , Schematic diagram of the relationship between and n . The rotation axis vector n can be calculated by using a cross product system:

The rotation matrix A can be calculated by using the system of Equation 3 below:

In Equation 3, Is a unit vector in the direction of the rotation axis n , where Represents the standard L ² norm. vector with The cosine of the angle can be passed through the dot product To calculate, where , .

Referring again to FIG. 3A, the image pre-processing module 310 can compensate for the color variations outlined above in the input image 301 and output a plurality of output maps including the color-corrected FoV image and the adaptive gray-scale image. Like 311, each one includes one or more focal planes therein. The next stage of the processing pipeline of image analysis system 300 is candidate object detection module 320. Candidate object detection module 320 is configured to look for image locations that are likely to be analytes (eg, malaria parasites). To find the location of such potential analytes, the candidate object detection module 320 can use a plurality of adaptive grayscale transformed images of the plurality of output images 311 and a plurality of color corrected (eg, white balance transformed) )image. The plurality of output images 311 includes a plurality of adaptive grayscale transformed images, and the plurality of color corrected images are determined and output by the image preprocessing module 310.

Candidate parasite nuclei can be detected by applying a dark threshold to a standard grayscale intensity image, which is calculated by the weighted sum shown in Equation 2. This weighted sum can be viewed as a projection of the red, green and blue pixel spaces previously introduced and shown in FIG. The projection is in the direction of the vector defined by Equation 4:

Use the red, green, and blue values of the pixel as column vectors , the grayscale projection in Equation 2 can be written as . To detect candidate parasite nuclei, a dark threshold can be applied to the standard grayscale intensity image intensity of each pixel. Then, one or more regions, colors, and shape screening programs (eg, candidate object clusters) can be applied to the blobs detected by applying a dark threshold. The standard dark threshold is a screening program that acts, at least in part, on the difference between the grayscale intensity of each pixel of the candidate object and the grayscale intensity of the other non-analyte pixels present in the background or sample. Thus, a standard dark threshold can be used to filter (select or delete) pixels that do not exceed (eg, above) a dark threshold.

The techniques mentioned above have limited properties for detecting the sensitivity and specificity of candidate parasite nuclei. Although the overall trend of the parasite nuclei is dark and the background is light, the overlap between the parasite nuclei and the background grayscale pixel values is large. FIG. 6A shows a grayscale intensity bar graph of background pixel 601, white blood cell nuclear pixel 602, and parasitic cell nuclear pixel 603. The overlap between the parasite nuclei and the background grayscale intensity values is shown as cross-hatched region 604 in Figure 6A.

Minimizing the overlap between parasite nuclei and background grayscale intensity values enhances the sensitivity and specificity of the detection algorithms herein. The systems and methods herein determine (eg, acquire) and apply adaptive grayscale projection vectors instead of the standard grayscale projection vectors defined in Equation 4. . These determinations can be done using machine learning techniques. Such an application can provide a greater separation of grayscale intensity values corresponding to white blood cell nuclear pixels and analyte (eg, malaria parasite) pixels from grayscale intensity values corresponding to background pixels.

The minimization of overlap disclosed herein utilizes the presence of blood components that are readily detected in standard grayscale intensity images and that are similar to staining of parasite cell nucleoplasm.

Under the Giemsa dye, as described above, the nucleus of the ring parasite is dyed in red. Specifically, the nucleoplasm is generally darker than the surrounding background material, including red blood cell (RBC) substances that have been cleaved by the action of water used in the Giemsa staining process, and other blood such as platelets. ingredient. This background material can stain a wide range of colors from light pink to medium blue. In addition to parasites (if blood is so infected), lysed RBCs and platelets; WBC is ubiquitous in blood smears. As described above, the WBC nuclei are stained in deep magenta under Giemsa, the same color as the parasite nucleus, but the stained WBC nuclei are in most cases darker than the stained parasite nuclei because they are larger and Absorb more light. WBC nuclei are easier to detect and classify because they are larger, regular in shape, and have a dark magenta color. Thus, in some embodiments, the WBC cell nucleus can serve as an easily detectable analog of the parasite nucleus. The systems and methods herein apply a dark threshold to a standard grayscale intensity image, followed by one or more of the region, color or shape filters to obtain WBC nuclei with sufficiently high sensitivity and specificity.

Referring again to the schematic diagram of the image pre-processing module of FIG. 4, the WBC detector sub-module 420 is applied to the subset 411 of grayscale FoV images using the direct WBC detection algorithm outlined above, thereby generating an indication Which image pixels are a series of binary images 421 that are part of the WBC cell nucleus. The sub-module 430 accumulates random samples of the R , G , and B values of the detected WBC cell nuclear pixels 431 and stores them in the data memory. Pixels that are not part of the WBC are classified as potential background pixels. Dark pixels are excluded from the background pixels to avoid background pixels being affected by parasite nuclei pixels (because they are too small to be detected by the WBC detector) or from dark areas corresponding to staining artifacts (eg, RBC, platelets, etc.) Pollution of either pixel. The systems and methods herein can include a sub-module 440 that can accumulate random samples of qualified background pixels 441 and store the random samples in the data memory.

The WBC cell nuclear pixel value 431 and the background pixel value 441 can be used by a machine learning algorithm (or module) to determine an adaptive grayscale projection vector that optimizes the separation between the WBC cell core and the background. (in the red, green, and blue pixel value spaces). In one embodiment, ridge regression techniques can be used (eg, by at least one processor being stored in at least one memory storage medium) to learn optimal vectors. . In some embodiments, the design matrix X can be constructed by stacking red, green, and blue values for the WBC cell core and background pixels as according to the following matrix: Where N is the number of WBC cell nuclear pixels and M is the number of accumulated background pixels. The corresponding target variable η vector can be constructed to stack N 1s above M zeros, as according to the matrix below:

In some embodiments, the ridge regression aimed at finding that the L ² having the formula defined by the following equation 5 is - regularization optimization problem that minimizes a vector : Where c is the normalized constant of the appropriate choice. The methods and systems of this paper can be used to have a formula Projection calculation adaptive grayscale intensity Adaptive gray direction vector .

As shown in FIG. 6B, the use of an adaptive grayscale intensity image instead of a standard grayscale intensity image results in a separation between the WBC cell core and the background grayscale intensity values greater than that found in the standard grayscale intensity image, and thus This results in a greater separation between parasite nuclei and background grayscale intensity values. A grayscale intensity bar graph of the background pixel 611, the WBC cell nuclear pixel 612, and the parasitic cell nuclear pixel 613 for the adaptive grayscale intensity image is shown in FIG. 6B, wherein it can be seen that the overlap region 614 is compared to The overlap region 604 determined using the standard grayscale intensity image in Figure 6A is significantly reduced.

In some embodiments, polynomial regression can be used in place of linear regression as described above. Polynomial regression is an extension of linear regression and allows a nonlinear relationship between the target variable η vector and one or more predictor variables (eg, ξ ). For example, polynomial regression can be used to find a linear relationship between the target variable η and the second-order polynomial predictor variable 通过 by the methods and systems herein. In one embodiment, the second order polynomial predictor variable ζ may be defined by Equation 6 below.

In some embodiments, higher order polynomials can be incorporated into the regression used to determine the adaptive grayscale intensity to provide an adaptive grayscale intensity image. This concept can be further extended to predictor variables that include rational functions for R , G , and B values. In one embodiment, the 24-component predictor variable ζ can be used to determine the adaptive grayscale intensity to provide an adaptive grayscale intensity that separates the background pixel from the intensity value between the WBC and the analyte pixel. image. In one embodiment, the 24-component predictor variable ζ may have a formula defined by the following equation (7): among them, To properly select the constant to prevent the denominator of the ratio from disappearing. In other embodiments, other nonlinear functions of the R, G, and B components are used. The introduction of a nonlinear relationship between the target and predictor variables serves to further enhance the separation between the parasitic cell nuclear and background pixels in the adaptive grayscale intensity image. Some formalized forms are used in the regression calculations disclosed above. Normalization is used to counteract the negative consequences of multicollinearity between the components of the predictor variable ζ. In various embodiments, the regular regression technique is selected from the following: ridge regression, lasso regression, principal component regression, and partial least squares regression.

Referring again to FIG. 4, sub-module 480 calculates a regression model between the predictor variable ξ or ζ and the target variable η. The parameters of the regression model 481 can be stored in the data memory and used by the sub-module 490, along with the input image 301, to calculate the adaptive grayscale intensity image 491. The color corrected image 471 and the adaptive grayscale intensity image 491 are the output image 311 of the image preprocessing module 310 (Figs. 3A and 3C). The output image 311 includes n _xy FoVs, each FoV including n _z focal planes, each focal plane including image (s) of color corrected red, green, and blue components and an adaptive grayscale intensity image (Multiple), as shown in Figure 3C.

As previously mentioned, located in the FoV parasites may be any of a n _z optimum focus focal planes in the captured in. 7 is a side-by-side comparison of images in different FoVs with multiple focal planes, where one FoV includes parasites and the other FoV includes artifacts, according to one embodiment. Figure 7 shows a side-by-side comparison of FoV with multiple focal planes, one FoV including an analyte (eg, a parasite) and the other FoV including artifacts (eg, platelets) therein. The image analysis system herein is configured to inspect all focal planes for each input FoV to find the location of potential parasites. The appearance of the parasite will be different in each focal plane image. Each FoV may include one or more focal planes (eg, 1, 2, 3, 4, 5, 6, 7, 8, 9, or more than 9 focal planes). Figure 7 left column shows a small portion of the parasite containing FoV having n _z = 7 focal planes (e.g., seven different focal planes) in one embodiment. In some embodiments, one or more clusters (eg, spots or spots) indicating pixels of the candidate object can be detected in the vicinity of the parasite in one or more focal planes, such as by A threshold is applied to the adaptive grayscale intensity image of the focal plane. In this same manner, the candidate object can be detected near an artifact that is darker than the background, such as in the vicinity of the platelets. The right column of Figure 7 shows a small portion of the FoV containing candidate objects that are not parasites but artifacts (e.g., it can be a collection of platelets or stains).

B. Candidate object detection module

FIG. 8A is a schematic diagram of the candidate object detection module 320 also shown in FIG. 3A. The output image 311 (eg, setting the color corrected RGB and the adaptive gray image) is input to the candidate object detecting module 310. The candidate object detection module 310 can include a plurality of sub-modules each configured as described below. Sub-module 810 can perform a thresholding operation on the adaptive grayscale image and output one or more detection masks 811. The sub-module 820 can be configured to correlate the detected pixel clusters (hereinafter referred to as "spots") of candidate objects that are close to each other (in the <x, y> image coordinates) as part of the candidate object and output the object cluster 821 location. The sub-module 830 can be configured to determine a focal plane for each candidate object or portion thereof by determining a focal plane with a highest focus score for image repair containing the detected candidate object (eg, a segment of the FoV with the candidate object) Find the plane of the best focus 831. Sub-module 830 can determine, select, and output one or more focal planes having the highest focus score 831 for each candidate object. In one embodiment, the Brenner score can be used to find the best focus 831, which is represented by z _* . Other focus scores can be used in other embodiments. In the embodiment shown in Figure 7, z _* = 5 is the best focal plane for the candidate object (parasite) in the left column. The best focal plane for the candidate object (artifact) in the right column of Figure 7 is z _* =4. Sub-module 830 also identifies the darkest spot in the best focal plane and discriminates (eg, determines, assumes, or at least temporarily allocates) this spot to represent the candidate of interest. In another embodiment, the most rounded spots are assigned to represent candidate objects of interest. Rounded spots can more closely correspond to malaria parasites or parts thereof (such as cytoplasm or nucleus). In various embodiments, other attributes or combinations of attributes are used to select representative spots. In the two columns of Figure 7, the center of the spot is indicated by a cross, z _* = 5 and z _* = 4, respectively.

Referring to Figure 8A, sub-module 840 is configured to determine (e.g., calculate) attributes 841 of the primary spot for each candidate object. Attributes (e.g., area, roundness, grayscale intensity, etc.) are calculated by sub-module 840. Sub-module 850 can be configured to filter the candidate objects based at least in part on the determined attributes. Filtering the candidate objects based at least in part on the determined attributes reduces the number of artifacts in the set of candidate objects as shown at 851. Sub-module 850 can be configured or include an artifact classifier configured to score candidate objects based at least in part on one or more attributes. Sub-module 850 can be configured to determine a score for a candidate object based on one or more of any of the determined attributes disclosed herein, such as with one or more based at least in part on one or more candidate objects Characteristics (strength, color, shape, size, etc.) The candidate object is a likelihood-related score for the analyte. Sub-module 850 can be configured to discard candidate objects having scores below a threshold score.

The artifact classifier of sub-module 850 can be pre-trained by an object image known to the annotation process using its ground truth individual (eg, an analyte or non-analyte), whereby the parasite is pre-marked by a human expert . The labeling process stores the <x,y> position of the parasite and the optimal focal plane. Candidate objects close to known parasite locations are considered to represent parasites. Candidate objects that do not approach known parasite locations are considered to represent artifacts. The properties of the known parasites and artifacts and the ground truth category are used for the pre-training artifact classifier 850. In one embodiment, the artifact classifier is configured as a non-linear core SVM. In other embodiments, other classifiers are used. Sub-module 860 can be configured to extract and output image patches 861 of filtered candidate objects. Image patch 861 is a small portion of the color corrected RGB image and adaptive grayscale intensity image containing the candidate object. These image patches 861 (321 in Figure 3A) are output to a feature extraction module, which is shown at block 330 in Figure 3A.

Other aspects of the candidate object detection module are disclosed below (Figs. 8B-8G). For example, the systems and methods of the present disclosure can be used to detect portions that are very small and include a negligible proportion of total pixels in the image (eg, 10% or less, 5% or less, or 2% or less). Object. The system and method of the present disclosure can calculate a spatial variation/adaptive threshold (for grayscale intensity) that avoids false positive detection in high noise regions by ridding the noise floor, and which reduces the grayscale intensity threshold by Respond to low noise areas to achieve maximum sensitivity in low noise areas. The systems and methods disclosed herein do not rely on the presence of large dark objects, such as WBCs (or bright objects), to infer reasonable thresholds; while considering known large dark objects, such as WBC (or bright objects) ) to avoid their distortion of the space-varying noise floor calculation. Other aspects of the candidate object detection module are disclosed below (Figs. 8B-8G).

FIG. 8B is a schematic diagram of the spot detection sub-module 810 of the candidate object detection module 320 of FIGS. 3A and 8A. The spot detection sub-module 810 can receive one or more output images 311 (eg, an adaptive grayscale intensity image from the image pre-processing module 310) as input; and one or more binary images 421 ( For example, from image pre-processing module 310), such as a WBC detection input (eg, a mask). The adaptive grayscale intensity images of the output image 311 and the binary image 421 are received by the threshold determination sub-module 812 of the spot detection sub-module 810. Based on the number of operations performed on the adaptive grayscale intensity image and the binary image 421, the threshold determination sub-module 812 can output an adaptive grayscale (intensity) of each image patch and/or FOV of the image. Threshold. The adaptive grayscale (intensity) threshold output from threshold determination sub-module 812 may illustrate whether the spot is a candidate for the image, WBC, background, or any other aspect. The adaptive grayscale intensity threshold is then applied by the spot recognition sub-module 814 to detect and locate the blobs (eg, candidate target clusters) output from the spot recognition sub-module 814 as one or more (candidate target) detection masks 811.

Typical backgrounds for grayscale intensity images or portions thereof (eg, FoV or image patching) may include staining and other noise (eg, artifacts such as platelets, partially lysed or unlysed red blood cells, stained aggregates, etc.) ). Candidate parasite nuclei (eg, candidate items) are typically darker than the background in the image from the stained slide. Therefore, they can be detected by applying a grayscale threshold to the grayscale intensity image or equivalently by applying a luminance threshold to the inverted grayscale intensity image. These darkness and/or brightness thresholds can be expressed as grayscale intensity thresholds. Throughout this disclosure, conventions for inverting grayscale intensity images can be employed. Thus, a brightness threshold can be applied to invert the grayscale intensity image to detect potential parasite locations.

The value of the grayscale intensity threshold may be critical to the detection sensitivity achieved by an image analysis system or technique. However, a single grayscale intensity threshold for the entire image, FOV, and/or image patching can result in false positives and missing parasites (eg, parasites that are not easily distinguishable from the background). Such false positives and/or missing parasites may be due to local variations in features and content of images, FoV or image patches (eg, changes in one or more colors or grayscale intensities of the background, or the presence of WBC and/or RBC) ) caused by.

A local grayscale (intensity) threshold having a selected value can effectively divide an image pixel into one of two categories: a pixel having a grayscale intensity equal to or lower than a threshold or a pixel having a grayscale intensity above a threshold. Depending on whether the brightness of the image has been inverted, a gray level intensity equal to, below or above the threshold may indicate that the pixel is part of the background or part of the candidate object. Furthermore, when the WBC is included in the calculation of the overall grayscale intensity of the image or FoV, the calculated grayscale intensity threshold may be too dark or too bright to provide reliable candidate object detection, especially at low populations of target parasites. Under the level.

In the image analysis applications and systems disclosed herein, high noise regions may benefit from high grayscale intensity thresholds to avoid false positives that may be triggered by artifacts in the region. The low noise region may benefit from a low grayscale intensity threshold (eg, below a high grayscale intensity threshold) to provide relatively high sensitivity to candidate objects (eg, parasites). A single global threshold applied to all FoV or image patches of an image may be a compromise between the conflicting requirements of high noise regions and low noise regions in the image. A single global threshold applied to an image, FOV, or image patch may result in some false positives and missing candidate objects (eg, parasites) due to loss of local sensitivity around candidate objects and artifacts.

Some threshold techniques assume a bimodal distribution of pixels in a FoV or image patch, such as a category or population above a threshold and a category or population below a threshold, and calculate a threshold based on a bimodal population. Such techniques may minimize the weighted average of the grayscale intensity variations within the category, or equivalently maximize the inter-category differences in the average grayscale intensity (hereinafter referred to as "simple bimodal techniques"), but when detected Such techniques are not as accurate as the techniques disclosed herein when the parasite constitutes a small portion of the image (eg, less than 10%, less than 5%, less than 2%, less than 1%, or less than 0.5%). For example, when the object to be detected represents a small fraction of the total number of pixels in the image (eg, less than 10%, less than 5%, less than 2%, less than 1%, or less than 0.5%), such a simple bimodal technique is based on Relatively balanced image content (eg, relatively close segmentation of two types of pixels, eg, 70%: 30% to 50%: 50% or even 80%: 20%) loses accuracy or does not identify the object. This loss of accuracy may occur when a small malaria parasite (compared to WBC) that may constitute a relatively small portion of a blood sample will be detected in the grayscale intensity image of the blood smear.

Some thresholding techniques can model each type of bimodal pixel class at or below the threshold, respectively, and the ratio, mean, and variance of the Gaussian distribution are calculated from the pixels in the category (hereinafter referred to as " Gaussian double peak technology"). Such a Gaussian bimodal technique may select a threshold that minimizes the error between the modeled distribution and the empirical grayscale intensity distribution calculated from the grayscale intensity image itself. However, the Gaussian bimodal technique still does not recognize the grayscale intensity threshold that separates the object from the background when the imbalance of the category is extreme, such as when a category includes only 10% less than the total pixel, less than 5% of the total pixel. Less than 2% of the total pixels, less than 1% of the total pixels or less than 0.5% of the total pixels, which may be the case for malaria parasites on blood smear images. In the case where the WBC and the parasite appear in the same FoV, the Gaussian bimodal technique can calculate a reasonable threshold if the population is relatively balanced. However, the Gaussian bimodal technique relies on the presence of the WBC to provide some of the pixels in the category, and the WBC does not appear and cannot rely on appearance in every image, FoV or image patch. Therefore, the threshold determined by the Gaussian bimodal technique may not be reliable.

The aforementioned thresholding technique (simple and Gaussian bimodal techniques) calculates a single constant grayscale intensity threshold for the entire FoV only. In many FoVs, there may be areas with dense artifacts (high noise floor) and regions with few artifacts (low noise floor). The words "noise" herein are used to indicate image elements that are not of interest (eg, artifacts such as platelets, partially lysed or uncleaved red blood cells, stained aggregates, etc.). The noise in the present disclosure may also be referred to as a background. This is in contrast to most image processing applications where the term "noise" refers to small unwanted elements in the image (eg, missing pixels) in order to reduce noise and preserve the background. As explained in more detail below herein, the "noise floor" can be a window, image patch, or local median grayscale intensity value (discount WBC) of FoV.

The systems and methods disclosed herein both determine grayscale intensity thresholds for one or more windows, image patches, or regions in each FoV, and do so without pixel information or features of objects (eg, WBCs) that are not of interest. Incorporate into the determination. For example, the adaptive grayscale intensity threshold is determined by replacing the grayscale intensity value of the WBC pixel with a replacement median grayscale intensity value (eg, individual and unique for each of a plurality of image patches (eg, regions) of FoV) Grayscale intensity threshold), which may be pixels from the entire adaptive grayscale intensity image (eg, random sampling with a sufficiently large population to ensure the accuracy of the average grayscale intensity of the pixel) Median or average grayscale intensity. After such an alternative, the median grayscale intensity from the pixels of the entire window is used to calculate an adaptive grayscale intensity threshold that includes a replacement intermediate grayscale intensity value for replacing the WBC pixel value. The local adaptive threshold may include a local median grayscale intensity value calculated in each viewport, or may include some value that deviates from the selected amount (eg, a value that is 10% brighter or darker than the local median grayscale intensity value). The calculation of the local median grayscale intensity value and the local adaptive threshold based thereon is further described below.

8B is a schematic diagram of a spot detection sub-module of the candidate object detection module of FIGS. 3A and 8A, according to an embodiment. The threshold determination sub-module 812 shown in Figure 8B can calculate an accurate local threshold for the local median grayscale intensity value (which can vary between window, FoV, and/or image patching) even when the object to be detected This is also true when representing a small portion of a pixel in an image (eg, 10% or less, 5% or less, 2% or less, 1% or less, or 0.5% or less). The threshold determination sub-module 812 does not rely on the presence of the WBC in the FoV to calculate an accurate or valid threshold. The WBC present in the FoV also does not reject threshold determinations (e.g., threshold determination calculations) as disclosed herein. The threshold determination sub-module 812 calculates a spatially varying threshold (eg, a local adaptive threshold), selects a high threshold in the high noise region, thereby avoiding false positive rates in these regions, while selecting a low threshold in the low noise region Therefore, high sensitivity is also achieved in these areas. Threshold determination sub-module 812 can calculate a threshold for spatial variation (eg, local adaptation) by locally estimating the noise floor.

The spot detection sub-module 810 can include and perform a plurality of operations on the image to identify spots therein. A schematic diagram of the operations performed by the blob detection sub-module 810 is depicted in Figures 8C-8D. The threshold determination sub-module 812 can determine a local adaptive grayscale intensity threshold for the image, as described below.

Figure 8C is a field of view image input to the spot detection sub-module of Figure 8B. Figure 8C shows the FoV 870 of the blood smear image. The FoV 870 can be provided as an adaptive grayscale intensity image from the output image 311 and/or a binary image 421. The input FoV 870 shown in Figure 8C contains four malaria parasites 876, 878, 882 and 886. The input FoV 870 also contains three WBCs 872, 874 and 880. Input FoV 870 has artifacts 884 (eg, platelets or stain aggregates). In addition, the input FoV 870 has a high noise area 871 and a low noise area 881.

The threshold determination sub-module 812 can estimate the noise floor by locally determining a median grayscale intensity value on one or more of the windows 890 in the image (eg, an adaptive grayscale intensity image). Threshold determination sub-module 812 can determine a full-image or partial (eg, one or more discrete windows in FoV) median grayscale intensity values for FoV or window 890. For example, threshold determination sub-module 812 can calculate (eg, determine) a median grayscale intensity value for window 890 shown in FIG. 8C. The median grayscale intensity value of the location in the image (eg, the window) may provide a value for the local adaptive threshold, and pixel intensity values from within the window above or below the value may indicate the presence of the candidate object or its cluster ( For example, spots).

Figure 8D is a modified field of view input image of Figure 8C. Figure 8D shows the corrected FoV 870' of the blood smear image. Threshold determination module 812 can receive information regarding the presence of WBCs 872, 874, and 884 in FoV 870 from a WBC detection mask input (eg, binary image 421). If the input FoV 870 includes one or more WBCs 872, 874, and 884, the threshold determination module 812 can use the median grayscale pixel intensity from the entire image (eg, the entire image or one or more portions thereof) The value of the grayscale intensity, or the median grayscale pixel intensity of the pixel with the WBC, is substituted for the pixels belonging to WBCs 872, 874, and 880 to produce a corrected FoV 870'. This is schematically illustrated in Figure 8D, in which the grayscale intensities of the pixels belonging to WBCs 872, 874 and 880 of Figure 8C have been used for the full image shown in objects 892, 894 and 896, respectively. The median grayscale pixel intensity value is replaced. After replacing the WBC pixel by the median grayscale pixel intensity in the corrected FoV 870', the noise floor or threshold estimate (eg, the window's median pixel intensity estimate) will not respond to the WBC by raising the threshold near the WBC. Considering WBC will reduce the adverse effects on parasite sensitivity near WBC. In other words, WBC considerations in determining the median pixel intensity estimate of the window (eg, local noise floor estimate) will bias the noise floor estimate toward a threshold that reduces sensitivity to parasites (eg, parasites, such as in grayscale intensity) On malaria similar to WBC). Thus, the threshold determination sub-module 812 can estimate by locally determining the median grayscale intensity values on one or more of the windows in the image while reducing any changes in the grayscale intensity of the intermediate pixels due to the presence of the WBC. Noise floor. The location of the WBC is known and is provided in the WBC detection mask. In some embodiments, at least some of the windows may include one or more candidate objects (eg, spots) therein, and the pixels from the candidate object contribute to calculating the median pixel intensity of the window when performing noise floor estimation. For example, when calculating the median grayscale intensity or the local noise floor, the grayscale intensity value of the pixel in the candidate object is considered, because the pixel corresponding to the hitherto unknown candidate object or its cluster does not correspond to the WBC, and thus is used for Calculated. In some embodiments, the amount of candidate objects in the viewport may represent a population of pixels that are so small (eg, less than 10%, less than 5%, less than 2%) such that the adaptive grayscale threshold determined therefrom is not It is biased to provide inaccurate results. Local noise floor can be used to set or determine a local adaptive grayscale (intensity) threshold. For example, the local noise floor can be used as a local adaptive grayscale threshold, or some grayscale intensity values above or below the local noise floor can be used as the local adaptive grayscale threshold.

Returning to Figure 8C, a median grayscale intensity value (for pixels) on one or more of the plurality of windows 890 in the image can be determined. Window 890 can be positioned (eg, tiled or placed) in/in an image (eg, FoV) in a regular pattern in a so-called "sliding window filter." Although a sliding window method can be applied to reduce noise in the image, in the present disclosure, a sliding window filter is used to estimate noise (eg, background) in the FoV or window 890. The size of the window 890 can be selected to provide a desired noise floor estimate sample size or spatial resolution. For example, a larger window may result in a more robust estimate of the noise floor, but it also increases the spatial extent of the noise floor estimate, which may result in the loss of small low noise regions. Conversely, smaller windows may allow for better spatial resolution, but may not be robust in noise floor estimation. In some embodiments, the size of the window 890 can have at least one dimension (eg, width and/or height) of at least about 10 pixels, such as from about 10 pixels to about 100,000 pixels, from about 100 pixels to about 10,000 A pixel, from about 10 pixels to about 1000 pixels, less than about 10,000 pixels, or less than about 100,000 pixels.

The "step" of the sliding window filter (eg, the distance between successive applications of the window filter) can be selected to provide a selected resolution or computational burden. For example, the sliding window filter can be calculated with a step of one pixel such that when calculating the median grayscale intensity value of the entire window 890, the window 890 is shifted one pixel to the right, and the median grayscale intensity value is calculated again. and many more. In an embodiment, each local median grayscale intensity value (eg, calculated by replacing a WBC pixel) may be associated with a respective window from which to determine them. The stride of one pixel can calculate the median filtered grayscale intensity image with the same resolution as the original image, but the computational burden can be very high (eg, up to four times the stride of two pixels). In an embodiment, the stride may be two or more pixels, such as at least two pixels, at least five pixels, at least 10 pixels, at least 50 pixels, at least 100 pixels, at least 1000 pixels or at least 10,000 Pixel. This can reduce the computational burden, but reduce the resolution of the filtered image, potentially reducing the fidelity of the noise floor estimate. In some embodiments, when the sliding window stride is greater than one pixel, the median filtered image can be interpolated to the original resolution of the input image. In some embodiments, different strides can be selected in response to a particular selected resolution or computational burden.

In some embodiments, one or more windows 890 can be used to determine a median grayscale intensity value for each portion of the image (eg, a noise floor estimate for an adaptive grayscale threshold), such as each of the images Partial local variation / adaptive median grayscale intensity (eg, noise floor). In some embodiments, the WBC pixels in the image may be replaced with a grayscale intensity value of the full image or a median grayscale intensity value of the local variation of the window or a window therein to determine a local adaptive grayscale threshold. This technique enables closer approximation of the actual background (eg, noise) in the image by reducing the effects of variations due to known non-analytes such as WBC. For example, a pixel corresponding to a known WBC in a viewport may be replaced with a median grayscale intensity value determined for the viewport or with a median grayscale intensity value for the full image. As noted above, the median grayscale intensity value can vary over the image or portion thereof. Similarly, the determined local adaptive grayscale (intensity) threshold can vary over the image or portion thereof. Thus, systems and methods for detecting analytes in fluids such as blood can apply multiple local adaptive grayscale (intensity) thresholds to corresponding windows of an image to produce noise in the image or portion thereof. Bottom spatial variation/adaptive estimation (eg, background and median grayscale intensity values for candidate objects).

The noise floor can be estimated for each window in FoV, the entire FoV, or the entire image (eg, by threshold determination module 812), and the noise floor can be spatially varied depending on the portion of the image in question. The median filtered grayscale intensity image or noise floor image is a spatially varying/adaptively estimated image of the noise floor in the FoV or image. Most of the pixels of the object of interest may have grayscale intensity values above the median grayscale intensity value (eg, local adaptive threshold) of the noise floor image at its location in the image. Objects of interest may include parasites and WBCs. The object of interest can be detected by subtracting the noise floor image from the grayscale intensity image and applying a threshold.

The blob detection sub-module 814 can apply a threshold (eg, a local adaptive threshold) to identify the presence and/or location of any object of interest (eg, a blob) in the FoV/image. The local adaptive threshold may include some grayscale or color intensity values that are above or below a selected value (eg, a noise floor estimate or an amount above the noise floor estimate). For example, a local adaptive threshold may be selected to identify grayscale or color intensity values above the noise floor or some values above these values. Therefore, the noise floor image plus the local adaptive threshold can be considered as a (spatial change/adaptive) threshold image. Any pixel in an image having a grayscale intensity above (eg, or below, depending on whether the image is a non-inverted grayscale intensity image) value in the threshold image may be considered an object of interest . In other words, the sense can be identified or detected by subtracting the noise floor image from the grayscale intensity image and identifying any pixels (eg, grayscale values) having grayscale intensity values greater than the selected (grayscale intensity) threshold. Interest objects (eg, spots) (eg, applying a threshold). A set of pixels or clusters of objects of interest may indicate the presence of one or more spots. The spot detection module 814 can identify one or more groups or clusters of pixels in one or more windows, FoV, and/or focal planes as spots, and output one or more detection masks 811 indicating the location of the spots , for example, in a particular window, FoV, and/or focal plane. The spot detection module 814 can output one or more detection masks 811 to the spot cluster sub-module 220 of FIG.

Figure 8E is a grayscale intensity bar graph 900 of the pixels of the field of view image of Figure 8C. FIG. 8E shows the grayscale intensity bar graph 900 of the FoV image of FIG. 8C in solid line 904. Bar graph 900 depicts a plot of inverse grayscale intensity (represented from right to left, from 0 to 1.0, where 1 is 100%) and number of pixels (in arbitrary units). The grayscale intensity of the background extends from zero to about 0.75. The bumps 908 at about 0.9 in the bar graph may correspond to the grayscale intensity values of the WBC in the image 870 (Fig. 8C). The grayscale intensity value (Fig. 8C) of the parasite in image 870 can range from about 0.3 to about 0.8. Therefore, the grayscale intensity of the background and parasites may overlap. Furthermore, since the number of pixels is negligible compared to the entire image, the parasite has no distinct peaks in the bar graph.

Vertical line 912 corresponds to a (constant) threshold calculated by a simple two-peak technique. It can be seen that applying a simple bimodal technique threshold to the grayscale intensity image will result in a large amount of false positive detection (the value to the right of line 912), which may overwhelm the image analysis system. Vertical line 914 corresponds to a (constant) Gaussian bimodal technique threshold. As shown in Figures 8C and 8E, the application of the Gaussian bimodal technique threshold will successfully detect a parasite in the high noise region (e.g., as shown, by more than about 0.76 (e.g., above 0.76 but below A threshold grayscale intensity value of about 0.8) indicates the object, but the other three parasites are completely missed (eg, objects below the grayscale intensity range of about 0.76).

The thresholding techniques disclosed herein may provide local variation thresholds that are capable of distinguishing objects having similar grayscale intensities from a particular region of an image. For example, region 916 corresponds to a range of thresholds that can be calculated by the techniques of the present disclosure. These are the values of the pixels in the above threshold image. Thus, any value above the value of the pixel in the threshold image calculated above may indicate the presence of an object or spot of interest (eg, a parasite).

The illuminated view of these results can be obtained by examining the grayscale intensity and threshold along a path through the image 870 of Figure 8C. Figure 8F is an illustration of the path through the field of view image of Figure 8C. Figure 8F shows path 899 through image 870 of Figure 8C. Window 890 (Fig. 8C) may be tracked along path 899 to produce a varying gray level threshold determination of the pixels along path 899. Path 899 passes through WBC 874, parasite 876, parasite 878, parasite 882, artifact 884, and parasite 886. Path 899 passes through high noise region 871 and low noise region 881.

The local adaptive threshold can vary depending on the location in the image. The value of the local adaptive threshold along path 899 in Figure 8F is shown in Figure 8G as a line 917 with a variable value (e.g., a threshold distribution). Figure 8G is a graph 950 of inverted grayscale intensity as a function of position on path 899 of Figure 8F (when path 899 travels from left to right, in arbitrary units of associated locations). The high noise region 871 corresponds to the portion of the graphic 950 to the left of the abscissa position 965, and the low noise region 881 corresponds to the portion of the graphic 950 to the right of the abscissa position 965 in FIG. 8G. Peaks 974, 976, 978, 982, 984, and 986 in Figure 8G correspond to WBC 874, parasite 876, parasite 878, parasite 882, artifact 884, and parasite 886, respectively (in Figure 8F, respectively). In Figure 8G, a simple bimodal technique threshold 912, a Gaussian bimodal technique threshold 914, a variable threshold (local adaptive threshold, shown as a threshold distribution at line 917 and calculated as described herein), and an actual grayscale intensity 960 are show. As can be seen, the simple bimodal technique threshold 912 has a constant value that can identify the WBC 874 and all four parasites 876, 878, 882, and 886, as represented by the actual grayscale intensity 960 extending over the simple bimodal technique threshold 912. Corresponding peaks 974, 976, 978, 982, and 986 demonstrate WBC 874 and all four parasites 876, 878, 882, and 886. However, many false positive detections may also be indicated by a simple bimodal technique threshold 912 (eg, because of the large imbalance of the respective populations of pixel classes in the bimodal population), as previously described, this may cause problems in the image analysis system. . For example, a peak 984 corresponding to artifact 884 may be erroneously identified as an object of interest.

It can also be seen that the Gaussian bimodal technique threshold 914 has a constant value and can only detect WBC 874 and parasite 878 as shown by peaks 974 and 978 extending beyond the Gaussian bimodal technique threshold 914. Using the Gaussian bimodal technique threshold 914 may result in parasites 876, 882, and 886 (each in Figure 8F) being undetected because the corresponding peaks 976, 982, and 986 are below the Gaussian dual peak technique threshold 914.

The local adaptive (grayscale intensity) threshold (eg, the grayscale intensity of the threshold image) along path 899 of FIG. 8F is shown as a dotted line (threshold distribution of line 917) in FIG. 8G. The local adaptive threshold for a particular path, line or window of the image may adaptively vary between any value of region 916 (Fig. 8E). The grayscale intensity value of the threshold image along path 899 corresponds to the threshold distribution at line 917. As shown, the threshold distribution at line 917 (and techniques for determining and applying the thresholds described herein) may provide a variable or adaptive threshold to identify an object of interest (eg, a blob) (otherwise the object passes The simple Gaussian bimodal technique threshold 914 will not be detected) and will prevent exceeding the included threshold bimodal technique threshold 912 (preventing false positives); while allowing artifacts 884 (Fig. 8F) and/or other non-analytes (eg, non-parasitic Insects are excluded as objects of interest.

Although the threshold distribution set on line 917 detects all of the required objects, WBC 874 and all four parasites 876, 878, 882, and 886, it does not detect a large number of false positives because it rides over. Noise floor. That is, the threshold distribution set at line 917 roughly tracks the local average gray level intensity in the image. For example, it can be seen that the value of the threshold distribution set at line 917 in high noise region 871 is typically higher than the actual grayscale intensity value 960 (eg, except at the peak corresponding to the object of interest), such that false positives It is avoided in the high noise area 871. In the low noise region 881, the value of the threshold distribution set at line 917 is relatively low in response to the lower noise level in the region, but is still substantially higher than the actual grayscale intensity value 960 (eg, in addition to the corresponding The peak value of the object of interest). Because the artifact 884 does not exist in the WBC detection input mask of the binary image 421 of FIG. 8B (eg, the artifact 884 is not recognized as a WBC), the threshold distribution set at line 917 can be crossed at peak 984 ( For example, the extension exceeds the artifact 884. When the grayscale intensity threshold of the window containing artifacts 884 is determined, such non-existence in the WBC input detection mask of binary image 421 will cause artifact 884 to be considered a background, thereby resulting in crossing artifact 884. Grayscale intensity threshold for grayscale intensity values. Unlike the extremely small parasites (e.g., at least 50%, 75%, or 90% smaller than the artifact 884), the relative size of the artifacts 884 can cause the grayscale intensity threshold to be determined to be far enough to cause the line 917 to be set. The threshold distribution (eg, the local adaptive threshold) exceeds the artifact grayscale intensity. The inventors currently believe that, in most cases, certain parasites (e.g., circulatory malaria parasites) are not large enough to cause threshold determination module 812 (Fig. 8B) to bias local grayscale intensity thresholds sufficiently to threshold The distribution crosses the parasite. Such determination may depend on the relative size of the window and the artifacts and/or parasites therein. For example, a window with a large percentage of pixels (eg, 50% or more) corresponding to an artifact may produce an adaptive threshold for any parasites that pass over it. In contrast, a window with only a tiny amount of parasite-containing pixels (eg, less than 10%, 5%, 2%, 1%) may not cause the calculated adaptive threshold to be higher than the pixel corresponding to the parasite. Strength of. Accordingly, the size of the window can be selected to provide an adaptive (grayscale intensity) threshold that can override artifacts 884 while detecting parasites.

Returning to Figure 8B, and in accordance with Figure 8G, the plaque identification module 814 can be used to identify the presence of one or more peaks that exceed the actual grayscale intensity 960 of the threshold distribution (or local adaptive threshold) set at line 917. One or more peaks of the actual grayscale intensity 960 that exceed the threshold distribution set at line 917 may be identified by the spot recognition sub-module 814 as an object of interest (eg, a blob) and output as a detection mask 811. The spot recognition sub-module 814 can output the detection mask 811 to the spot cluster sub-module 820 (Fig. 8A) as described herein.

C. Feature extraction module

FIG. 9 is a schematic diagram of feature extraction module 330 also shown in FIG. 3A. The feature extraction module 330 is configured to represent each candidate object as a feature vector and output the feature vector. The feature vector(s) may be classified by the object classifier module 340 of Figure 3A as a parasite (even the species or stage of its parasite) or an artifact. As shown in FIG. 9, feature extraction module 330 is configured to calculate at least one of two types of features. Features can be manual features or automatic features. Feature extraction module 330 has two sets of inputs, one for manual feature extraction and the other for automatic feature extraction. Feature extraction module 330 can operate in one of two modes (manual feature extraction on (ON) or manual feature extraction off (OFF)). In various embodiments, manual feature extraction may be ON or OFF, while automatic feature extraction is always ON.

The first method of feature extraction is manual feature extraction or feature engineering in the computer field of view. These are features that are deliberately designed to measure the specific attributes of the candidate object and rely heavily on domain knowledge that has been learned (eg, previously known or pre-programmed).

Wherein for manual input 901 comprising a candidate object by the color correction of the R, G, B image patch and all its focal plane n _z. The sub-module 910 of the feature extraction module 330 assigns three manual features 911 to the feature vector.

The first manual feature is the best focus score of the candidate object (eg, Brenner score). Referring back to FIG. 7, the focal plane for n _z calculated for each repair area focused on the image points, the optimal focal plane is the focal plane with the highest focus score. The second manual feature is the standard deviation (and/or other deviation measure) of the focus score across the focal plane of the FoV having the candidate object feature therein. The motivation behind this is that some artifacts (such as bubbles and dust particles on the sample) will have the same focus score on all focal planes because they are far from focusing, while the ring malaria parasite (or other analyte) will There is a narrow focus score distribution surrounding the best focal plane, thus having a small focus score standard deviation.

The sub-module 910 can be configured to extract a third manual feature, which is the so-called redshift score ("redshift" is used here as a descriptive term and is not related to the redshift phenomenon caused by the Doppler effect. ). The redshift score helps distinguish between parasites and artifacts. The redshift score depends on the fusion of the two concepts. The first concept is light dispersion, which refers to a change in refractive index depending on the wavelength. This means that an uncorrected, simple lens will focus light of different wavelengths at different focal planes (eg, from different lengths of the lens).

10A and 10B are schematic views of light rays being refracted to different focal planes by a simple lens and a lens having achromatic aberration correction, respectively. In FIG. 10A, three light rays having representative wavelengths in the red, green, and blue portions of the spectrum are shown to be focused on planes 1001, 1002, and 1003, respectively. As the light passes through the simple lens 1010, the red, green, and blue wavelengths are refracted to different focal planes. The focus and wavelength curve 1030 of the simple lens is shown in Figure 10C, and the representative focal planes for the light rays beginning to focus at 1001, 1002, and 1003 are represented by points 1031, 1032, and 1033 on curve 1030, respectively.

Lenses with achromatic correction help limit the amount of chromatic aberration caused by dispersion. The achromatic correction lens is shown in Figure 10B with three representative wavelengths in the red, green and blue portions of the spectrum. The achromatic correction lens may include a simple lens element 1010 (eg, a enamel glass element) that is, for example, convex, mounted or bonded to a concave achromatic element 1020 (eg, a flint glass element). The achromatic correction lens is designed to focus the two wavelengths on the same plane, such as focusing on plane 1005 as shown in Figure 10B. As shown, in some embodiments, the two wavelengths are in the red and blue portions of the spectrum.

The focus versus wavelength curve for the achromatic lens is shown as curve 1040 in FIG. 10C for representing a representative focal plane of light focused at 1004 and 1005 on curve 1040 by points 1044 and 1045, respectively. . As can be seen in Figure 10C, the portion of curve 1040 in the red region of the spectrum (640-700 nm) slopes more gently than the portion of curve 1040 in the blue region (450-500 nm). Thus, as the focus setting of the microscope moves toward the upper portion of the graph, the blue light will defocus more quickly than the red light. As the focus of the microscope moves up, the green light does not defocus as quickly as the red or blue component of the light. This can be seen from the relative flatness of the bottom of curve 1040 in Figure 10C, which is in the green region of the spectrum. The first concept relies on this change in the focal plane of the lens as the microscope is adjusted for focus.

The second concept upon which the redshift score relies is the light absorption properties of the analyte (eg, DNA) upon staining (eg, using Giemasa). Figure 11 is a graph of absorption spectrum 1101 showing peak absorption in the green region of the spectrum. The absorption of green light by the combination of methylene blue and eosin Y is amplified in the presence of DNA. This means that materials on microscope slides containing DNA (eg, nuclei) will absorb green light to a large extent and transmit red and blue light, which results in their finished red color on a transmitted light microscope. Artifact objects do not contain DNA and therefore tend to be less absorbed in the green portion of the spectrum. Therefore, artifacts do not present magenta in the image.

Changing the focal plane of the microscope upwards based on the above observation will cause the blue wavelength to defocus faster than the red wavelength, and then the magenta object will appear redder because the blue component of the light will have spread to a larger spatial area, More obvious than red light. This is the basis of the redshift score, which measures the red darkening in the darkest part of the detected candidate, which is the nucleus of the parasite cell for true malaria parasites. An artifact that is more evenly transmitted through red, green, and blue light will not become redder as the focus of the microscope moves up, as described above, which counteracts the redshift effect of the red and blue components. Therefore, the redshift score provides the basis for distinguishing between parasites and artifacts.

The systems and methods disclosed herein are configured to analyze candidate object images for redshifts and provide scores on this basis. As described above, the manual feature extraction sub-module 910 (and associated microscope) can be configured to determine a redshift score. While DNA, malaria parasites, and red are provided as examples, the concept of redshift scores can be applied to different colors and analytes without limitation.

The second type of feature extracted by the feature extraction module is an automatic feature that can be automatically learned by a system including at least one memory storage device and at least one processor, such as a Convolutional Neural Network (CNN). CNN is a deep learning model of multiple levels of learning representation (applied by computer systems). Starting with the original input layer, each successive layer (eg, convolution, aggregation, sub-sampling, or fully connected layer) represents information in the image at a slightly more abstract level. The weight at each level (screening program) is learned using standard learning steps such as error backprop. In CNN, each layer (calculated) is executed by a different plurality of neurons (processing modules), and the neurons in each convolutional layer are not completely interconnected with all neurons in adjacent layers of the system. . Conversely, neurons in the convolutional layer have only a selected linearity with adjacent convolutional layers to reduce the amount of input performed on successive convolutional layers. At each convolutional layer, the convolution kernel defines a linear region with the neurons in the front layer. The convolution kernel is sometimes referred to as the receptive field of the neurons in the convolutional layer. One or more final layers in the CNN are fully connected layers with complete linearity with the immediately preceding layer, effectively performing advanced reasoning based on the material provided (which has been repeatedly abstracted throughout the layer). In some embodiments, one or more ground truths (eg, image repairs containing ground truth objects that have been identified by a human expert) can be used to train the weights of the CNN through a learning step. The CNN can be stored and executed by a computer having one or more processors, such as a central processing unit (CPU) or a graphics processing unit (GPU). The ground live image or image patch may include a known positive sample (eg, for the CNN identified as having the analyte of interest) and a known negative sample (eg, for the CNN identified as having no analyte therein, Or only have known artifacts or other non-analyte objects in it). Thus, the CNN can learn weights from both known analyte and non-analyte species, which can be used to identify this weight in the sample.

In one embodiment, a computer vision system (eg, a microscope operatively coupled to a digital recorder) can be operatively coupled to the CNN. This system can outperform manpower performance in terms of accuracy. The automatic feature extraction sub-module 920 can be configured to perform feature extraction based at least in part on a feedforward application of weights, aggregations, and non-linear operations.

Due to the richness of the model, a lot of information is needed to train CNN. If there is not enough information available for training, over-fitting may occur, resulting in poor regular performance. In some embodiments, the systems and methods herein can increase the amount of training data by generating artifact data based, at least in part, on the training material itself. This process is called enhancement. Enhancements may take the form of one or more random transformations suitable for training images. Examples of enhanced transforms are translation, rotation, scaling, reflection, and color distortion.

One technique for color distortion includes the following steps. First, the principal component transformation of the training image in the R, G, B color space is calculated. The feature vectors are denoted by corresponding eigenvalues λ ₁ , λ ₂ , λ ₃ as p ₁ , p ₂ , p _{3 , respectively} . The three random numbers r ₁ , r ₂ , r ₃ are sampled from a bounded distribution such as a Gaussian distribution with a zero mean and a standard deviation of 0.1. To produce an enhanced image, the following amount is added to each pixel in the image:

Each image presentation samples a random number r ₁ , r ₂ , r ₃ during the training of CNN.

The above techniques for color distortion can result in unrealistic color images. It is desirable to introduce a color distortion method (and a system for performing the method) that produces an image with a realistic color while providing sufficient color distortion to avoid overfitting of the CNN. This color distortion can help normalize the color change in the image due to color changes from one sample to another. For example, under Giemsa dye, the relative amounts of basophilic blue and eosinophilic eosin (red) present in the stained sample depend on the pH of the dye and the pH varies in the region. The color normalization by the distortion method in this paper can achieve a more accurate diagnosis. In the second color enhancement method of the present disclosure, each of the red, green, and blue channels (eg, components) of the image may be gamma nonlinear distortion, which is also referred to as gamma correction distortion. But in this case it is used to transform the color of the image instead of correcting them. The gamma correction is defined by the following nonlinear transformation in Equation 8: among them, Is the input value, Is the output value, and 0 < γ < ∞ is a non-linear exponent, and α is a proportionality constant. When entering a value In the range of [0, 1], the proportionality constant α = 1. The color enhancement method of the present disclosure samples 4 random numbers from a Gaussian distribution with zero mean and standard deviation σ . Subsequently, the four values of γ are calculated by the relationship, which is the base of the natural logarithm. The enhanced red, green, blue, and adaptive gray channel/component images are generated by Equation 9, respectively, as follows: Each time you enhance each image sample, it will be random. . Correspondingly, each R, G, B and intensity The channels may be individually and collectively enhanced to provide a larger sample of the data used to train the CNN suitable for use with the systems and methods herein.

Referring again to Figure 9, image patching 921 is an input to feature extractor 930 of CNN. In some embodiments, an enhanced set of ground live image patches that have been enhanced using a data enhancement scheme can be used to train the CNN to identify analytes or non-analyte objects. That is, the original image or portions thereof such as image patching are enhanced using panning, rotation, scaling, reflection, and gamma-based color distortion as described above. In some embodiments, at least one processor (associated with the CNN) is configured to be based at least in part on ground live image patching, color corrected image patching, or grayscale that has been enhanced in accordance with any of the methods disclosed herein. One or more of the enhanced groups of intensity image patches are used to learn the weight of the group. For example, ground-based live image patching can be enhanced by a data enhancement scheme that includes random gamma correction of one or more of the red, green, blue, or grayscale intensity components of the ground live image patching. . In some embodiments, image patching at the best focal plane of each candidate object is presented for CNN training. In other embodiments, image fixes for all focal planes are presented for CNN training. In some embodiments, the at least one processor is configured to enhance the color corrected image patching and the adaptive grayscale intensity image patching using an enhancement scheme. In some embodiments, the output of the color corrected image spot and the adaptive gray intensity image patch may include image enhancement and color gray intensity image patching using enhanced enhancement schemes. In some embodiments, no enhancements are made during the testing phase of the CNN feature extractor. In other embodiments, the enhancement is performed during the test phase, and the output of the classifier module as shown in block 340 of Figure 3A is averaged over the enhanced version of each test sample. In some embodiments, the at least one processor is configured to classify the machine learning classifier on a feature vector corresponding to an enhanced version of each of the color corrected image patching and the adaptive grayscale intensity image patching The output is averaged.

The output of the CNN feature extraction sub-module 930 is the CNN component 931 of the feature vector. In embodiments that use both manual and CNN features, the manual features 911 and CNN features 931 can be combined to form a complete output feature vector 941. In an embodiment without artificial features, the manual feature extraction sub-module 910 is not executed and the manual feature 911 is not advanced to the output feature vector 941.

Returning to the system diagram in FIG. 3A, the output of feature extraction module 330 is the feature vector 331 of the candidate object.

D. Object classifier module

The object classification module 340 is configured to classify the feature vectors 331 corresponding to analytes (eg, parasites) or artifacts. The object classifier module 340 is configured to classify the feature vectors 331 or outputs from the feature vector extraction module 330 as parasites or artifacts using a machine learning classifier. The machine learning classifier can be a program stored in one or more memory storage media that can be executed by one or more processors (e.g., in a computer system or network). Different implementations of the object classifier module 340 can include different types of classifiers. In one embodiment, the object classifier module 340 is configured as a linear support vector machine. For example, a linear support vector machine can include a computing device configured to perform linear support vector classification. In various implementations, the object classifier module 340 can be configured as one or more of the following types of classifiers: nonlinear kernel support vector machine, neural network, logistic regression, random forest decision tree, gradient promotion decision Tree, AdaBoost or Naïve Bayes classifier.

The output of the object classifier module 340 can include a calibrated probability that the candidate object is a parasite (eg, an analyte) or an artifact. The object classification module 340 is configured to output the classified item data 341 (Fig. 3A). The classified item data 341 can include one or more scores corresponding to the similarity (eg, indicating its range) between one or more ground live objects and one or more candidate objects. Similarity can be expressed as the probability that a candidate article (or one or more aspects thereof) is an analyte, such as a parasite (or one or more aspects thereof). In some embodiments, the object classifier module 340 (machine learning classifier) can be configured to average the output of the machine learning classifier by over the feature vector corresponding to the enhanced version of each of the input image patches ( For example, probability) to classify one or more feature vectors.

E. Diagnostic module

The diagnostic module 350 (Fig. 3A) can be configured to be based, at least in part, on the classified item data 341 (i.e., whether positive - the sample does contain malaria parasites, or negative - the sample does not contain malaria parasites) The diagnostic result 351 for the sample (eg, blood slide) is determined and output. Diagnostic result 351 can include an estimate of parasitemia (as used in Equation 10 below) ). In some embodiments, the diagnostic module 350 can be configured to determine parasitaemia. In some embodiments, the diagnostic module is configured to run for its object classifier score above a certain threshold Candidate object The number of calculations is performed for the diagnostic algorithm. In some embodiments, more than one type of candidate item (eg, a circular malaria parasite and a late parasitic object) can be counted at the same time. Subsequently, the object classifier scores higher than The number of candidate objects at a certain level The lower is thresholded. In other words, if , the sample is marked as positive, otherwise it is marked as negative. Threshold with Optimization can be performed on a validation set whose diagnostic results are known, or by a human expert through microscopy or molecular testing such as polymerase chain reaction (PCR). Optimization is based, at least in part, on a given goal for the validation set, such as maximizing balance accuracy, or maximizing sensitivity at a fixed level of specificity.

The image analysis system disclosed herein (as a real world system) may have some residual noise that depends on the threshold applied to the object classifier. In other words, under the threshold of some object classifiers, some non-parasitic objects will have a score above the threshold. In some embodiments, a median object level negative positive rate is calculated as a function of the object classifier score threshold on the negative sample of the validation set. . At the same time, calculate the same classifier threshold on the positive sample of the validation set The median object-level real rate of the function. Subsequently, the estimated protozoa rate is calculated using Equation 10 as follows: among them Is above the threshold The number of candidate objects for the classifier score. To understand, Object classifier score threshold The function. Classifier score threshold It is determined by optimizing a given target (such as the protozoal rate mean variance) over the entire validation set.

F. System hardware

Figure 12 is a schematic illustration of a system 1200 for determining the presence of an analyte in a sample, in accordance with one embodiment. In some implementations, system 1200 can be configured to perform one or more of any of the algorithms or other operations disclosed herein. The system can include a computing device 1202. In some implementations, computing device 1202 can include at least one memory storage medium 1210 and at least one processor 1220. In some implementations, computing device 1202 can include a user interface 1230. System 1200 can include an imaging device 1240 operatively coupled thereto. Aspects of system components are described in more detail below.

In some implementations, computing device 1202 can include one or more of a personal computer, a computer network, one or more servers, a laptop, a tablet, or a cellular phone. In some embodiments, one or more of the elements in computing device 1202 can be integrated into a microscope (imaging device). In some embodiments, one or more elements of the computing device can be located at a remote location of the imaging device. In such an embodiment, one or more elements of computing device 1202 can be operatively coupled to imaging device 1240 by a wired or wireless connection 1206. In some embodiments, one or more elements of the computing device can be configured to receive an image indirectly captured by the imaging device, such as by a compact disc, a flash memory drive, an email, or other means.

The at least one memory storage medium 1210 can include one or more of a hard disk drive, a solid state hard disk, a magnetic disk, or any other tangible, non-volatile memory storage device. The at least one memory storage medium 1210 can include any of the modules or sub-module machines on which the readable and executable programs are stored as disclosed herein. In some embodiments, system 1200 can include a plurality of memory storage media 1210, each memory storage medium 1210 having one or more modules or sub-modules stored thereon.

At least one processor 1220 can be configured to read and execute one or more programs stored on the at least one memory storage medium 1210. For example, at least one processor 1220 can be configured to read and execute one or more of any of the modules or sub-modules disclosed herein. In some implementations, at least one processor 1220 can include multiple processors. In such an embodiment, each of the plurality of processors can be configured to read and execute one or more modules or sub-modules stored on the at least one storage medium 1220. In some embodiments, each of the plurality of processors 1220 can be operatively coupled to one of the respective plurality of memory storage media 1220 and dedicated and configured to run only the modules or sub-modules herein. one of the.

In some embodiments, the user interface 1230 can include a display screen, a keyboard, a touch screen, one or more indicators (eg, lights, buzzers, speakers, etc.) or one or more buttons (eg, a power source) Or one or more of the start buttons). In some embodiments, the user interface can be physically connected to the computing device. In some implementations, the user interface 1230 can be configured to display an output or input from any of the modules or sub-modules disclosed herein. For example, the user interface 1230 can be configured to display one or more of diagnostic results, protozoa rates, or any material or image disclosed herein. In some embodiments, the user interface can be configured to receive input from a user, such as via a keyboard, USB port, or the like. The user interface 1230 can be operatively coupled to the computing device via a wired or wireless connection. In some embodiments, the user interface 1230 can be located at a remote location of the computing device 1202, such as on a computer remote from the computing device 1202, on a tablet, or on a cellular phone. In such an embodiment, one or more modules may be remotely located from the user interface 1202.

In some implementations, computing device 1202 can include a power source 1208. Power source 1208 can include one or more batteries (eg, a lithium ion battery, a lead acid battery, a nickel cadmium battery, or any other suitable battery), a solar cell, or an electrical plug (eg, a wall plug). Power source 1208 can be operatively coupled to any of the elements of system 1200 and configured to provide power to the element.

Imaging device 1240 can include a microscope having a digital image recorder thereon, such as a high power microscope. Digital imaging device 1240 can be configured to hold sample slide 1250 thereon. Digital imaging device 1240 can include a high power lens and a digital image recorder to capture one or more high resolution images of the sample slide. The one or more high resolution images may include an image of one or more FoVs of the sample slide 1250 and an image of one or more focal planes of each FoV. The imaging device can be directly connected (eg, wired or wirelessly connected) or indirectly connected (eg, via a computer network) to a computing device (eg, one or more memory storage media, one or more processors connected to the computing device) Or one or more of the user interfaces). In such an embodiment, imaging device 1240 can be configured to output one or more sample images to at least one memory storage medium 1210 or at least one processor 1220. In some implementations, imaging device 1240 can be configured to respond to one or more instructions from a computing device (or an element thereof, such as a processor). In such an embodiment, imaging device 1240 can operate based at least in part on operational instructions stored on at least one memory storage medium 1210 and executed by at least one processor 1220. For example, imaging device 1220 can change the distance or number between focal planes or FoV based, at least in part, on instructions from computing device 1202.

Any of the individual modules or sub-modules disclosed herein may be stored, included in a machine learning device or computer as disclosed herein, or applied using a machine learning device or computer as disclosed herein.

In some embodiments, a computer system for determining the presence of an analyte in blood can include at least one memory storage medium configured to store a plurality of images of a sample slide. The plurality of images may comprise a plurality of fields of view, each field of view comprising a unique x and y coordinate of the sample slide and a plurality of focal planes, each focal plane having a unique z coordinate of the sample slide. The memory storage medium can include operational instructions (eg, one or more modules) stored therein. The computer system can include at least one processor operatively coupled to the at least one memory storage medium. The at least one processor can execute one or more machine readable instructions stored in a memory storage medium. The one or more machine readable instructions can include one or more modules or sub-modules as disclosed herein that can be executed by a single processor or each executed by a separate processor dedicated to the module. The at least one processor can determine a white balance transform and apply the white balance transform to each of the plurality of images to efficiently generate a plurality of color corrected images as disclosed herein. The at least one processor can determine an adaptive grayscale transform and apply an adaptive grayscale transform to each of the plurality of images to provide an adaptive grayscale intensity map for each of the plurality of images disclosed herein image. As disclosed herein, the at least one processor can detect and identify one or more candidate objects in the color corrected image and the adaptive grayscale intensity image. As disclosed herein, at least one processor can perform an adaptive thresholding operation on the adaptive grayscale intensity image and output one or more candidate objects based thereon. The at least one processor may cluster the one or more detected candidate objects into a cluster comprising one or more neighboring candidate objects/clusters and associate (eg, aggregate) the cluster of detected candidate objects (indicating The cluster of one or more adjacent candidate objects is a single candidate object) and outputs the location of the cluster of one or more adjacent candidate objects. As disclosed herein, a location may include one or more image patches that include one or more adjacent candidate objects. As disclosed herein, the at least one processor can position the focal plane with the best focus for each individual candidate. As disclosed herein, at least one processor can determine an attribute (eg, color, roundness, shape) of each individual candidate object in each of the focal planes that have the best focus for each individual candidate object. As disclosed herein, the at least one processor can filter each individual candidate object based at least in part on one or more determined attributes. As disclosed herein, the at least one processor can extract and output one or more image patches, each image patch comprising at least one filtered single candidate object of one or more candidate objects.

The system disclosed herein may include a candidate object detection module and a spot detection module (eg, having a threshold determination sub-module and a spot recognition sub-module) to locally estimate an adaptive gray-scale intensity map as disclosed herein. Locally adapting the gray level in at least some of the plurality of fields of the adaptive grayscale image and the plurality of windows of the plurality of focal planes in the one or more windows of the image Threshold. As disclosed herein, a candidate object detection module (eg, a spot recognition module therein) can identify one or more spots in the adaptive gray intensity image based at least in part on the local adaptive threshold.

The computer system disclosed herein can include a machine readable program for booting (and the system can perform) any of the actions disclosed herein. The system can include one or more imaging devices (eg, a microscope with a camera). Such systems can provide automatic detection of parasites (eg, malaria) in samples that are much lower in concentration than currently used. Such systems may allow for early detection (eg, hypoparasemia) and early treatment of diseases (eg, malaria) that were previously not achievable with automated systems. The system in this paper enables reliable and early detection of parasites without the availability of trained human microscopy specialists.

G. Method of diagnosing analytes

FIG. 13 is a flow diagram of a method 1300 for determining the presence of an analyte in a sample, in accordance with an embodiment. The methods and individual actions for diagnosing analytes in a sample are also as described above with respect to each of the modules and sub-modules disclosed herein, and for the sake of brevity, are not described literally with respect to method 1300. Method 1300 includes using a plurality of images of a sample slide to determine the presence of an analyte in a sample. Method 1300 can include an act 1305 of receiving a plurality of images of a sample slide using a memory storage medium or processor. The plurality of images can include: a plurality of FoVs each including a unique x and y coordinate of the sample slide; and a plurality of focal planes, each having a unique z coordinate of the sample slide. Method 1300 can include performing any of the actions disclosed herein using one or more elements of system 1200.

The method 1300 can include an act 1310 of applying a white balance transform to each of the plurality of images to effectively generate a plurality of color corrected images. Method 1300 can include an act 1320 of applying an adaptive grayscale transform to each of the plurality of images to provide an adaptive grayscale intensity image for each of the plurality of images. Method 1300 can include an act 1330 of detecting and identifying one or more candidate objects in a plurality of color corrected (eg, white balanced) images and adaptive grayscale intensity images. Method 1300 can include filtering one or more candidate objects based at least in part on a score of one or more of its characteristics and outputting one or more color corrected image patches and one or more adaptive grayscale intensity image patches Action 1340. Method 1300 can include an act 1350 of extracting one or more feature vectors from color corrected image patching and adaptive grayscale intensity image patching and outputting the one or more feature vectors. Method 1300 can include an act 1360 in which each feature vector is classified corresponding to an artifact or analyte. Method 1300 can include an act 1370 of determining whether the classified feature vector is above or below a threshold level associated with a positive diagnosis. Each action 1310 through 1370 will be discussed in more detail below.

The act of applying white balance to each of the plurality of images to effectively generate a plurality of color corrected images may be performed using any of the techniques disclosed with respect to the image preprocessing module 310 disclosed above. . For example, act 1310 can include selecting a plurality of brightest pixels from a subset of the selected plurality of images such that the probability that the clear pixels are present in the subset is close (substantially) 1, as disclosed herein. Act 1310 can include calculating and applying a standard grayscale intensity of each pixel of a subset of the plurality of images to determine a plurality of brightest pixels within each of the subset of the plurality of images as disclosed herein. Act 1310 can include determining a red value R, a green value G, and a blue value B for each of the plurality of brightest pixels as disclosed herein. Act 1310 can include calculating an average color vector defined by an average color of the plurality of brightest pixels as disclosed herein. Act 1310 can include determining a white vector and determining an axis vector that is normal to both the average color vector and the white vector and is computed by the intersection of both the average color vector and the white vector. Act 1310 can include calculating an affine transformation matrix from an axis vector and an angle between the white vector and the average color vector; and applying an affine transformation matrix to each pixel in each of the plurality of images to A plurality of color corrected images are provided.

An act 1320 of applying adaptive grayscale transformation to each of the plurality of images to provide an adaptive grayscale intensity image for each of the plurality of images may use an image preprocessing module as disclosed above Any of the techniques disclosed in 310 are performed. For example, act 1320 can include receiving a plurality of color corrected images and standard grayscale intensity images as inputs, and thresholding the standard grayscale intensity image at the selected dark threshold to detect potentially potential It is a nuclear spot of white blood cells. Act 1320 can include filtering potential white blood cell nuclear spots by attributes (eg, color, area, or shape screening programs) to determine white blood cell nuclei as disclosed herein. Act 1320 can include outputting a red value R, a green value G, and a blue value B of one or more pixels from the color corrected image of the input of the white blood cell nuclei as white blood cell vector data. Act 1320 can include outputting, as the background vector data, a red value R, a green value G, and a blue color of the plurality of qualified background pixels determined by random sampling of pixels in which the gray level intensity is darker than the dark threshold in the color corrected image Value B. Act 1320 can include determining an adaptive grayscale projection vector from the white blood cell vector data and the background vector data. Act 1320 can include outputting a plurality of adaptive grayscale intensity images.

The act 1330 of detecting and identifying one or more candidate objects in the plurality of color corrected images and the adaptive grayscale intensity image may use any of the techniques disclosed with respect to the disclosed candidate object detection module 320. One to carry on. For example, detecting and identifying one or more candidate objects can include determining one or more potential analytes based on one or more of the plurality of color corrected images or the plurality of adaptive gray intensity images. position. Act 1330 can include determining which of the plurality of FoVs include one or more candidate objects therein. Act 1330 can include a cluster of one or more candidate objects therein to provide a cluster of candidate objects defined by adjacent (eg, adjacent or overlapping) candidate objects therein. The cluster is based, at least in part, on proximity or distance between candidate objects. Act 1330 can include determining a focal plane having an optimal focus score for each of the one or more candidate objects, as disclosed herein.

An act 1340 of filtering one or more candidate objects based at least in part on a score of one or more of its characteristics and outputting one or more color corrected image patches and one or more adaptive grayscale intensity image patches may This is done using any of the techniques disclosed with respect to the candidate object detection module 320 disclosed above. Act 1340 can include outputting a score for one or more characteristics of each of the one or more candidate objects, the one or more characteristics including at least one of area, grayscale intensity, shape, or color. Act 1340 can include filtering the candidate object based at least in part on a score based at least in part on the one or more characteristics. Filtering the one or more candidate objects can include comparing a score based at least in part on one or more characteristics of the one or more candidate objects to a threshold based at least in part on the one or more characteristics. Filtering the candidate object may include outputting one or more candidate objects having a score above a threshold score as a location of the potential analyte and rejecting one or more candidate objects having a score below a threshold score. Act 1340 can include outputting an adaptive grayscale and color corrected image patch and an associated focal plane having a potential analyte location therein.

The act 1350 of extracting one or more feature vectors and outputting one or more feature vectors from color corrected image patching and adaptive grayscale image patching may be disclosed using the feature extraction module 330 disclosed above. Any of the techniques are carried out. For example, act 1350 can include receiving, as input, a plurality of color corrected image patches and a plurality of adaptive grayscale image patches corresponding to one or more potential analyte locations in the plurality of images and outputting respective One or more feature vectors representing potential analytes. Act 1350 can include receiving one or more color corrected image patches and one or more adaptive grayscale intensity image patches, and teaching CNN groups based at least in part on one or more ground truth image patches Weights. In some embodiments, teaching weights in groups includes enhancing one or more ground live images (eg, image patching) using a data enhancement scheme. The data enhancement scheme may include random gamma correction of one or more of the red, green, blue, or grayscale intensity components of the ground live image patch. In some embodiments, teaching the group of weights to the CNN can include accepting one or more labeled images of the analyte in the ground truth sample as ground truth and one or more artifacts in the ground truth sample Annotated image. The annotated images can include known analytes and artifacts that are configured to train the CNN to identify characteristics of the known analytes and artifacts. In some embodiments, one or more of the labeled images of the analyte in the ground truth sample as ground truth and one or more labeled images in the ground truth sample may include at least a portion The weighting of the grouping of machine learning classifiers is taught based on one or more ground truth images. Act 1350 can include determining and extracting one or more of one or more of the plurality of color corrected images and the plurality of adaptive grayscale intensity images corresponding to the one or more potential analyte locations Features (eg, one or more manual features or automatic features). Act 1350 can include representing one or more extracted features as one or more feature vectors.

The act of classifying each feature vector as a classification corresponding to an artifact or analyte 1360 can be performed using any of the techniques disclosed with respect to the object classifier module 340 disclosed above. For example, act 1360 can include receiving one or more feature vectors as input candidate objects and classifying one or more feature vectors as corresponding to artifacts or analytes. Classification may be performed on a feature vector score by a machine learning classifier that has been trained using a set of ground truth images or associated vectors as disclosed above, where high scores (eg, high probability) are classified as analysis Things, low scores (eg, low chances) are classified as something other than analytes, such as backgrounds or artifacts. In some embodiments, classifying one or more feature vectors may include averaging a score of a machine learning classifier on a feature vector corresponding to an enhanced version of the color corrected image patch and adaptive gray intensity image patching . In some implementations, the method can include outputting one or more image patches (eg, classified as analytes or artifacts) in which the candidate object is included for review by the user. This image patching can be output to a user interface, such as a computer screen.

The act 1370 of determining whether the classified feature vector is above or below a threshold level associated with a positive diagnosis can be performed using any of the techniques disclosed in the diagnostic module 350 disclosed above. For example, determining whether the classified analyte is above or below a threshold level associated with a positive diagnosis can include determining whether an analyte is present and giving an analysis based on the amount of one or more feature vectors classified as an analyte. An indication of the presence or absence of an object, or its relationship to a threshold or background noise value. In one embodiment, method 1300 can include outputting a diagnostic result or analyte concentration to, for example, a user interface (eg, a diagnostic result showing an analyte concentration).

In some embodiments, method 1300 can include the act of obtaining a sample, such as obtaining a blood sample, from a subject. In some embodiments, method 1300 can include applying a sample to a sample slide. In some embodiments, method 1300 can include acquiring a plurality of images of a plurality of sample slides. Multiple (sample) images may include multiple FoV and focal planes. In one embodiment, method 1300 can include outputting a plurality of images from the imaging device (sample). Method 1300 can include receiving a plurality of (sample) images at a computing device.

In some embodiments, method 1300 can include determining the concentration or amount of analyte in a sample (eg, Plasmodium). In some embodiments, the analyte can include a parasite such as malaria, filaria, Helicobacter, helminth, tuberculosis, trypanosomiasis, or any other parasite. In some embodiments, the systems and methods herein can be used to detect the conformation or species of a particular parasite (eg, malaria) based on one or more of their characteristics.

In a simplified form, a method of detecting an analyte in a sample can include accepting a set of labeled images of an analyte (eg, a malaria parasite) in a biological sample from a geographic location as a ground truth. The method can include accepting a set of uncharacterized images from an automated microscope device, an uncharacterized image obtained from a biological sample acquired at a geographic location. The method can include pre-processing a set of uncharacterized images to create an image of a set of consistent color appearances. The method can include subjecting the images of the set of consistent color appearances to candidate location classification to generate an image of the set of candidate objects. The method can also include subjecting the images of the set of candidate objects to parasitic detection classification based in part on the ground truth to produce a set of labeled objects. The method can include subjecting the set of labeled articles to a segmented analysis of structures (eg, nuclei and cytoplasm) depicted in each of the sets of labeled objects. The method can include performing a feature extraction analysis on each of the set of marked objects. The method can further include classifying each of the marked items with a classifier score that relates to the probability of the presence of an analyte (eg, a malaria parasite) in each labeled item. In some embodiments, the method 1300 can include inputting and coming from a memory based at least in part on meta-data corresponding to one or more of a geographic location, a season, or other criteria associated with the sample. The ground truth material associated with one or more candidate parasite species stored in the body, and the ground truth material is used to determine or identify the species, stage or type of parasite in the sample as disclosed above.

14 is a flow diagram of a method 1400 for determining the presence of an analyte in a sample. Methods and single actions for diagnosing analytes in a sample are also described above with respect to each of the modules and sub-modules disclosed herein, and are not repeated verbatim relative to method 1400 for the sake of brevity. Method 1400 can include an act 1410 of receiving a plurality of images of a sample slide, the plurality of images including a plurality of fields of view, each field of view including a unique x and y coordinate of the sample slide; and a plurality of focal points Plane, each focal plane has a unique z coordinate of the sample slide. Method 1400 includes an act 1420 of applying a white balance transform to each of a plurality of images to efficiently generate a plurality of color corrected images. The method 1400 includes an act 1430 of applying an adaptive grayscale transform to each of the plurality of images to provide an adaptive grayscale intensity image for each of the plurality of images. Method 1400 includes an act 1440 of detecting and identifying one or more candidate objects in a plurality of color corrected images and adaptive grayscale intensity images. Act 1440 includes performing an adaptive thresholding operation on the adaptive grayscale intensity image and outputting one or more candidate objects based thereon. Act 1440 includes clustering the one or more detected candidate objects into a cluster comprising one or more candidate objects/clusters, and indicating that one or more adjacent candidate objects are detected candidate objects for a single candidate object The cluster is associated and outputs the location of the cluster of one or more adjacent candidate objects, the location including one or more image patches of the cluster containing the one or more adjacent candidate objects. Act 1440 includes identifying the focal plane with the best focus for each individual candidate object; determining the properties of each individual candidate object in the focal plane with each of the individual candidates having the best focus. Act 1440 includes filtering each individual candidate object based at least in part on one or more determined attributes. Act 1440 includes extracting and outputting one or more image patches, each image patch comprising at least one filtered single candidate object of the one or more candidate objects. In an embodiment, one or more actions of method 1400 may be omitted or performed in a different order than provided above. For example, act 1410 can be omitted.

Method 1400 can include an act 1410 of receiving a plurality of images of a sample slide, the plurality of images including a plurality of fields of view, each field of view including a unique x and y coordinate of the sample slide; and a plurality of focal points Plane, each focal plane has a unique z coordinate of the sample slide. In an embodiment, receiving the plurality of images of the sample slide can include receiving a plurality of images from a microscope associated with a computer vision system, such as system 1200 or any of the systems disclosed herein. In an embodiment, receiving the plurality of images of the sample slide can include receiving a plurality of images at the image pre-processing module.

Method 1400 includes an act 1420 of applying a white balance transform to each of a plurality of images to efficiently generate a plurality of color corrected images. The act 1420 of applying a white balance transform to each of the plurality of images to effectively generate the plurality of color corrected images may be similar or identical to the act 1310 disclosed above in one or more aspects. For example, act 1420 of applying a white balance transform to each of a plurality of images to effectively generate a plurality of color corrected images can be performed using any of the techniques disclosed herein with respect to image pre-processing module 310. For example, act 1420 can include selecting a plurality of brightest pixels from a subset of the selected plurality of images such that the probability of the presence of clear pixels located in the subset is close (is substantially close) as disclosed herein. . Act 1420 can include calculating and applying a standard grayscale intensity of each pixel of the subset of images as disclosed herein to determine in each of the subset of the plurality of images The plurality of brightest pixels. Act 1420 can include determining a red value R, a green value G, and a blue value B for each of the plurality of brightest pixels as disclosed herein. Act 1420 can include calculating an average color vector defined by an average color of the plurality of brightest pixels as disclosed herein. Act 1420 can include determining a white vector and determining an axis vector that is perpendicular to both the average color vector and the white vector and is a cross product of both the average color vector and the white vector Calculation. Act 1420 can include calculating an affine transformation matrix from the axis vector and an angle between the average color vector and the white vector, and applying the affine transformation matrix to each of the plurality of images Each pixel in the image provides a plurality of color corrected images. In an embodiment, applying the white balance transform may include applying the white balance transform to a color of each of the plurality of images defined by a red value R, a green value G, and a blue value B therein The vector, and based on it, outputs a color corrected image.

The method 1400 includes an act 1430 of applying an adaptive grayscale transform to each of the plurality of images to provide an adaptive grayscale intensity image for each of the plurality of images. The act of applying an adaptive gamma transform to each of the plurality of images to provide an adaptive grayscale intensity image for each of the plurality of images may use any of the disclosures disclosed herein with respect to the image pre-processing module 310. Technology to implement. For example, act 1430 can include one or more of the following: receiving a plurality of color corrected images and a standard grayscale intensity image as inputs; thresholding the standard grayscale intensity image at a dark threshold To detect one or more spots; filter at least one of the detected color, area or shape of one or more spots to locate and identify white blood cell nuclei under high sensitivity and specificity; output as white blood cells The red data R, the green value G, and the blue value B of the vector data from one or more pixels of the color corrected image containing the white blood cell nuclei; the output as the background vector data for the color corrected Multiple qualified background pixels determined by random sampling of pixels in the image that are brighter than the dark threshold in grayscale intensity (or darker than the luminance threshold for non-inverted grayscale intensity images) A red value R, a green value G, and a blue value B; or an adaptive grayscale projection vector is determined based on the white blood cell vector data and the background vector data. Act 1430 can include applying an adaptive gamma transform to one or more of the plurality of images or portions thereof to provide one or more adaptive grayscale intensity images. In an embodiment, applying the adaptive gamma transform to the plurality of images may include, for example, outputting a plurality of adaptive grayscale intensity images to the candidate object detection module (or the threshold determination sub-module therein).

In an embodiment, applying adaptive grayscale transformation can include determining and applying the plurality of white blood cell pixels, a plurality of qualified background pixels, and regression (eg, using any of the regression techniques disclosed herein) as a vector. Adaptive grayscale projection. In an embodiment, applying the adaptive grayscale transform may include calculating an adaptive grayscale projection vector and applying the adaptive grayscale projection vector to each of the plurality of color corrected images to effectively provide more Adaptive grayscale intensity images. Applying an adaptive gray level transform may include receiving a plurality of color corrected images and a standard gray intensity image as inputs and determining localized one or more portions thereof (eg, window, FoV, image patching) Adapt to grayscale intensity. Act 1430 can include determining an adaptive grayscale transform of the image using the locally adaptive grayscale intensity.

Act 1430 can include filtering potential WBC nuclear spots by attributes (eg, color, area, or shape filters) to identify WBC cell nuclei disclosed herein. Filtering potential WBC nuclear spots by attributes can include thresholding a standard gray intensity image at a dark threshold to detect spots that may be WBC nuclei. Act 1430 can include outputting a red value R, a green value G, and a blue value B of one or more pixels of the input color corrected image from which the WBC cell core is contained, as WBC vector data. Act 1430 can include red value R, green as a background vector data output from a plurality of qualified background pixels determined for random sampling of pixels that are brighter in grayscale intensity than the dark threshold in the color corrected image The value G and the blue value B. Act 1430 can include determining an adaptive grayscale projection vector from the WBC vector data and the background vector data. Act 1430 can include outputting a plurality of adaptive grayscale intensity images and a WBC detection mask.

Method 1400 includes an act 1440 of detecting and identifying one or more candidate objects in a plurality of color corrected images and adaptive grayscale intensity images. Act 1440 can include performing an adaptive thresholding operation on the adaptive grayscale intensity image and outputting one or more candidate objects based thereon. Act 1440 can include clustering the one or more detected candidate objects into a cluster comprising one or more candidate objects/clusters, and indicating that one or more neighboring candidate objects are detected candidates for a single candidate object A cluster of objects is associated (eg, aggregated) and outputs a location of the cluster of one or more neighboring candidate objects, the location including one or more of the clusters comprising the one or more neighboring candidate objects Image patching. Act 1440 can include identifying the focal plane with the best focus for each individual candidate object and determining the properties of each individual candidate object in the focal plane with each of the individual candidate objects having the best focus. Act 1440 can include filtering each individual candidate object based at least in part on one or more determined attributes. Act 1440 can include extracting and outputting one or more image patches, each image patch comprising at least one filtered single candidate object of the one or more candidate objects.

Performing an adaptive thresholding operation on the adaptive grayscale intensity image and detecting the mask based on its output of one or more candidate objects (spots) may include determining an adaptive threshold for one or more windows of the FoV or image (eg, Local adaptive gray intensity threshold distinguished from adaptive grayscale image). For example, performing an adaptive thresholding operation can include determining an adaptive threshold using any of the thresholding techniques disclosed herein with reference to Figures 8A-8G. For example, performing an adaptive thresholding operation may include determining an adaptive (grayscale intensity) threshold of the image or portion thereof, and applying an adaptive threshold to determine if any pixels are above or below an adaptive threshold, such pixel indication The presence of objects of interest, such as candidate objects and/or spots.

In an embodiment, performing the adaptive thresholding operation can include receiving one or more adaptive grayscale intensity images and receiving one or more WBC detection masks, the one or more WBC detection masks including about the WBC Information on multiple fields of view and locations in multiple focal planes. Moreover, performing an adaptive thresholding operation can include determining one or more regions of the grayscale intensity of the one or more regions of the adaptive grayscale intensity image using one or more adaptive grayscale intensity images and a WBC detection mask. Adapt to the threshold. For example, determining a local adaptive threshold of grayscale intensities of one or more regions in the adaptive grayscale intensity image may include determining a plurality of fields of view and a plurality of focal planes in the adaptive grayscale image The locally adaptive (grayscale intensity) threshold of at least some of the plurality of windows, the adaptive grayscale image comprising at least some of the windows of the one or more candidate objects included therein, the determination being localized The noise floor of the at least some windows is estimated to be implemented. Determining the adaptive threshold may include determining a noise floor for the window, image patching, or FoV, and selecting an adaptive threshold based thereon. The adaptive threshold can be set at a noise floor or some value above or below the noise floor (eg, noise floor plus some incremental gray level intensity above).

Locally estimating the noise floor of at least some of the windows can be performed using the estimation techniques disclosed above with respect to Figures 8A-8G. For example, local estimation of noise floor in at least some of the windows can be performed by determining a median grayscale intensity of pixels in each of at least some of the windows. In an embodiment, locally estimating the noise floor of at least some of the windows may include determining a median gray level intensity of pixels in each of the at least some of the adaptive gray level images, the discount being due to the presence of the WBC Any resulting change in the median pixel grayscale intensity value, such as by replacing the pixel value of the WBC with a full image, full FoV, or full window median grayscale intensity value. For example, determining a median pixel grayscale intensity value in each of at least some of the adaptive grayscale intensity images may include receiving a field of view recorded in the plurality of fields of view (eg, from WBC detection) Information about the presence and location of the WBC within one or more of the identified windows in the mask. Determining the median pixel grayscale intensity value when the WBC is indicated as being present may include replacing the one or more of the fields of view with a replacement median grayscale pixel intensity value determined from all pixels in the field of view The pixels of the WBC that are included in a particular area of the identified window. In an embodiment, determining a median pixel grayscale intensity value in each of at least some of the adaptive grayscale intensity images may further comprise: replacing the median grayscale intensity value with the WBC pixel After replacement, a local median pixel grayscale intensity value for all pixels in each of the one or more identified windows is determined. This median grayscale intensity is the noise floor. The noise floor can be set to a local adaptive (grayscale intensity) threshold, or can be modified by some increment in the grayscale intensity to set the adaptive threshold above or below the noise floor. In an embodiment, determining the median pixel grayscale intensity value in each of the at least some of the adaptive grayscale intensity images may further comprise: outputting for each of the one or more identified windows Local adaptive threshold. The local adaptive threshold of the window, image patch, or FoV can be based on the local median grayscale intensity value therein. For example, the local adaptive threshold may be a local median grayscale intensity value (eg, a noise floor) or some value above or below the value.

In an embodiment, the method 1400 can also include applying a local adaptive threshold to each of at least some of the windows of the adaptive grayscale intensity image. In an embodiment, applying the local adaptive threshold to respective image patches in the plurality of fields of view comprises determining that one or more candidate objects are present in the respective image patch, the one or more candidate objects having a higher or lower value The grayscale intensity of the local adaptive threshold depends on whether the grayscale intensity of the image patch has been inverted. For example, applying a local adaptive threshold to each of at least some of the windows of the adaptive grayscale image and determining that one or more candidate objects are present in the corresponding image patch may include determining whether any of the pixels in at least some of the windows are A grayscale intensity value having a local adaptive threshold below (or above, for inverted grayscale intensity). A pixel having a grayscale intensity value below (or above, for a reverse grayscale intensity image) a local adaptive threshold may indicate that there is an object of interest (eg, a candidate object) at the pixel. For example, determining that one or more candidate objects are present in the respective image patching may include determining that there is one or more candidate objects having grayscale intensities below a local adaptive threshold (eg, grayscale intensity values from pixels corresponding thereto) Instructed). In an embodiment, determining that there is one or more candidate objects having a grayscale intensity lower than a local adaptive threshold may include determining each of the lower than the local adaptive threshold based on a dark threshold of the adaptive grayscale intensity image One or more candidate objects are present in the image patch (eg, pixels below a dark threshold indicate objects of interest). In an embodiment, the method may include: inverting a brightness of the adaptive gray-scale intensity image to generate a plurality of inverted gray-scale intensity images; determining, based on the plurality of inverted gray-scale intensity images Determining a local adaptive threshold; and determining, based on a brightness threshold of the plurality of inverted grayscale intensity images (eg, a brighter pixel indicating an object of interest), determining each map above the local adaptive threshold The one or more candidate objects are present in the patch. A pixel having a higher value (or a lower value depending on whether the gradation intensity is inverted) may be output as an object of interest (eg, a candidate object).

In an embodiment, it may be performed by the speckle detection sub-module disclosed herein that an adaptive gradation transformation is applied to each of the plurality of images to provide adaptive grayscale intensity for each of the plurality of images. The image, an adaptive thresholding operation is performed on the adaptive grayscale intensity image, and based on this output one or more candidate object cluster detection masks (eg, detection mask 811, FIG. 8B).

In an embodiment, the method 1400 can include clustering one or more detected candidate objects into a cluster comprising one or more candidate objects/clusters, and associating (eg, grouping) the detected clusters of candidate objects with Indicates that one or more adjacent candidate objects are a single candidate object. In an embodiment, method 1400 can include outputting a location of a cluster of one or more neighboring candidate objects (eg, spots), the location including one or more images of a cluster comprising one or more neighboring candidate objects repair. In an embodiment, the clustering of the detected candidate objects indicating that the one or more adjacent candidate objects are a single candidate object and outputting the location of the cluster of one or more adjacent candidate objects may include determining the plurality of Which fields of view of the field of view include one or more candidate objects therein, and clustering one or more candidate objects based at least in part on the distance between adjacent candidate objects of one or more candidate objects in the field of view, To provide a cluster of candidate objects defined by adjacent candidate objects therein.

In an embodiment, identifying a focal plane having the best focus for each individual candidate object can include determining a focal plane having the highest focus score for each image patch having each individual candidate object. In an embodiment, the method may further comprise (automatically) selecting a respective focal plane having the highest focus score for each candidate object and outputting the corresponding focal plane, for example to sub-module 840 (eg, for speckle attribute extraction) .

In an embodiment, the method 1400 can include determining an attribute of each individual candidate item in a focal plane having the best focus for each individual candidate item. Determining an attribute of each individual candidate object in a focal plane having the best focus for each individual candidate object can include determining an area of each individual candidate object in a focal plane having the best focus for each individual candidate object, One or more of roundness, shape, or grayscale intensity. In an embodiment, determining the attributes of each individual candidate object can include identifying one or more spots based on one or more of the determined attributes or features (eg, in a focal plane having the highest focus score). For example, method 1400 can include identifying the darkest spot in the focal plane with the highest focus score for each individual candidate object and assigning the darkest spot as a candidate for interest. The method 1400 can include identifying a most rounded spot in a focal plane having the highest focus score for each individual candidate object and assigning the most rounded spot as a candidate for interest. In an embodiment, the method 1400 can include outputting one or more determined attributes of each individual candidate object (in each of the focal planes with the best focus of each individual candidate object) and determining based on the one or more The attributes classify each individual candidate object as an artifact or candidate object.

In an embodiment, method 1400 includes filtering each individual candidate object based at least in part on one or more determined attributes. For example, filtering each individual candidate object based at least in part on one or more determined attributes can include using an artifact classifier configured to score each individual candidate object based at least in part on one or more determined attributes. In an embodiment, method 1400 can include determining a score for each individual candidate based on one or more determined attributes as disclosed herein. For example, determining a score can include rating one or more determined attributes based on known attributes corresponding to known analytes. Known attributes can be used by the artifact filtering sub-module (e.g., sub-module 850) as a template or standard to set a threshold score for the one or more attributes. For example, filtering each individual candidate object based at least in part on the one or more determined attributes can include determining a threshold score based on attributes of ground truth objects trained into the memory storage medium and accessed by the at least one processor, and Each individual candidate object is filtered based on the score of the determined attribute relative to the threshold score. In an embodiment, filtering each individual candidate based at least in part on the one or more determined attributes may include discarding a single candidate item having a score below a threshold score and retaining a single candidate item having a score above the threshold score . The one or more image patches of the retained single candidate object may include color corrected red, blue, and green images and an adaptive grayscale intensity image comprising the field of view and focal plane of at least one single candidate object. Small area.

In an embodiment, method 1400 can include extracting and outputting one or more image patches, each image patch comprising at least one filtered single candidate object of one or more candidate objects. For example, extracting and outputting may include extracting and outputting one or more image patches of a single candidate object that is retained (eg, retained based on a single candidate object having a score above a threshold score). In an embodiment, the extracting and outputting may include outputting one or more image patches of a single candidate object that retains feature extraction for a single candidate object (eg, a single candidate object that is retained).

In an embodiment, method 1400 can include filtering one or more candidate objects based at least in part on a score based at least in part on one or more features of the one or more individual candidate objects, and outputting one of each filtered candidate object Or multiple color corrected image patches and one or more adaptive grayscale intensity image patches. Method 1400 can include extracting the one or more feature vectors from the color corrected image patch and the adaptive grayscale intensity image patch and outputting one or more feature vectors. In an embodiment, extracting one or more feature vectors from the color corrected image patching and the adaptive grayscale intensity image patching may include receiving as input one or more of the plurality of images A plurality of color corrected image patches of the locations of potential analytes and a plurality of adaptive grayscale intensity image patches; and outputting one or more feature vectors each representing a potential analyte. In an embodiment, extracting one or more feature vectors from the color corrected image patch and the adaptive gray intensity image patch may include determining and extracting locations corresponding to the one or more potential analytes One or more features of the plurality of color corrected image patches and one or more candidate objects of the plurality of adaptive grayscale intensity image patches; and the one or more candidate objects The associated one or more features are represented as one or more feature vectors.

In an embodiment, determining and extracting one or more features (eg, attributes) of one or more candidate objects includes extracting one or more automatically learned features from one or more candidate objects. Extracting the automatically learned features can include teaching the machine learning modules to group the weights based at least in part on the ground truth image patching having one or more ground live objects therein. The one or more ground live objects may include samples of the analyte and/or samples of the artifacts. The machine learning module includes a convolutional neural network or any other machine learning module. In an embodiment, teaching the machine learning module can include receiving one or more annotated images of the analyte in the ground live sample and one or more annotated images of the artifacts in the ground live sample as ground truth. In an embodiment, teaching the weight of the machine learning modality group based at least in part on the ground truth image patching may include using a data enhancement scheme to enhance ground live image patching. For example, the data enhancement scheme can include a random gamma correction of one or more of the red, green, blue, or grayscale intensity components of the ground truth image patch as disclosed herein.

In an embodiment, extracting the one or more feature vectors from the color corrected image patch and the adaptive grayscale intensity image patching may include determining the containing content based at least in part on the best focus score An optimal focal plane for each of the plurality of color corrected image patches and adaptive gray intensity image patches of one or more candidate objects. The best focus score may include a highest score from a plurality of focus scores of the plurality of focal planes in an image patch having candidate objects therein. Extracting the one or more feature vectors from the color corrected image patch and the adaptive grayscale intensity image patch may include determining a scene in which each image patch of the candidate object is present a standard deviation of focus scores on all of the plurality of focal planes; and for each image based at least in part on a red change in the darkest portion of the candidate object between the plurality of focal planes patched by each image Patching determines the redshift score.

Method 1400 can include classifying each feature vector to correspond to an artifact or analyte. In an embodiment, classifying each feature vector to correspond to an artifact or an analyte comprises: receiving one or more feature vectors of the candidate object as an input and classifying the one or more feature vectors to correspond to the An artifact or one of the analytes. Method 1400 can include determining whether a feature vector classified as an analyte is above or below a threshold level associated with a positive diagnosis. In an embodiment, classifying each feature vector to correspond to an artifact or an analyte may comprise: using a machine learning classifier that outputs a score, the score representing one or more feature vectors of the one or more candidate objects Each of them corresponds to an analyte.

In an embodiment, outputting the color corrected image patch and the adaptive grayscale intensity image patching may comprise enhancing the color corrected image patch and the adaptive grayscale using a data enhancement scheme Intensity image patching, and classifying the one or more feature vectors may include features in an enhanced version corresponding to each of the color corrected image patch and the adaptive grayscale intensity image patch The output of the machine learning classifier is averaged over the vector. Any data enhancement scheme used herein, such as the color corrected image patch or the color corrected red, green, blue, or adaptive grayscale intensity component in the adaptive grayscale image patching, can be used. Random gamma correction of one or more of them.

In an embodiment, determining whether the feature vector classified as corresponding to the analyte is above or below a threshold level associated with the positive diagnosis can include: based on one classified as an analyte (eg, a parasite) Or a magnitude of the plurality of feature vectors to determine if an analyte is present and output an indication of the presence or absence of the analyte. In embodiments, the methods herein may include at least in part based on one or more image characteristics (eg, determined attributes of candidate objects) including one or more of the shape, size, or color of the candidate object. Identify the type of one or more candidate objects.

In embodiments, the methods disclosed herein can include recording one or more images of one or more sample slides (eg, blood slides) with a microscope.

In an embodiment, the methods disclosed herein may be performed using at least one memory storage medium comprising one or more modules and/or sub-modules as disclosed herein. For example, each of the image preprocessing module, the candidate object detection module, the feature extraction module, the classification module, and the diagnostic module stored in the memory storage medium can be used as a computer readable program to execute the text. In the disclosed method, the computer readable program can be executed by at least one processor operatively coupled to the at least one memory storage medium. In an embodiment, the candidate object detection module may include a candidate object cluster (eg, speckle) detection sub-module, a candidate object (eg, a speckle) cluster sub-module, a best focus detection sub-module, and a candidate object cluster attribute extraction. The sub-module, the artifact filtering sub-module and the thumbnail extraction sub-module are as described above. In an embodiment, the candidate object detection cluster sub-module may include a threshold determination sub-module and a spot recognition sub-module as disclosed with reference to FIGS. 8A-8G.

In one embodiment, a method for determining the presence and/or concentration of an analyte in blood can include receiving a plurality of images of a sample slide, the plurality of images comprising: a plurality of fields of view, each The field of view includes unique x and y coordinates of the sample slide; and a plurality of focal planes, each focal plane having a unique z coordinate of the sample slide. The method can include applying a white balance transform to each of the plurality of images to effectively generate a plurality of color corrected images. The method can include applying an adaptive grayscale transform to each of the plurality of images to provide an adaptive grayscale intensity image for each of the plurality of images. The method can include detecting and identifying one or more candidate objects in the plurality of color corrected images and the adaptive grayscale intensity image, including performing adaptation on the adaptive grayscale intensity image Thresholding operations and based on this output one or more candidate object clusters. Performing the adaptive thresholding operation can include receiving one or more adaptive grayscale intensity images and receiving a white blood cell detection mask including information regarding locations of the white blood cells in the plurality of fields of view and the plurality of focal planes. Performing an adaptive thresholding operation can include determining the grayscale intensity of one or more regions in the adaptive grayscale intensity image using the one or more adaptive grayscale intensity images and white blood cell detection masks Local adaptive threshold. Determining the local adaptive threshold may include determining the local adaptive threshold of the plurality of fields of view and the at least some of the plurality of windows of the plurality of focal planes in the adaptive grayscale image, the The adaptive grayscale image includes at least some of the windows containing one or more candidate objects, the determination being achieved by locally estimating the noise floor of the at least some of the windows, the estimating being performed by determining the adaptive grayscale intensity A median grayscale intensity value in each of the at least some of the windows in the image, discounting any change in the median pixel grayscale intensity value due to the presence of white blood cells. Determining a median grayscale intensity value in each of the at least some of the plurality of views in the adaptive grayscale intensity image comprises: receiving one or more of the fields of view recorded in the plurality of fields of view Information on the presence and location of white blood cells within the identified window. Determining a median grayscale intensity value in each of the at least some of the windows in the adaptive grayscale intensity image comprises: when white blood cells are indicated as being present, from all of the fields of view The pixel determined replacement median grayscale pixel intensity value replaces pixels in the particular region of the one or more identified windows of the field of view that comprise the white blood cells. Determining a median grayscale intensity value in each of the at least some of the windows in the adaptive grayscale intensity image comprises: replacing the median grayscale pixel intensity value in the white blood cell containing the pixel After replacement, a local median grayscale intensity value for all pixels in each of the one or more identified windows is determined. Determining a median grayscale intensity value in each of the at least some of the windows in the adaptive grayscale intensity image comprises: outputting the local median grayscale intensity value based therein Or a local adaptive threshold for each of the plurality of identified windows. Detecting and identifying the one or more candidate objects in the plurality of color corrected images and the adaptive grayscale intensity image comprises: associating the cluster indicating one or more adjacent candidate objects as detected candidates of a single candidate object A cluster of objects and outputting the location of the cluster of one or more adjacent candidate objects, the location including one or more image patches including the one or more adjacent candidate objects. Detecting and identifying one or more candidate objects in the plurality of color corrected images and the adaptive grayscale intensity image comprises: identifying a focal plane having the best focus for each individual candidate object, and determining each individual candidate The object has the properties of each individual candidate object in the best focused focal plane. Detecting and identifying the one or more candidate objects in the plurality of color corrected images and the adaptive grayscale intensity image comprises: filtering each individual candidate object based at least in part on the one or more determined attributes; and extracting And outputting one or more image fixes, each image patch comprising at least one filtered single candidate object of one or more candidate objects. The method includes filtering the one or more candidate objects based at least in part on a score based at least in part on one or more characteristics of the one or more candidate objects, and outputting for each filtered single candidate object One or more color corrected image patches and one or more adaptive grayscale intensity image patches. The method includes extracting one or more feature vectors from the color corrected image patch and the adaptive grayscale intensity image patch and outputting the one or more feature vectors. The method includes classifying each feature vector to correspond to an artifact or an analyte. The method includes determining whether the feature vector classified as an analyte is above or below a threshold level associated with a positive diagnosis.

Any of the actions, system components, modules or sub-modules disclosed herein can be used with any of the embodiments disclosed herein.

The reader will be aware that the prior art has evolved to the extent that there is little difference between the hardware and software implementations of various aspects of the system; the use of hardware or software is generally (but not always, in some contexts, The choice between hardware and software becomes important) a design choice that represents a trade-off between cost and efficiency. The reader should understand that there are a variety of vectors (e.g., hardware, software, and/or firmware) that can function as described herein and/or systems and/or other techniques, and that the preferred carrier will be adapted to the method employed. And/or changes in the context of the system and/or other technologies. For example, if the implementer determines that speed and accuracy are of the utmost importance, then the implementer can select the primary hardware and/or firmware carrier; alternatively, if flexibility is most important, the implementer can choose primarily software. Implementation; or alternatively, the implementer may select some combination of hardware, software, and/or firmware. Thus, there are several possible vectors that can be implemented by the methods and/or devices and/or other techniques described herein, each of which is inherently superior to others, as any carrier to be utilized is dependent upon the carrier being employed The background and the specific considerations of the implementer (eg speed, flexibility or predictability) may change. The reader should be aware that the optical aspects of the implementation will typically employ optically oriented hardware, software and or firmware.

The foregoing detailed description has set forth various embodiments of the device and/ With respect to these structural diagrams, flowcharts, and/or examples, including one or more functions and/or operations, it should be understood that each function and/or operation within such a block diagram, flowchart, or example can be A wide range of hardware, software, firmware or virtually any combination thereof is implemented separately and/or collectively. In one embodiment, several portions of the subject matter described herein may be implemented by special application integrated circuits (ASICs), field programmable logic gate arrays (FPGAs), digital signal processors (DSPs), or other integrated formats. However, those of ordinary skill in the art to which the invention pertains will appreciate that some aspects of the embodiments disclosed herein may be implemented in whole or in part in an integrated circuit as one or more computers. One or more computer programs (eg, as one or more programs running on one or more computer systems), as one or more programs running on one or more processors (eg, As one or more programs running on one or more microprocessors, as a firmware or as virtually any combination thereof, and designing the circuit and/or writing code to the software and or firmware just happens to be in this Those having ordinary skill in the art to which the invention pertains are within the technical scope of the present disclosure. In addition, the reader should understand that aspects of the subject matter described herein can be distributed as various forms of programming products, and that illustrative embodiments of the subject matter described herein are applied regardless of the particular type of signal bearing media actually used to implement the distribution. . Examples of signal bearing media include, but are not limited to, the following: recordable media such as floppy disks, hard disk drives, compact discs (CDs), digital video discs (DVDs), digital tapes, computer memory, etc.; and transmission media For example, digital and/or analog communication media (eg, fiber optic cables, waveguides, wired communication links, wireless communication links, etc.).

In a general sense, the various embodiments described herein can be implemented individually and/or collectively by various types of electromechanical systems having a wide range of electrical components such as hardware, software, firmware or Any combination thereof; and a wide range of elements that can impart mechanical force or motion, such as rigid bodies, spring or torsion mechanisms, hydraulic and electromagnetic actuation devices, or any combination thereof. Thus, an "electromechanical system" as used herein includes, but is not limited to, an electronic circuit operatively coupled to a transducer (eg, an actuator, an electric motor, a piezoelectric crystal, etc.), an electronic circuit having at least one discrete circuit, An electronic circuit having at least one integrated circuit, an electronic circuit having at least one special application integrated circuit, a general purpose computing device configured to be programmed by a computer program (eg, by a computer program that at least partially performs the processes and/or devices described herein) An electronic circuit of a configured general purpose computer, or a microprocessor that at least partially performs the computer program configuration of the processes and/or devices described herein, an electronic circuit that forms a memory device (eg, in the form of a random access memory), An electronic circuit forming a communication device (eg, a data machine, communication switch, or optoelectronic device), and any non-electrical analog connected thereto, such as optical or the like. It will also be appreciated by those of ordinary skill in the art to which the present invention includes, but is not limited to, various consumer electronic systems and other systems (such as motorized transportation systems, factory automation systems, security systems, and communication/computing systems). Those of ordinary skill in the art will recognize that "electromechanical" as used herein is not necessarily limited to systems having both electrical and mechanical actuation, unless the context may otherwise specify.

In a general sense, some aspects described herein that can be implemented individually and/or collectively by a wide range of hardware, software, firmware, and/or any combination thereof can be considered to encompass various types of "electronics. Circuit". Accordingly, as used herein, "electronic circuit" includes, but is not limited to, an electronic circuit having at least one discrete circuit, an electronic circuit having at least one integrated circuit, and an electronic circuit having at least one special application integrated circuit formed by a computer program a computing device (eg, a general purpose computer configured by a computer program that at least partially performs the methods and/or devices described herein, or a computer program configured to at least partially perform the methods and/or devices described herein An electronic circuit of a microprocessor, forming an electronic circuit of a storage device (eg, in the form of a random access memory), and/or an electronic circuit forming a communication device (eg, a data machine, communication switch, optoelectronic device, etc.). The subject matter described herein can be implemented in an analogous or digital manner or some combination thereof.

The present disclosure has been made with reference to various exemplary embodiments. However, it will be appreciated by those of ordinary skill in the art that the present invention may be modified and modified without departing from the scope of the disclosure. For example, various operational steps and elements for performing the operational steps can be implemented in alternative ways depending on the particular application or consideration of any number of cost functions associated with operation of the system; for example, one or more steps can be deleted, modified Or combined with other steps.

Furthermore, the principles of the disclosure (including components) of the present disclosure may be embodied in computer readable storage having computer readable code means embodied in a storage medium, as will be understood by those of ordinary skill in the art to which the present invention pertains. Computer program products on the media. Any tangible, non-transitory computer readable storage medium can be used, including magnetic storage devices (hard disks, floppy disks, etc.), optical storage devices (CD-ROM, DVD, Blu-ray, etc.), flash memory And/or the like. These computer program instructions can be loaded onto a general purpose computer, a special purpose computer or other programmable data processing device to produce a machine, such that instructions executed on a computer or other programmable data processing device create means for performing the specified function. . The computer program instructions can also be stored in a computer readable memory that can be booted by a computer or other programmable data processing device to operate in a particular manner, such that instructions stored in the computer readable memory produce articles of manufacture, including implementation implementations. The device for the specified function. The computer program instructions can also be loaded onto a computer or other programmable data processing device to perform a series of operational steps on a computer or other programmable device to produce a computer implemented process that causes the computer or other programmable The instructions executed on the design device provide steps for implementing the specified functions.

In one embodiment, the printing system disclosed herein can be integrated in such a way that the printing system operates as a unique system of printing configuration (eg, three-dimensional printing) dedicated configuration, and any associated printing system The computing device operates as a dedicated computer for the purpose of the claimed system, rather than as a general purpose computer. In one embodiment, at least one associated computing device in the printing system operates as a dedicated computer for the purposes of the claimed system, rather than as a general purpose computer. In one embodiment, at least one of the associated computing devices of the printing system is hardwired with a particular ROM to indicate the at least one computing device. In one embodiment, those of ordinary skill in the art to which the present invention pertains recognize that the printing apparatus and printing system affect improvements in at least the technical field of three-dimensional printing.

For the sake of conceptual clarity, the elements (eg, steps), devices, and articles described herein, as well as the discussion accompanying them, are used as examples. Therefore, the specific examples and accompanying discussion set forth herein are intended to represent their more general categories. In general, any specific example used herein is also intended to represent a category, and such specific elements (e.g., steps), devices, and articles are not to be construed as limiting.

The reader is to be construed as a singular and/or singular as the singular as the singular and/or singular. For the sake of clarity, various singular/plural permutations are not explicitly set forth herein.

The subject matter described herein sometimes illustrates different elements included in other different elements or different elements connected to other different elements. It should be understood that this description architecture is merely exemplary and, in fact, many other architectures that achieve the same functionality can be implemented. In a conceptual sense, any component setup that achieves the same function is effectively "associated" to achieve the desired functionality. Accordingly, any two elements herein combined to achieve a particular function can be seen as "associated" with each other in order to obtain the desired functionality, regardless of the architecture or the intermediate components. Likewise, two elements so associated are also considered to be "operably connected" or "operably coupled" to each other to obtain the desired function, and any two elements that can be associated are also Operately coupled to achieve the desired functionality. Specific examples of operably coupleable include, but are not limited to, physically compatible and/or physically interacting elements; and/or wirelessly interactable, and/or wirelessly interacting elements And/or logically interacting elements, and/or logically interactable elements, and the like.

In some cases, one or more elements may be referred to herein as "configured to." The reader should recognize that "configured to" may generally include active state elements and/or inactive state elements and/or standby state elements, unless the context requires otherwise.

While the specific aspects of the subject matter described herein have been illustrated and described, it is apparent that changes and modifications may be made without departing from the scope of the subject matter described herein. All such changes and modifications that fall within the true spirit and scope of the subject matter described herein are intended to be included within the scope of the appended claims. In addition, it is to be understood that the invention is defined by the scope of the appended claims. In general, the terms used herein, and particularly the terms in the scope of the appended claims (e.g., the subject matter of the appended claims), are generally intended to be "open" terms (for example, the term "comprising" should be understood to mean "Including, but not limited to," the term "having" is understood to mean "having at least" and the term "comprising" is understood to mean "including but not limited to," and the like. It should be further understood by those of ordinary skill in the art to which the present invention is directed, if it is intended to represent a particular number of claims that are introduced, the meaning is explicitly stated in the claim, and in the absence of such a representation, There is no such meaning. For example, to assist understanding, the scope of the following appended claims may include the use of the terms "at least one" and "one or more" to introduce the claim. However, the use of such phrases should not be construed as implying that the indefinite article "a (a)" or "an" is intended to mean that any particular claim that includes the recited claim is limited to The invention includes only one such expression, even when the same claim includes the phrase "one or more" or "at least one" and, for example, the <RTI ID=0.0> / or "an" should generally be understood as "at least one" or "one or more" as well; the same is true for the use of the definite article used to introduce the expression of the claim. Moreover, even if the specific number of expressions of the introduced claim item is expressly stated, such expression should generally be understood to mean at least the number of expressions (eg, the straightforward expression of "two expressions", without other modifiers, Usually means at least two expressions, or two or more expressions). Further, in the case of using idioms similar to "at least one of A, B, and C, etc.", such a structure is generally intended to mean a conventional meaning (for example, "having at least one of A, B, and C" "System" will include, but is not limited to, systems with only A, systems with only B, systems with only C, systems with both A and B, systems with both A and C, systems with both B and C, and/or Or there are systems of A, B and C, etc.). In the case of using idioms similar to "at least one of A, B or C, etc.", such a structure is generally intended to mean a conventional meaning (for example, "a system having at least one of A, B or C" Will include, but are not limited to, systems with only A, systems with only B, systems with only C, systems with both A and B, systems with both A and C, systems with both B and C, and/or A, B and C systems, etc.). Substantially any antisense conjunction and/or phrase providing two or more alternative terms, whether in the scope of the specification or the drawings, should be understood to include any of the terms, any one of the terms or The possibility of two terms. For example, the phrase "A or B" is generally understood to include the possibility of "A" or "B" or "A and B."

The operations recited therein can generally be performed in any order, for the scope of the appended claims. Examples of such alternative sequences may include overlapping, interleaved, interrupted, reordered, incremental, preliminary, supplemental, simultaneous, reversed, or other ordering of changes, unless the context indicates otherwise. For context, even adjectives such as "in response to", "involving" or other past tense generally do not exclude such variants unless the context indicates otherwise.

The various aspects and embodiments disclosed herein are intended to be illustrative, and not restrictive, and the true scope and spirit of the invention is specified by the following claims.

201‧‧‧chromatin spots

202‧‧‧ cytoplasm

300‧‧‧ system

301‧‧‧ images

310‧‧‧Image Preprocessing Module

311‧‧‧ Output image

320‧‧‧ Candidate Object Detection Module

321‧‧‧Image repair

330‧‧‧Feature Extraction Module

331‧‧‧Character Vector

340‧‧‧ Object Classifier Module

341‧‧‧Classified object information

350‧‧‧Diagnostic Module

351‧‧‧Diagnosis

400‧‧‧Submodule

401‧‧‧ subset

410‧‧‧Submodule

411‧‧‧ Grayscale intensity image

420‧‧‧Submodule

421‧‧‧ binary image

430‧‧‧Submodule

431‧‧‧ pixel value

440‧‧‧Submodule

441‧‧‧ pixel value

450‧‧‧Submodule

451‧‧‧ pixel value

460‧‧‧Submodule

461‧‧‧White balance transformation parameters

470‧‧‧Submodule

471‧‧‧ images

480‧‧‧Submodule

481‧‧‧Regression model

490‧‧‧Submodule

491‧‧‧ Images

, , n ‧‧‧ vector

Θ‧‧‧ angle

601‧‧‧Background pixels

602‧‧‧ white blood cell nuclear pixels

603‧‧‧ Parasite nuclear nuclei

611‧‧‧Background pixels

612‧‧‧WBC nuclear pixel

613‧‧‧ Parasite nuclear pixels

810‧‧‧Submodule

811‧‧‧Detection mask

812‧‧‧Submodule

814‧‧‧Submodule

820‧‧‧Submodule

821‧‧‧ object cluster

830‧‧‧Submodule

831‧‧ Best Focus

840‧‧‧Submodule

841‧‧‧ attributes

850‧‧‧Submodule

851‧‧‧ artifacts

860‧‧‧Submodule

861‧‧‧Image repair

870‧‧‧FoV

870’ ‧‧‧

871‧‧‧High noise area

872‧‧‧WBC

874‧‧‧WBC

876‧‧‧ Parasite

878‧‧‧ Parasite

880‧‧‧WBC

881‧‧‧Low noise area

882‧‧‧ Parasite

884‧‧‧ artifacts

886‧‧‧ Parasite

899‧‧‧ Path

890‧‧‧Window

892‧‧‧ objects

894‧‧‧ objects

896‧‧‧ objects

900‧‧‧ Bar Chart

901‧‧‧Enter

904‧‧‧solid line

908‧‧‧Bumps

910‧‧‧Submodule

911‧‧‧Manual features

Line 912‧‧

914‧‧‧ line

916‧‧‧Area

921‧‧‧Image repair

930‧‧‧Feature Extractor

931‧‧‧CNN ingredients

941‧‧‧ Output feature vector

950‧‧‧ graphics

965‧‧‧Horizontal position

974‧‧‧ peak

976‧‧‧ peak

978‧‧‧ peak

982‧‧‧ peak

984‧‧‧ peak

986‧‧‧ peak

1001‧‧ plane

1002‧‧‧ plane

1003‧‧‧ plane

1004‧‧‧ plane

1005‧‧‧ plane

1010‧‧‧ lens

1020‧‧ Achromatic components

1030‧‧‧ Curve

1031‧‧ points

1032‧‧ points

1033‧‧‧ points

1040‧‧‧ Curve

1044‧‧‧ points

1045‧‧ points

1101‧‧‧ Curve

1200‧‧‧ system

1202‧‧‧ Computing equipment

1206‧‧‧Connect

1208‧‧‧Power supply

1210‧‧‧Memory storage media

1220‧‧‧ processor

1230‧‧‧User interface

1240‧‧‧ imaging device

1250‧‧‧Sample slide

1300‧‧‧ method

1305-1370‧‧‧ Action

1400‧‧‧ method

1410-1440‧‧‧ Action

Figure 1 is a life cycle diagram of malaria.

Figure 2A is a schematic illustration of a circular parasite.

Figure 2B is a schematic illustration of a circular parasite.

2C is a schematic diagram of multiple images in accordance with an embodiment.

3A is a schematic illustration of multiple modules of a system for automatically detecting and quantifying one or more analytes in a sample, in accordance with one embodiment.

FIG. 3B is a schematic diagram of multiple images input into a module of the system of FIG. 3A, in accordance with one embodiment.

3C is a schematic diagram of multiple images input into a module of the system of FIG. 3A, in accordance with one embodiment.

4 is a detailed schematic diagram of an image pre-processing module of the system of FIG. 3A, in accordance with an embodiment.

5 is a schematic diagram of the relationship between different vectors in the color value space of the red, green, and blue axes, according to one embodiment.

6A is a grayscale intensity bar graph for different pixels of different grayscale images, in accordance with one embodiment.

6B is a grayscale intensity bar graph for different pixels of different grayscale images, in accordance with one embodiment.

7 is a side-by-side comparison diagram of images in different FoVs having multiple focal planes, in accordance with one embodiment.

8A is a detailed schematic diagram of a candidate object detection module of the system of FIG. 3A, in accordance with an embodiment.

8B is a detailed schematic diagram of a spot detection sub-module of the candidate object detection module of FIGS. 3A and 8A, according to an embodiment.

Figure 8C is a FoV input image of the spot detection module of Figure 8B, in accordance with one embodiment.

Figure 8D is an already modified FoV input image of Figure 8C, in accordance with one embodiment.

Figure 8E is a grayscale intensity bar graph of a pixel of the FoV image of Figure 8C, in accordance with one embodiment.

Figure 8F is an illustration of a path through the FoV image of Figure 8C, in accordance with one embodiment.

Figure 8G is a graph of inverted grayscale intensity as a function of position on the path of Figure 8F, in accordance with one embodiment.

9 is a detailed schematic diagram of a feature extraction module of the system of FIG. 3A, in accordance with an embodiment.

Fig. 10A is a schematic view of light refracted to a focal plane by a simple lens and a lens with achromatic aberration correction.

10B is a schematic diagram of light refracted to different focal planes by a simple lens and a lens with achromatic aberration correction.

FIG. 10C is a graph of focus and wavelength curves for the simple lens and the achromatically corrected lens shown in FIGS. 10A and 10B.

Figure 11 is an absorption spectrum of a Giemsa-stained DNA sample according to one embodiment.

Figure 12 is a schematic illustration of a system for determining the presence of an analyte in a sample, in accordance with one embodiment.

Figure 13 is a flow diagram of a method for determining the presence of an analyte in a sample, in accordance with one embodiment.

14 is a flow chart of a method for determining the presence of an analyte in a sample, in accordance with one embodiment.

Claims (86)

A system for determining the presence of an analyte in blood, the system comprising: at least one memory storage medium configured to store a plurality of images of a sample slide, the plurality of images comprising: a plurality of a field of view, each field of view including a unique x and y coordinate of the sample slide; and a plurality of focal planes, each focal plane having a unique z coordinate of the sample slide; at least one processor capable of Operatively coupled to the at least one memory storage medium, the at least one processor configured to: determine a white balance transform and apply the white balance transform to each of the plurality of images to effectively generate a plurality of a color corrected image; determining an adaptive grayscale transform and applying the adaptive grayscale transform to each of the plurality of images to provide an adaptive grayscale intensity map for each of the plurality of images Detecting and identifying one or more candidate objects in the color corrected image and the adaptive grayscale intensity image, and the at least one processor is further configured to: gray The intensity image performs an adaptive thresholding operation and outputs one or more candidate objects based thereon; clustering the one or more detected candidate objects into a cluster comprising one or more adjacent candidate objects/clusters, and correlating A cluster indicating one or more adjacent candidate objects is a cluster of detected candidate objects of a single candidate object and outputs a location of the cluster of one or more adjacent candidate objects, the location including the one Or repairing one or more images of a plurality of adjacent candidate objects; locating the focal plane with the best focus of each individual candidate; determining the focal plane in which each individual candidate has the best focus An attribute of each individual candidate object; filtering each individual candidate object based at least in part on one or more determined attributes; and extracting and outputting one or more image patches, each image patch comprising the one or more At least one of the candidate objects is filtered by a single candidate. The system of claim 1, further comprising: a threshold determination module configured to determine a local adaptive threshold of grayscale intensity of one or more regions in the adaptive grayscale intensity image, and An image pre-processing module operatively coupled to receive one or more adaptive gray-scale intensity images therefrom; and a white blood cell detection module configured to receive a white blood cell detection mask therefrom, The white blood cell detection mask includes information about the location of white blood cells in the plurality of fields of view and the plurality of focal planes. The system of claim 2, wherein the threshold determination module is configured to determine in the plurality of views of the plurality of fields of view and the plurality of focal planes in the adaptive grayscale image The local adaptive threshold of grayscale intensity of at least some of the windows, the adaptive grayscale image comprising at least some windows, the at least some of the windows containing one or more candidate objects therein, the determination being by local estimation The noise floor of the at least some of the windows is implemented. The system of claim 3, wherein the threshold determination module is configured to locally estimate a noise floor of at least some of the windows by: determining the in the adaptive grayscale intensity image a median grayscale intensity value in each of at least some of the windows, discounting any change in the median grayscale intensity value due to the presence of white blood cells to each of the at least some of the windows The local adaptive threshold is generated in the windows. The system of claim 4, wherein the threshold determination module is configured to receive information recording the presence and location of white blood cells within one or more identified windows of the fields of view in the plurality of fields of view Replacing the inclusion in a particular region of the one or more identified windows of the field of view with a replacement median grayscale intensity value determined from all pixels in the field of view when indicating the presence of white blood cells a pixel of white blood cells; determining a local median gray of all pixels in each of the one or more identified windows after the white blood cells containing the pixels have been replaced by the replacement median grayscale intensity values An intensity value; and outputting the local adaptive threshold for each of the one or more identified windows based on the local median grayscale intensity value therein. The system of claim 5, wherein the threshold determination sub-module is operatively coupled to a speckle recognition sub-module, the speckle recognition sub-module configured to receive the local adaptive threshold and to adapt the local adaptation A threshold is applied to each of the adaptive grayscale intensity images corresponding to the at least some of the windows. The system of claim 2, further comprising a spot recognition sub-module operatively coupled to the threshold determination sub-module, the speckle recognition sub-module configured to receive one or from the threshold determination sub-module Multiple local adaptive thresholds. The system of claim 7, wherein the spot recognition sub-module is configured to apply the local adaptive threshold to a corresponding image patch in the plurality of fields of view and determine one of the corresponding image patches Or the presence of a plurality of candidate objects having a grayscale intensity below the local adaptive threshold. The system of claim 7, wherein the spot recognition sub-module is configured to determine the one or more candidates in each image patch below the local adaptive threshold based on its darkness threshold The existence of the object. The system of claim 1, further comprising a speckle detection sub-module, wherein the speckle detection sub-module is configured to invert a brightness of the adaptive gray-scale intensity image to generate a plurality of inverted grays a degree-intensity image; determining a local adaptive threshold of the grayscale intensity based on the plurality of inverted grayscale intensity images; and determining to be higher than a brightness threshold based on the plurality of inverted grayscale intensity images The presence of the one or more candidate objects in each image of the local adaptive threshold. The system of claim 1, wherein the at least one processor is further configured to determine the focal plane having the highest focus score for each image patch having each individual candidate object. The system of claim 11, wherein the at least one processor is configured to select and output a respective focal plane having the highest focus score for each candidate object. The system of claim 11, wherein the at least one processor is configured to identify a darkest spot in the focal plane that has the highest focus score for each individual candidate object, and to use the darkest spot as a sense Candidate object assignment of interest. The system of claim 11, wherein the at least one processor is configured to identify a most rounded spot having the highest focus score for each individual candidate object in the focal plane, and using the most rounded spot as Candidate object assignments of interest. The system of claim 1, wherein the at least one processor is configured to determine one or more of an area, a roundness, or a grayscale intensity of each individual candidate. The system of claim 1, wherein the at least one processor is configured to output a determined attribute of each of the single candidate objects having the best focus at each individual candidate object to be based on The one or more determined attributes classify each individual candidate object as an artifact or candidate item. The system of claim 16, wherein the at least one processor is further configured or includes an artifact classifier configured to determine each of the artifact classifiers based at least in part on one or more determined attributes Single candidate object score. The system of claim 17, wherein the at least one processor is configured to determine a score for the candidate object based on the one or more of the determined attributes. The system of claim 18, wherein the at least one processor is configured to discard a single candidate item having a score below a threshold score and to retain each individual candidate item having a score above the threshold score. The system of claim 19, wherein the at least one processor is configured to extract and output one or more image fixes for each of the individual candidate objects retained. The system of claim 20, wherein the one or more image fixes for each individual candidate object retained comprises: a color corrected red, blue, and green image and at least one single candidate object A small area of the adaptive gray-scale image of the field of view and the focal plane. The system of claim 21, wherein the at least one processor is configured to output the one or more images of each individual candidate object retained for feature extraction of each of the individual candidate objects retained repair. The system of claim 1, wherein the at least one processor is configured to determine a white balance transform and apply the white balance transform to each of the plurality of images to effectively generate a plurality of color corrected Determining an adaptive grayscale transform and applying the adaptive grayscale transform to each of the plurality of images to provide an adaptive grayscale intensity image for each of the plurality of images; Extracting and rating the one or more of the one or more image fixes based at least in part on one or more characteristics of the one or more candidate objects, based at least in part on a score Filtering the one or more candidate objects and outputting one or more color corrected image patches and one or more adaptive grayscale intensity image patches for each filtered candidate object; from the color Correcting image patching and the adaptive grayscale intensity image patching extracts one or more feature vectors and outputs the one or more feature vectors; each of the one or more feature vectors As corresponding to artifact or analytes; and determining the analyte is classified into the feature vector is above or below a threshold level associated with a positive diagnosis. The system of claim 23, wherein the at least one memory storage medium comprises an image pre-processing module, a candidate object detection module stored therein as a computer readable program executable by the at least one processor , feature extraction module, classification module and diagnostic module. A method for determining the presence of an analyte in blood, the method comprising: receiving a plurality of images of a sample slide, the plurality of images comprising, a plurality of fields of view, each of the fields of view including the sample a unique x and y coordinate of the slice; and a plurality of focal planes, each focal plane having a unique z coordinate of the sample slide; applying a white balance transform to each of the plurality of images to effectively generate more a color corrected image; applying adaptive grayscale transformation to each of the plurality of images to provide an adaptive grayscale intensity image for each of the plurality of images; detecting and identifying The one or more candidate objects of the plurality of color corrected images and the adaptive grayscale intensity image, comprising: performing an adaptive thresholding operation on the adaptive grayscale intensity image and based on Outputting one or more candidate objects, clustering the one or more detected candidate objects into a cluster comprising one or more candidate objects/clusters, and associating indicates that one or more adjacent candidate objects are single candidate objects Checked a cluster of candidate objects to and output a location of a cluster of the one or more adjacent candidate objects, the location comprising one or more image patches comprising a cluster of the one or more adjacent candidate objects; Each of the individual candidate objects has the focal plane that is optimally focused; determining attributes of each of the individual candidate objects in the focal plane having the best focus for each individual candidate item; based at least in part on one or more determined The attribute filters each individual candidate object; and extracts and outputs one or more image patches, each image patch comprising at least one filtered single candidate object of the one or more candidate objects. The method of claim 25, wherein performing an adaptive thresholding operation on the adaptive grayscale intensity image and outputting one or more candidate objects based thereon comprises: receiving one or more adaptive grayscale intensity images And receiving a white blood cell detection mask including information about locations of the plurality of fields of view and white blood cells in the plurality of focal planes; and detecting the mask using the one or more adaptive grayscale intensity images and white blood cells A mask determines a local adaptive threshold of grayscale intensities of one or more regions in the adaptive grayscale intensity image. The method of claim 26, wherein determining a local adaptive threshold of grayscale intensity of one or more regions in the adaptive grayscale intensity image comprises determining a location in the adaptive grayscale image The local adaptive threshold of at least some of the plurality of fields of view and the plurality of plurality of focal planes, the adaptive grayscale image comprising at least some windows, the at least some of the windows being included One or more candidate objects, the determination being achieved by locally estimating the noise floor of the at least some of the windows. The method of claim 27, wherein locally estimating the noise floor of the at least some of the windows comprises: determining each of the at least some of the windows in the adaptive grayscale intensity image A median grayscale intensity value that discounts any change in the median grayscale intensity value due to the presence of white blood cells. The method of claim 28, wherein determining a median grayscale intensity value in each of the at least some of the adaptive grayscale intensity images comprises: receiving the plurality of fields of view Information on the presence and location of white blood cells in one or more identified windows of the field of view; when indicating the presence of white blood cells, the median grayscale intensity value determined from all pixels in the field of view Substituting pixels of white blood cells in a particular region of the one or more identified windows of the field of view; determining that the white blood cells comprising the pixels have been replaced by the replacement median grayscale intensity values a local median grayscale intensity value for all pixels in each of the one or more identified windows; and outputting for each of the one or more identified windows based on the local median grayscale intensity value therein The local adaptive threshold of the windows. The method of claim 29, further comprising applying the local adaptive threshold to each of the at least some of the windows in the adaptive grayscale intensity image. The method of claim 26, further comprising applying the local adaptive threshold to a corresponding image patch in the plurality of fields of view and determining that one or more candidate objects are present in the corresponding image patch, The one or more candidate objects have a grayscale intensity that is lower than the local adaptive threshold. The method of claim 31, wherein determining that one or more candidate objects are present in the corresponding image patch, the one or more candidate objects having a grayscale intensity lower than the local adaptive threshold comprises: based on The dark threshold of the adaptive grayscale intensity image determines the presence of the one or more candidate objects in each image patch below the local adaptive threshold. The method of claim 31, further comprising: inverting brightness of the adaptive grayscale intensity image to generate a plurality of inverted grayscale intensity images; based on the plurality of inverted grayscale intensity images Determining the local adaptive threshold; and determining, based on a brightness threshold of the plurality of inverted grayscale intensity images, the one or more candidate objects in each image patch above the local adaptive threshold The presence. The method of claim 25, wherein identifying the focal plane with each of the individual candidate objects having the best focus comprises determining the focal plane having the highest focus score for each image patch having each individual candidate object . The method of claim 34, further comprising selecting and outputting a respective focal plane having the highest focus score for each candidate object. The method of claim 34, further comprising identifying a darkest spot in the focal plane that has the highest focus score for each individual candidate object and assigning the darkest spot as a candidate of interest. The method of claim 34, further comprising identifying a most rounded spot having the highest focus score for each individual candidate in the focal plane and assigning the most rounded spot as a candidate of interest. The method of claim 25, wherein determining the attributes of each of the focal planes in the focal plane having the best focus for each individual candidate comprises: determining the best focus for each individual candidate One or more of the area, roundness, shape, or grayscale intensity of each individual candidate object in the focal plane. The method of claim 25, further comprising outputting one or more determined attributes of each of the individual candidate objects in the focal plane having the best focus for each individual candidate object based on the one or A plurality of determined attributes classify each individual candidate object as an artifact or candidate object. The method of claim 39, wherein filtering each individual candidate object based at least in part on the one or more determined attributes comprises: using an artifact classifier, the artifact classifier configured to be based at least in part on One or more determined attributes are described to score for each individual candidate. The method of claim 40, further comprising determining a score for each individual candidate based on the one or more determined attributes. The method of claim 41, wherein determining the score comprises scoring the one or more determined attributes based on known attributes corresponding to known analytes. The method of claim 41, wherein filtering each individual candidate based at least in part on the one or more determined attributes comprises discarding a single candidate item having a score below a threshold score and retaining having a higher than said The single candidate object of the score of the threshold score. The method of claim 43, further comprising extracting and outputting one or more image fixes of the retained single candidate object. The method of claim 44, wherein the one or more image fixes of the single candidate object retained comprises: color corrected red, blue, and green images and at least one single candidate object A small area of the adaptive gray-scale image of the field of view and the focal plane. The method of claim 43, further comprising outputting the one or more image fixes of the single candidate object retained for feature extraction of the retained single candidate object. The method of claim 25, further comprising: filtering the one or more candidate objects based at least in part on a score based at least in part on one or more characteristics of the one or more candidate objects, and Each filtered candidate object outputs one or more color corrected image patches and one or more adaptive grayscale intensity image patches; from the color corrected image patches and the adaptive grayscale The intensity image patch extracts one or more feature vectors and outputs the one or more feature vectors; classifies each feature vector to correspond to an artifact or an analyte; and determines that the feature vector classified as an analyte is Above or below the threshold level associated with a positive diagnosis. The method of claim 47, wherein the method is performed using at least one memory storage medium, the at least one memory storage medium comprising as at least one operably coupled to the at least one memory storage medium The computer-readable program executed by the processor stores each of an image pre-processing module, a candidate object detection module, a feature extraction module, a classification module, and a diagnostic module. The method of claim 48, wherein the candidate object detection module comprises a candidate object cluster detection module, a candidate object cluster module, a best focus detection module, a candidate object cluster attribute extraction module, and an artifact filtering. Module and thumbnail extraction module. The method of claim 49, wherein the candidate object cluster detection module comprises an intensity threshold determination sub-module and a speckle recognition sub-module, the speckle recognition sub-module being configured to identify gray having a grayscale intensity different from a threshold A cluster of candidate objects of degree strength. The method of claim 47, wherein applying the white balance transform to the plurality of images comprises using a plurality of brightest pixels in the plurality of images. The method of claim 51, wherein applying the white balance transform comprises determining the white balance transform, the determining the white balance transform comprising: selecting a plurality of most from a subset of the plurality of randomly selected images Bright pixels such that the probability of existence of clear pixels therein is substantially 1; calculating and applying a standard gray level intensity of each of said subset of said plurality of images to determine in said plurality of images The plurality of brightest pixels in each of the subsets; determining a red value R, a green value G, and a blue value B of each of the plurality of brightest pixels; An average color vector of the average color definition of the bright pixels; determining a white vector; determining an axis vector perpendicular to both the average color vector and the white vector, and by the average color vector and the white A cross product of the vectors is calculated; and an affine transformation matrix is determined, the affine transformation matrix being calculated from the axis vector and an angle between the average color vector and the white vector. The method of claim 52, wherein applying the white balance transform comprises applying the white balance transform to each of the plurality of images defined by a red value R, a green value G, and a blue value B therein A color vector of one pixel, and based on this, outputs a color corrected image. The method of claim 47, wherein applying the adaptive gradation transformation to the plurality of images comprises outputting a plurality of adaptive grayscale intensity images. The method of claim 54, wherein applying the adaptive grayscale transform comprises: receiving a plurality of color corrected images and a standard grayscale intensity image as inputs; and the standard grayscale intensity map at a dark threshold Like thresholding to detect one or more spots; filtering at least one of the detected color, area or shape of one or more spots to locate and identify white blood cell nuclei under high sensitivity and specificity; Outputting a red value R, a green value G, and a blue value B from one or more pixels of the color corrected image including the white blood cell nuclei as white blood cell vector data; outputting as a background vector data a red value R, a green value G, and a blue value B of a plurality of qualified background pixels determined by randomly sampling pixels that are brighter than the dark threshold in grayscale intensity in the color corrected image; An adaptive grayscale projection vector is determined based on the white blood cell vector data and the background vector data. The method of claim 54, wherein applying the adaptive grayscale transform comprises determining and applying the adaptive grayscale projection vector using a plurality of white blood cell pixels, a plurality of qualified background pixels, and a regression. The method of claim 56, wherein using the regression comprises using one or more of a ridge regression, a lasso regression, a principal component regression, or a partial least squares regression. The method of claim 54, wherein applying the adaptive grayscale transform comprises: calculating an adaptive grayscale projection vector and applying the adaptive grayscale projection vector to each of the plurality of color corrected images to Effectively provide multiple adaptive grayscale intensity images. The method of claim 54, wherein applying the adaptive grayscale transform comprises calculating and applying a polynomial regression using a second order or higher order polynomial predictor variable matrix. The method of claim 54, wherein applying the adaptive grayscale transform comprises calculating and applying a polynomial regression using a predictor variable matrix of rational functions having a red value R, a green value G, and a blue value B. The method of claim 47, wherein detecting and identifying the one or more candidate objects comprises determining one or more based on one or more of the plurality of color corrected images or the plurality of adaptive grayscale intensity images The location of multiple potential analytes. The method of claim 47, wherein associating the cluster indicating one or more adjacent candidate objects is a cluster of detected candidate objects of a single candidate object and outputting the cluster of one or more adjacent candidate objects Positions include: determining which of the plurality of fields of view include one or more candidate objects therein; based at least in part on adjacent ones of the one or more candidate objects in the field of view The distance between the clusters is to cluster one or more candidate objects to provide a cluster of candidate objects defined by the adjacent candidate objects therein. The method of claim 47, wherein filtering each individual candidate object based at least in part on the one or more determined attributes comprises: based on ground live objects that are trained into the memory storage medium and accessed by the at least one processor The properties to determine the threshold score. The method of claim 47, wherein extracting the one or more feature vectors from the color corrected image patch and the adaptive grayscale image patch comprises: receiving as input corresponding to the Multiple color corrected image patches and multiple adaptive grayscale intensity image patches for the location of one or more potential analytes in the plurality of images; and output one or more of each representing potential analytes Feature vector. The method of claim 64, wherein extracting the one or more feature vectors from the color corrected image patch and the adaptive grayscale intensity image patch comprises: determining and extracting corresponding to the One or more features of the plurality of color corrected image patches of the location of the one or more potential analytes and one or more candidate objects of the plurality of adaptive grayscale intensity image patches; One or more features associated with the one or more candidate objects are represented as one or more feature vectors. The method of claim 65, wherein determining and extracting one or more features of the one or more candidate objects comprises extracting one or more automatically learned features. The method of claim 66, wherein extracting the automatically learned features comprises teaching a weight of a machine learning model group based at least in part on ground truth image patching having one or more ground truth objects therein, Wherein the one or more ground live objects comprise a sample of the analyte and a sample of artifacts. The method of claim 67, wherein the machine learning module comprises a convolutional neural network. The method of claim 68, wherein teaching the weight of the machine learning model group based at least in part on ground truth image patching comprises using the data enhancement scheme to enhance the ground truth image patching. The method of claim 69, wherein the data enhancement scheme comprises a random gamma correction of one or more of the red, green, blue, or grayscale intensity components of the ground live image patch. The method of claim 64, wherein extracting the one or more feature vectors from the color corrected image patch and the adaptive grayscale intensity image patch comprises: based at least in part on a best focus score Determining an optimal focal plane for each of the plurality of color corrected image patches and adaptive gray intensity image patches containing the one or more candidate objects, the best focus score comprising a highest score from a plurality of focus scores of the plurality of focal planes in an image patch having candidate objects therein; determining all of the plurality of focal planes in which each image of the candidate object is patched A standard deviation of focus points on; and determining a redshift score for each image patch based at least in part on a red change in the darkest portion of the candidate object between the plurality of focal planes patched by each image. The method of claim 47, wherein classifying each feature vector to correspond to an artifact or an analyte comprises: receiving one or more feature vectors as input candidate objects and classifying the one or more feature vectors To correspond to one of the artifacts or the analyte. The method of claim 72, wherein classifying each feature vector to correspond to an artifact or an analyte comprises: using the output to represent each of the one or more feature vectors of the one or more candidate objects corresponding to A machine learning classifier for the score of an analyte. The method of claim 73, wherein outputting the color corrected image patch and the adaptive grayscale intensity image patching comprises using a material enhancement scheme to enhance the color corrected image patch and the Adaptive grayscale intensity image patching, and including the one or more feature vector classifications in enhancements corresponding to each of the color corrected image patching and the adaptive grayscale intensity image patching The output of the machine learning classifier is averaged over the version's feature vector. The method of claim 74, wherein the data enhancement scheme comprises color corrected red, green, blue, or self in the color corrected image patch or the adaptive grayscale intensity image patch A random gamma correction that accommodates one or more of the grayscale intensity components. The method of claim 47, further comprising: accepting one or more of the one or more labeled images of the analyte in the ground live sample and the artifacts in the ground live sample as a ground truth Tagged images. The method of claim 76, wherein one or more of the one or more labeled images of the analyte in the ground truth sample and one or more artifacts in the ground live sample are received as ground truth The marked image includes teaching the weight of the machine learning classifier group based at least in part on one or more learned ground truth sample image patches. The method of claim 77, wherein the machine learning classifier comprises a convolutional neural network and teaches the machine learning classifier to be grouped based at least in part on the one or more ground truth sample image patches The weighting includes loading the one or more marked images of the analyte in the ground live sample and the one or more marked images of the artifacts in the ground live sample To the convolutional neural network. The method of claim 78, wherein teaching the weight of the machine learning classifier group based at least in part on the one or more ground truth sample image patches comprises: using a data enhancement scheme to enhance the one or Multiple ground live sample images are patched. The method of claim 79, wherein the data enhancement scheme comprises a random gamma of one or more of the red, green, blue, or grayscale intensity components of the one or more ground sample live image patches. Horse correction. The method of claim 47, wherein determining whether the feature vector classified as corresponding to the analyte is above or below a threshold level associated with the positive diagnosis comprises: based on one or more classified as an analyte The magnitude of the feature vectors to determine if the analyte is present and gives an indication of the presence or absence of the analyte. The method of claim 47, further comprising identifying the category of the one or more candidate objects based at least in part on one or more image characteristics including one or more of a shape, a size, or a color. The method of claim 47, wherein the analyte comprises a parasite. The method of claim 83, wherein the parasite comprises a malaria parasite. The method of claim 47, further comprising recording one or more images of the one or more sample slides with a microscope. A method for determining the presence of an analyte in blood, the method comprising: receiving a plurality of images of a sample slide, the plurality of images comprising: a plurality of fields of view, each field of view including the sample a unique x and y coordinate of the slice; and a plurality of focal planes, each focal plane having a unique z coordinate of the sample slide; applying a white balance transform to each of the plurality of images to effectively generate more a color corrected image; and applying an adaptive grayscale transform to each of the plurality of images to provide an adaptive grayscale intensity image for each of the plurality of images; detecting and identifying And one or more candidate objects in the plurality of color corrected images and the adaptive grayscale intensity image, comprising: performing an adaptive thresholding operation on the adaptive grayscale intensity image and based on This outputs one or more clusters of candidate objects, receives one or more adaptive grayscale intensity images and receives white blood cell detection masks including information about locations of the plurality of fields of view and white blood cells in the plurality of focal planes Cover The one or more adaptive grayscale intensity images and white blood cell detection masks to determine a local adaptive threshold of grayscale intensities of one or more regions in the adaptive grayscale intensity image, including, determining The local adaptive threshold of at least some of the plurality of fields of view and the plurality of windows of the plurality of focal planes in the adaptive grayscale image, the adaptive grayscale image comprising at least some a window, the at least some of the windows including one or more candidate objects, the determining being performed by locally estimating a noise floor of the at least some of the windows, the local estimation being performed by: determining the adaptive gray a median grayscale intensity value in each of the at least some of the windows in the intensity image, discounting any change in the median grayscale intensity value due to the presence of white blood cells; The median grayscale intensity value in each of the at least some of the windows in the adaptive grayscale intensity image comprises: receiving one or more of the fields of view recorded in the plurality of fields of view Information on the presence and location of white blood cells within the identified window; when white blood cells are indicated as being present, replacing the field of view with a replacement median grayscale intensity value determined from all pixels in the field of view Determining pixels of the white blood cells in a particular region of the one or more identified windows; determining the one or more identified ones after the white blood cells containing the pixels have been replaced by the replacement median grayscale intensity values a local median grayscale intensity value for all pixels in each window in the window; and outputting for each of the one or more identified windows based on the local median grayscale intensity value therein A local adaptive threshold, correlating a cluster of detected candidate objects indicating that one or more neighboring candidate objects are a single candidate object, and outputting a location of the cluster of one or more adjacent candidate objects, the location Include one or more image patches comprising the one or more adjacent candidate objects; identifying the focal plane with each of the individual candidate objects having the best focus; Attributes of each individual candidate object in a focal plane with the best focus in each individual candidate; filtering each individual candidate object based at least in part on one or more determined attributes; and extracting and outputting one or more images Patching, each image patching comprising at least one filtered single candidate object of the one or more candidate objects, based at least in part on based at least in part on one or more characteristics of the one or more candidate objects Scores to filter the one or more candidate objects, and outputs one or more color corrected image patches and one or more adaptive grayscale intensity image patches for each filtered single candidate object; Color-corrected image patching and the adaptive gray-scale intensity image patching extract one or more feature vectors and output the one or more feature vectors; classifying each feature vector to correspond to artifacts or analysis And determining whether the feature vector classified as an analyte is above or below a threshold level associated with a positive diagnosis.