EP3803495A1

EP3803495A1 - Analyzer for three-dimensionally analyzing a medical sample by means of a light-field camera

Info

Publication number: EP3803495A1
Application number: EP19728133.0A
Authority: EP
Inventors: Thomas Engel; Gaby MARQUARDT; Gabriele Hörnig; Lukas RICHTER
Original assignee: Siemens Healthcare Diagnostics Inc
Current assignee: Siemens Healthcare Diagnostics Inc
Priority date: 2018-05-30
Filing date: 2019-05-16
Publication date: 2021-04-14
Also published as: EP3575848A1; JP2021525866A; CN112236705A; US20210217190A1; WO2019229573A1

Abstract

The invention relates to an analyzer for analyzing a medical sample. The analyzer comprises an optical microscope for imaging a light field in an object region in order to image the sample. The microscope comprises a light source for illuminating the sample, an objective comprising a converging lens for concentrating and focusing light beams coming from the illuminated sample, and a digital recording device for recording the light beams. A light-field camera for capturing the light field from the object region, which light field is imaged in the microscope, is provided on the microscope.

Description

description

Analyzer for the three-dimensional analysis of a medical sample by means of a light field camera

The invention is in the field of automatic analyzers and relates to a hematology analyzer for analyzing cells in a sample using a microscope apparatus having a light field camera.

For automated analysis of cells, so-called "automated cell counters" are used with increasing success. Examples include the Advia 2120, Sysmex XE-2100, CellaVision DM96 and CellaVision DM1200 device. These automated devices, apart from their high throughput, provide some advantages, such as high objectivity (no variability depending on the observer), elimination of statistical variations, which are usually associated with manual counting (counting high cell counts), as well as the determination of numerous parameters that would be unavailable on a manual count and, as mentioned, more efficient and cost-effective handling. Some of these devices can handle 120 to 150 patient samples per hour.

The technical principles of automatic Einzelzellzäh ment usually based either on an impedance (resistance) measurement or on an optical system (scattered light or absorption measurement). Further, imaging systems are established, e.g. Automatically make and evaluate cells of a blood smear.

In the impedance method, cell counting and sizing are based on detection and measurement of changes in electrical conductivity (Resistance) caused by a particle moving through a small opening. Particles, such as blood cells, are themselves non-conductive, but are suspended in an electrically conductive diluent. When such a suspension of cells is passed through an aperture, when a single individual cell passes, the impedance (resistance) of the electrical path between the two electrodes located on either side of the aperture temporarily increases.

In contrast to the impedance method, the optical method involves passing a laser light beam or an LED light beam through a dilute blood sample that is in continuous flow from the laser beam or the LED

Light beam is detected. The corresponding light beam may be e.g. be conducted to the flow cell by means of an optical waveguide. Each cell that passes through the detection zone of the flow cell scatters the focused light. The scattered light is then detected by a photo detector and converted into an electrical impulse. The number of pulses generated here is directly proportional to the number of cells passing through the detection zone in a particular period of time.

In optical methods, the light scattering of an individual cell passing through the detection zone is measured at different angles. As a result, the Streuver hold the respective cell for the optical radiation he summarizes that allows conclusions about eg the cell structure, shape and reflectivity. This scattering behavior can be used to differentiate different types of blood cells and to use the derived parameters to diagnose deviations of the blood cells of this sample from a standard obtained, for example, from a variety of normal classified reference samples. In the automated evaluation of cells in blood smears, today's analyzers use microscopes with a high numerical aperture and with immersion between the slide and the objective to achieve a high resolution. However, this results in a comparatively low depth of field, which is significantly less than the thickness of the cells perpendicular to the surface of the Objektträ gers with the blood smear. Accordingly, with a two-dimensional image in only one focus setting, the entire depth information of the cell can not be sharply imaged.

Often, therefore, unclearly classified blood cells are known to those skilled in the art, e.g. a laboratory doctor must manually reclassify. To do this, the slide with the blood smear is placed under a microscope again, the appropriate cell is laboriously searched for and then inspected by the laboratory doctor. For a safe classification, the laboratory doctor focuses mostly through the cell in order to be able to better identify and evaluate the structure of the cell along the direction of the focus.

WO 2007/044725 A2, Bahram J., et al. "Three-dimensional identification of biological microorganism using integral aging" in Optics Express, Vol. 14, No. 25, pp. 12096-566, Kim J. et al .: "A single-shot 2D / 3D simultaneous imgaging mi - Croscope based on light field microscopy ", in Visual Communications and image Processing, Vol. 9655, pages 9655101 - 9655104 and WO 2010/121637 Al describe optical imaging devices.

The hematology devices currently in use include a microscope of approximately 100x magnification with effective lateral resolutions relative to the optical system a sensor element at the object level of 100 nm = 0.1 mpi. The usual cameras have pixel numbers of max. 0.3 to 1 million pixels. As a result, the field of view in the object is only a few 100 gm. In order to record and analyze the stained sample of a blood sample, which may be several mm wide and several tens of mm long, the surface of the area of the smear to be analyzed must then be scanned with a displacement unit in a meander scan procedure. In order to speed up the process a first area scan with low magnification, eg 10 times with a correspondingly 10 times larger field of view is performed and after a first image analysis to find the cells to be measured then only the regions of interest (Rol) with the Cells with the higher magnification subsequently targeted targeted. This means that a full area analysis of the high magnification sample is not done. In order to make the cells sufficiently visible to be able to analyze them with high-resolution optical microscopy, the blood smears are stained in an upstream step. Several dyeing protocols have been established worldwide, some of which differ globally from region to region. Thus, the comparability of the analysis of blood smears is regionally limited, since only images of cells which have been stained according to the same protocol can be compared well.

An object of the invention is thus to provide an automatic analyzer for analyzing cells in a sample and a method for determining a two-dimensional or three-dimensional information of a cell, wherein, for example, chemical pretreatment of the cell should be avoided as far as possible before image acquisition and so that the analysis takes place around the cell in one to investigate the situation as originally and least or not at all changed.

The object is achieved by an analyzer according to the invention and the inventive method according to the independent patent entitlements. Advantageous developments of the invention are given in particular by the dependent claims.

The invention particularly comprises an analyzer for analyzing cells in a sample, the analyzer comprising an optical microscope for imaging a light field in an object area and / or an object plane for imaging cells of the sample, the microscope comprising a

A light source for illuminating the sample and a converging lens for collecting and focusing light rays from the illuminated sample, and a digital recording apparatus for recording the light beams, wherein on the microscope, a light field camera for receiving the microscope in the field gebil Deten light field is provided from the object area and where when the light field camera the digital recording device sums.

The digital recording device preferably comprises a detection device for the light beams and allows a conversion of the detection signals of the detection device into digital data.

An analyzer according to the invention has the advantage that the hitherto necessary staining of the samples for the subsequent microscopy can be completely or partially dispensed with. The analyzer according to the invention enables image data of the highest quality with correspondingly high information content, which can be used, for example, for automatic evaluation and classification. The recorded image data have such high quality, so that a Follow-up examination of the sample by renewed microscopic examination is no longer necessary. Thus, according to the invention, a complete digitizing station of the cell is provided with a high resolution for lateral structures and also over the entire depth extent of the respective cell. In particular, a coloring and / or marking of the blood cells for the image acquisition is not necessary.

The analyzer according to the invention is preferably an automatic analyzer, particularly preferably a semi-automatic or fully automatic analyzer. The microscope preferably comprises the light field camera. Before given to the light field camera includes the digital recording device, which is designed to receive the imaged in the microscope light field. Preferably, the digital recording device comprises a charge-coupled device (CCD) chip or a plurality of CCD chips. Particularly preferably, the digital recording device is based on a complementary metal-oxide semiconductor (CMOS) technique and / or comprises a CMOS chip.

Preferably, the sample is transported within the automatic analyzer fully automatically, e.g. by ent speaking flow systems and / or experience of sample carriers within the analyzer, preferably by means of appropriately controlled actuators or robot systems.

Preferably, the light field camera comprises a microlens array with lenses of different focal lengths, the microlin senarray an intermediate image of the imaged in the microscope

Can image light field on the digital recording device. The microlens array is at least one focal distance from the intermediate image removed and there is a real Abbil tion. Thus, according to the invention, there are no aperture images, but real, small image sections in the sense of small part images of the object or the sample. The picture information becomes then projected back through the optics until the rays of corresponding pixels from different sub-images meet. In the present case, therefore, no images of the images are taken but small object images.

The invention is based on an optical microscope, which is equipped with game contrast with devices for differential interference contrast. Such microscopes can be adapted to the requirements of hematology in reference to the splitting. This adaptation takes place essentially by a specific choice of the beam offset on the beam away through the measurement object. The measurement object is, for example, a crossed out blood sample.

The beam offset is determined by the thickness of the DIC prisms in the illumination module and in the analyzer module. Here, the optical thickness of the two prisms and the direction of the beam offset must be coordinated with each other, so that the analy satormodul the beam offset from the lighting module can completely reverse and thus compensate. Due to different magnifications, the physical thicknesses of the prisms can deviate from each other. For microscopes, there may be several pairs of DIC prisms for one lens, allowing for different splitting.

Furthermore, a light field camera, also called a plenical camera, is provided on the microscope, which records the light field formed in the microscope from the object plane.

Preferably, the light field camera has an effective aperture number (working f #) in the range of 10 to 30, particularly preferably it is 26. Depending on the desired scanning of the measured object with sensor elements, a suitable magnification is selected for this purpose, the sorelement the size relationship of the real Sen, that in the range of typically 1 to 10 pm and the scanning of the object, which is in the range of 0.05 to 0.5 gm, which is preferably because of the high resolution requirements in the measurement of the range of 0.05 gm to 0.15 pm is taken into account. This then results in effec tive system magnifications from object to Detektionsein direction in the range of 10-fold to 100-fold, preferably in the range of 20-fold to 65-fold, and more preferably in the range of 40-fold to 65-fold. If one uses an immersion liquid for the required high resolution, the numerical aperture can assume values in the range of 1.4. If the sample is surrounded by air, the numerical aperture must be max. 1, technically limited to 0.95 or 0.9.

The image-side NA results from the object-side NA via the magnification NA_image = NA_Obj / V.

The numerical aperture NA and the f-number (f #) of the optics are linked via f # = 1 / (2 * NA).

This results in magnifications of 40 or 63 and nu merischen apertures of 0.95 and 1.4, in which case accordingly f # is in the range of 14 to 33.

For example, the light field camera is preferably a light field camera with the designation Raytrix R12 Micro Series from Raytrix GmbH, Kiel.

From the light field, the image of the object can be reconstructed for different depths, which corresponds to different focus settings of the microscope. This purely digital refocusing can be done retrospectively on the recorded data sets.

The analyzer according to the invention has the advantage that, in addition to the color information RGB, depth information D (depth) which is available in a downstream computer-based Evaluation as a further feature analogous to a color channel ver can be used.

Based on the additional depth information, the cells may e.g. segmented more easily, so recognized for further analysis and cut out of the picture. In addition to contrasting color transitions then arise in the height profile at the edges of the cell transitions that are well and accurately detectable.

In conventional systems, the problem of si cal determination of the edge of a cell based only on the color image so far, because usually a threshold is used, the system is already within the cell, since to achieve the threshold value must have already taken place a certain change , That is why cells are usually measured systematically too small. This is annoying in volume measurements and is then e.g. via interpolation algorithms tries to correct. With the provision and use of depth information, these problems can be solved easily and safely because the background of the slide represents a plane to which the profile falls. Then, known methods for the geometric determination of the cell borders due to height profiles can be applied accordingly.

By capturing the light field with the light field camera, the captured image, e.g. the blood smear or the cells, still refocused after taking the image who the. This can, on the one hand, be used to segment the cells and identify them for classification

to focus as well as possible, or to use several focal planes in parallel for the segmentation and classification. Alternatively, volume data or 3D point clouds can also be used. This allows the segmentation and Klassifika tion done with higher accuracy. If the images, eg of the cells, have been taken with the light field camera, it is possible subsequently to focus the images purely digitally through, ie offline. This would allow for unclear classifications or evaluations of cells by the computer, the doctor the ability to look at the cells directly in the digital image and not just the slide again under a microscope to le gene, to search the cell and then on the Fokusvariation a more accurate rating and classification. Since there are no coordinated coordinate systems between the cell analyzer and the microscope, the doctor can now position the cell to be examined only with an approximate location in the microscope and then has to search the cell in the final microscope. This is time consuming and must be successful in the laboratory. However, if the digital image of the light field camera according to the present invention is present, the physician can directly focus the cell or structure in question in the digital image.

This evaluation and classification is then also in the sense of telemedicine in an existing data connection of the doctor to the image or the image database possible so that he no longer has to come to the slide in the laboratory.

In the case of unclear findings, a consultation with a colleague can be made, who receives the image for analysis or, for example, makes it available via electronic networks, which can be done virtually in real time with digital data transmission. To evaluate and classify the recorded light field images of the blood cells, the respective physician only needs a corresponding analysis software on his digital terminal or in the sense of a cloud solution, both the image of the light field camera can be stored in the cloud and the evaluation, for example via a web - Browser Interface suc in a cloud-based application.

Preferably, the magnification is determined from the requirement of the optical resolution for the scanning of the measurement object, wherein with the choice of magnification in general, the Benö required effective f-number or the numerical aperture is a related party.

The lateral resolution is preferably 100 nm in the object level. The magnification then results together with the lateral dimension of a sensor element

V = lateral dimension sensor element / 100 nm.

This magnification is e.g. particularly advantageous for the formation of blood cells.

About 100 nm is the typical resolution limit of light microscopy in the case of water, oil or glycerin immersion. For a known pixel size of the camera of e.g. 4.5 gm would be the required magnification 45x. The aperture then results from the aperture in the object plane - with immersion in the range greater than 1, typically 1.2 to 1.4 - divided by the magnification, ie NA camera = 1.4 / V, for example. 1.4 / 45 = 0.031. The f-number then results from f # = 1/2 * NA to f # 16.

The f-number of the camera is preferably 2.4, 2.8, 5.6, 7.0 or 26.0. More preferably, the f-number of the camera is f # 26 and the magnification is 63 times.

With a pixel size of the camera of 2 gm, a magnification of 20x is preferred for the desired resolution in the object plane. Thus NA camera = 1.4 / 20 = 0.07 and thus the f-number f # 7. This has the advantage that with high-resolution camera chips with high pixel count large fields of view can be covered and also the cost of optics can be minimized. The analyzer is preferably a hematology analyzer, particularly preferably an automatic hematology analyzer.

Preferably, the medical sample comprises a cell and / or a medical preparation. The medical preparation is preferably a tissue section, sediments of body secretions and / or body fluids and / or microcrystals.

More preferably, the sample is a blood sample and / or the cell is a blood cell.

Instead of blood cells, the sample may preferably be any type of human, animal or plant cell. This has the advantage that various types of probes, including various cell types, can be examined and characterized.

The inventive combination of a light field camera with a microscope for hematology is still significantly supplemented by advanced contrasting methods such. Phase contrast or differential interference contrast (DIC).

The microscope is preferably an amplitude contrast, and / or a phase contrast, and / or a differential interference contrast (DIC) microscope.

Preferably, the microscope is a differential digital holographic microscope.

The extended contrasting methods, and in particular the DIC, make it possible to visualize phase differences of the light paths through the sample, in this case, for example, the cells. Different phase values, in particular during phase contrast then, for example, with different colors be displayed and measured with a color camera who the. This makes it possible to dispense with a dyeing of the samples and still depict the cells well contrasted. Previous systems for automated cell classification use exclusively the amplitude contrast, which results from the staining of the cells and the different paths of the light in the cell or alternatively the surrounding medium.

The combination according to the invention of a microscope and a light field camera gives additional 3D information about the cell, which can be used for a classification of the cell. Depending on the selected contrasting method in the microscope, the following data are thus obtained for classification of the cells. The classification may e.g. performed by healthcare professionals and / or computer-based systems. In the case of amplitude contrast, conventional image information and additional 3D information are present in the cell. In phase contrast, a phase image and additional 3D information of the cell is obtained. With DIC, a DIC image is obtained on differential phase images as well as additional 3D information of the cell.

According to the invention, the resulting 3D information of the cell may advantageously be e.g. be represented differently as follows.

Advantageously, the 3D information of the cell is displayed as a RGB image. Advantageously, RGB image as so-called total focus image over the entire Tiefenschärfebe empire of the light field camera is sharply displayed. Due to the increased depth of field compared to conventional 2D cameras, more depth information is captured.

A blood cell has - depending on the type and orientation - a thickness of about 1 to 2 gm, to about 20 gm. The depth of field of the At NA 1.3 and at 500 nm wavelength, however, the figure is only d = ± 1 / NA ^L 2 = ± 500 nm / 1.69 = ± 300 nm. Thus, in the conventional 2D image, none of the cells is sharply focused over the full depth , The depth of field increases by at least the factor four on the plenoptic effect from the light field camera. Thus, cells, in particular, for example, red blood cells (RGB) are fully focused. Also, larger cells, such as white blood cells (WBC), are imaged sharply to a considerably larger portion of the cell volume.

In a further advantageous embodiment, the 3D in formation of the cell is displayed as RGB D information. Each pixel contains a depth information called D. When used as a sample, e.g. For example, if blood cells are applied to an object carrier, D is e.g. the thickness of the blood cell. This information complements the color information and is also referred to as a 3D point cloud.

In a further advantageous embodiment, the 3D in formation of the cell as volumetric 3D information Darge is. Analogous to an image of a computer tomograph (CT) ent spatial information thus in the form of voxels. Since the cells, e.g. Blood cells for which radiation is at least partially transparent, so different Stel len the cell generate stray radiation, which is recorded by the camera and assigned to different depths.

In a further advantageous embodiment, an image stack can be calculated from the data set of the light field camera who the, which is calculated on different focal planes and thus contains the volumetric information. In the evaluation algorithms of the light field camera, the so-called virtual depth is the distance measure- ment. Alternatively, the focal planes can also be calculated with other distance values, which are selected equidistantly, for example, and one level with the max. Cross-section of the cell falls together.

In the case of contrasting with phase contrast, the image of the light field camera based only on the intensity modulations generated by the phase effect, since the amplitude or intensity modulation are filtered out of the object itself in the phase ring. This image is as colorfast as possible to the image with pure amplitude contrast.

In the case of contrasting with DIC, the image of the light field camera is based on an image of the cells, which contains information about the different paths of the light as it passes through the cell and thus the phase change of the light as a color-coded information via the differential interference contrast. This has the advantage that the color representation of the cell superimposes a fine structure, which is very helpful for the evaluation algorithms of the light field camera for a high lateral resolution in the calculation of the depth information.

In an advantageous embodiment, up to four color channels with RGB and white are used.

Advantageously, the spectrum of illumination includes the visible range and / or near IR wavelengths.

Preferably, a color balance of the light source to a camera, such as the light field camera. Furthermore, an adjustment of the exposure time and the gain for the 3-chip RGB color camera for the 2D images preferably takes place. Particularly preferably, the light source comprises a 4 LED light source with RGB and W and a common brightness control. In a further preferred embodiment of the invention, a further camera for recording an image in an object plane in the object area is provided in the analyzer on the microscope, wherein the further camera has a lateral resolution which is equal to or preferably higher than the lateral resolution of the light field camera, wherein the resolution of the further camera is twice, preferably three times, particularly preferably four times the resolution of the light field camera.

The further camera preferably has a larger field of view, also referred to as field of view, as the light field camera. This allows a better and faster Probenabde ckung.

Preferably, the focal plane of the further camera, before given to a 2D camera, is coupled to an excellent plane of the light field camera. This can be measured, for example, in vir tual depth. So it can be advantageously enough that both cameras deliver good pictures at the same time.

In a further preferred embodiment of the invention, the further camera is a color camera, preferably a color camera

Multi-chip color camera, particularly preferably a 3-chip Farbka mera.

This has the advantage that by means of the high-resolution color camera, such as a high-resolution RBG camera, a high resolution is guaranteed. This is of particular advantage since, in principle, a factor of two is lost laterally in the light field camera, and a factor of four in terms of resolution is lost in order to obtain information for calculating the depth resolution. This loss of resolution can be fully compensated before geous enough. The use of a multi-chip color camera, such as a 3-chip color camera, has the advantage that then a better color measurement and in particular pixel-by-pixel color determination without interpolation with more dynamics is possible, which brings more before parts for the classification of cells. Advantageously, an adjustment of color channels takes place for an optimized S / N ratio. In the case of, for example, colored samples, the compensation is preferably carried out separately for each staining protocol. Advantageously, the 3-chip color camera is a 3-chip CMOS camera with 3.2 megapixels per chip.

A 3-chip color camera is compared with e.g. a 1-chip color camera preferred because a 1-chip color camera usually use a Bayer pattern. Since, in the analysis of e.g. Blood cells in a hematology analyzer e.g. Particularly good color resolution and color fastness and good lateral resolution of the structure are the preferred features of the 3-chip camera. The high resolution color camera provides e.g. a high resolution RGB image with typically better, i. more accurate color and less noise by separately optimizing the exposure for each of the color channels. In the case of the 3-chip color camera, the color values are determined directly and without interpolation for each pixel separately.

Preferably, the color values of a color camera are subsequently transferred from the RBG representation to, for example, an HSV representation for color value (Hue), saturation (Saturation) and brightness (Value), then, for example, red blood cells, where the red Dye is homogeneously distributed in the cell, also well above the color value as a further or complementary method to segment. This approach works well for all cells and structures that have the same color value (Hue) or are colored with the same color. This segmentation can advantageously with the over the Depth values (D) are combined in order to have as precise a criterion as possible to excise the cell.

The images of the two cameras are advantageously registered via scaling and / or pixel interpolation to each other, if the effective pixel sizes in the images do not match or are commensurate with integer factor of e.g. 1, 2, 3, etc. For this purpose, lateral displacements, distortions, tilting, distortion and / or distortion in the images and / or a defocus are in principle advantageously to be corrected. If the images are then registered to the required extent, new image data is calculated which combines the higher latent color resolution with the depth information from the image of the light field camera. For this purpose, the image of Farbka mera can be set to the assigned focus position in the evaluated Tie fenbild the light field camera and from there via the known from the recording of the light field camera Propa gation of the light field and the color representations and Lateral len resolutions transmitted to adjacent focal planes who the.

The transmission can take place, for example, laterally via interpolation, for the colors e.g. via correspondence tables. Alternatively, a more elaborate transmission is also possible via a true propagation calculation based on the data measured by the light field camera together with e.g. a phase retrieval possible. In this case, evaluated image planes of the light field camera serve as support points and the additional additional points are supported by the neighboring points.

Based on the excellent plane and together with suitable continuity conditions for the optical fields and the associated calculated optical wavefronts, a comparatively accurate and extensive interpolation is possible. Advantageously, this more accurate solution, which is typically very computationally intensive, is used for fine diagnostics, eg on conspicuous image areas, cells with unclear findings and / or pathological cells, where less or no real-time capability is required. The fine diagnostics can then also be done, for example, on the complete image or even only on image sections, for example, on individual segmented cells possibly extended by certain margins around there.

Depending on the arrangement in the beam path are the following different Liche, each advantageous variants for arranging inven tion provided.

For the measurement of stained cells or specimens or those which provide a good contrast in the images without extended contrasting methods, the two cameras can be used and operated simultaneously on the microscope. In living or moving cells or samples or examinations using microfluidic cells, it may be advantageous if the two cameras also record the images synchronously in time.

Here, e.g. commercially available microscope tubes are used, which mechanically enable the parallel operation of two cameras on a microscope. There are various types of beam splitting, e.g. the splitting ratio, the splitting method such as spectral separation, polar separation, etc., which can be chosen to be advantageous.

In undyed cells requiring extended optical contrasting methods, further preferred technical embodiments of the system according to the invention are optionally provided when constructing the system. Preferably, only one polarization direction is used for the color image. For this purpose, a separation of the beam paths for the color camera and the light field camera between the lens and the DIC analyzer or alternatively between the lens and in front of the lens-side DIC prism is provided.

The division of the light field on the imaging side is already taken into account in the orientation of the polarization before the splitting in the illumination module in such a way that after the DIC prism and DIC analyzer the two polarization components satisfy the desired intensity ratio, e.g. 1: 1 for maximum contrast. In this way, light losses can be minimized in order to optimize the efficiency of the overall setup and to minimize the required exposure times for a quick measurement.

Preferably, the separation of the beam path takes place in the case of the cameras between the imaging lens and the DIC analyzer with a polarization-neutral beam splitter. In the beam path to the high-resolution color camera then a polarizer is still to be arranged so that it blocks the one polarization direction of the DIC beam path as completely as possible and let the other pass with the least possible attenuation to the camera. This polarizer can either work in transmission or in reflection or lead to a spatial separation of the differently polarized beams. This offers the advantage that the structure is less Comp complex in the design and no special polarization-dependent adjustment needed, however, go principle, about 25% of the amount of light lost.

Advantageously, the images of the light field camera and the color camera are mutually aligned and scaled so that, for example, in a topography image, the 3D or Height information of the light field camera can be used and at the same time the color information can be used and used by the color camera.

Preferably, the recording takes place by means of the light field camera and the color camera at the same time. This can e.g. be achieved by ent speaking triggering. The triggering can be implemented via hardware or via software. The triggering can also take place from a selected first camera to the second camera, which is also referred to as master-slave coupling. Functionally it must be ensured that the pictures are taken with a given temporal reference, ideally at the same time.

Due to possible latencies in the hardware and software of the cameras and the respective activation, the trigger signals may themselves have a time interval. The same time exposure is of particular advantage for the uptake of moving cells, e.g. in a flow cell, and / or living cells or measurement objects.

Advantageously, a combination of the images from the light field camera and the high-resolution, further camera only for image areas in sections, for example. only in one area in which measuring objects to be examined were detected or detected in an automated measuring process. This has the advantage that an increased measurement speed can be achieved, since corresponding computational effort and computing time can be saved.

Depending on the arrangement in the beam path, different configurations may be advantageous.

In an advantageous embodiment, only one polarization tion direction is used for the color image and the separation of the beam paths for color camera and light field camera is still done in front of the DIC analyzer or in front of an objective-side DIC prism. The division of the light field can be advantageously considered in the direction from the polarization prior to the splitting in the lighting module so that after the DIC prism and DIC analyzer, the two polarisati onsanteile the desired intensity ratio, such as 1: 1 for maximum contrast, have. This is especially advantageous if stained cells are to be imaged and analyzed.

In a further advantageous embodiment, the color camera for high-resolution DIC and the plenary optics for an extended depth measurement range for unambiguous unwrapping the color ranges for measuring thicker cells or Zellclus tern is used. A high-resolution relative thickness measurement, therefore, it follows by means of DIC in that the differential Kontrastin formations, for example. be integrated and a more spacious assignment is done by plenipotentics.

For unstained samples, e.g. Cells, the amplitude contrast barely contains no information and the color image is advantageously used only for phase or DIC contrast, but then has a high lateral resolution with a true color separation per pixel.

In an advantageous embodiment, the light field camera and the high-resolution camera, which advantageously comprises a color camera, are aligned relative to one another with respect to the axial directions of the sensor array and advantageously also the subpixel accurate shift along the axis directions of the array and / or a directional scaling factor for e.g. the x and / or y axis.

In an advantageous embodiment, the position of the focus of the high-resolution camera on the average measuring range of Light field camera set. This has the advantage that the light field camera can also be used in a process similar to an autofocus system.

With a smaller effective f-number f #, the achievable depth resolution of the light field camera increases. Thus, a f-number of 7 can be significantly more advantageous than a Blen denzahl 26. A lower f-number means a higher image-side aperture. Since the obj ektseitige aperture is limited to 1.4 for the immersion typical for biological samples, the maximum image-side aperture results in NA Ka mera = 1, 4 / V.

Advantageously, the object with the full imaging aperture is already illuminated on the microscope in order to illuminate on the image side the full aperture of the working f # so that the light field camera works well and can provide a high lateral scan. Conveniently NA lighting = NA Whether jektiv. This is also called incoherent illumination with sigma = s = NA illumination / NA objective = 1.

In an advantageous embodiment of the automatic analyzer according to the invention, the incoherent illumination Sigma is greater than 0.8, preferably greater than 0.9, be particularly preferably 1.0, where sigma is given as the Quoti ent from the numerical aperture of the illumination at the microscope the sample and the numerical aperture of the objective, and wherein the microscope is preferably a differential interference contrast (DIC) microscope.

In hematology, this s = 1 or s> 0.8 or better s> 0.9 is very important since the blood cells are weakly scattering objects and thus the aperture is sufficient for light even for a microlens array in the light field camera is filled. So far, in light microscopy to optimize the contrast - especially for visual observation, but also for cameras - Köhler illuminations were usually used with a sigma = 0.7. However, this would lead to an insufficiently filled with light aperture and thus could not be a more accurate depth determination. In addition, because of the unilluminated pixels, much of the spatial resolution would be lost. This is related to the fact that the number of pixels used for the calculation of the images is approximately square with the value

increases from sigma and reaches s = 1 the maximum near the full detector resolution.

In an advantageous embodiment of the analyzer according to the invention, the analyzer comprises a Probenzuführungseinrich device for slides by means of the analyzer samples can be supplied to a slide.

In an advantageous embodiment of the analyzer according to the invention, the analyzer comprises a flow cell for the supply of the sample, wherein preferably the object level of the Mik roskops is located in the flow cell. Preferably, the flow cell is a microfluidic flow cell.

This has the advantage that in particular cells or blood cells can be imaged and analyzed in the natural environment. In particular, it can be avoided that the cells must be applied by means of a smear on a slide, where they often dried and / or colored who, which can change their original natural properties such as shape significantly. Next eliminates the ent speaking preparation costs for the manufacture of strokes and staining and it is largely avoided from case by eliminating the consumption of eg Slides and coverslips for the smears. Also, storage capacities for the smears and consumables are avoided. At the system level of the analyzer, the devices for automatically changing the smears can be dispensed with, which, inter alia, enables a considerable simplification of the device structure.

Since the usable layer is typically several microns thick in a flow cell, sometimes even some 10 gm thick, it is difficult for the cells to precisely position in the object plane of the microscope used for the examination. In this regard, there is a conflicting requirement tion of the desire for high resolution on the one hand and the not so precise position of the cell in the direction of the optical axis on the other.

In this case, the depth measurement of the light field camera can preferably be used here for optimizing the activation parameters of the microfluidics, in order, for example, to adjust the focus appropriately so that the 2D high-resolution images are also sharp. An exposure and 3D images is also an added value without interferometry, as in DHM, which is complex and complex and currently. Interference by effects with temporally and / or spatially coherent radiation has.

As the numerical aperture NA of the optics increases, the spatial resolution becomes better over the relationship d = 0.5 l / NA. On the other hand, reduces the depth of field with the numerical aperture according to d = ± l / NA ^L 2. At 500 nm wavelength and NA = 0.7 then gives a depth of field of 2 gm, which speaks for example about the thickness of red blood cells ent.

In the hitherto conventional devices is used with an optics with apertures of NA = 0.5, which then at 500 nm a depth of field totaling d = 4 gm. The achievable lateral resolution is limited to d = 0.5 gm via the NA = 0.5. So far, a compromise had to be made with the choice of the numerical aperture to balance between lateral resolution and depth of field. The relationships mentioned describe the resolving power of optical instruments in current theory.

In an automatic analyzer according to the invention with a corresponding light field camera, the depth measuring range, ie the area where sharp images are recorded, can be increased by a factor of approximately 4 to approximately 6. In contrast, the lateral resolution of the light field camera decreases by a factor of 2.

Since the magnification and the aperture can be selected largely independently of one another in the selection of the optics used, however, this effect can be well compensated. If one chooses e.g. a larger magnification and a higher aperture, so the effect can be correspondingly positive ver.

If, for example, a magnification of 60x to 80x and an aperture of NA 0.7 are selected, the following values are obtained as a whole for 500 nm wavelength. NA = 0.7, d = 0.36 gm, depth of field d = 1.05 pm. The effect of increasing the depth of field from the plenoptic camera results in values for an increased depth of focus of d = 5.25 pm, assuming an increase of about a factor of 5.

If, for example, a magnification of 60 times to 80 times and an aperture of NA 0.9 is selected, the following values are obtained in terms of field strength for a wavelength of 500 nm. NA = 0.9, d = 0.28 pm, depth of field d = 0.62 pm. With the effect To increase the depth working range from the plenoptic camera, values for an increased sharpness depth of d = 3.10 gm result, assuming an increase of about a factor of 5.

By means of the use according to the invention of the light field camera in a cell analyzer, the depth working range can thus be greatly increased, which then makes it possible to use a flow cell accordingly. In particular, a more complete or complete detection of the expansion of a cell via the height extension of the flow through the flow cell is possible. This allows in particular precise investigations on typically unstained cells. For certain studies, however, it may also be advantageous or necessary for the cells to be stained in the medium of microfluidics.

A further advantage is that the parameters of the flow cell with regard to focusing and / or position of the cell can lie in a further parameter range and e.g. can be dispensed with a corresponding special optimization.

In a preferred embodiment, the field of view of the microscope includes the full width of the flow cell through which cells can flow. This area advantageously has a width of a few 1/10 mm to a few mm.

In a preferred embodiment of the automatic analyzer, the analyzer comprises both means for viewing slides with cells and / or for viewing cells in the flow cell. Advantageously, the dimensions of the flow cell on the dimensions of the object are carriers, in particular their thickness and thus the optical effects We matched. The cover of the flow cell - so the cover plate and possibly also fluidic media extending between the region or the plane of the cells and the äuße ren surface of the

Flow cell, advantageously also have optically the same effect, e.g. with regard to optical path length, dispersion and / or refractive index, so that the same optics can be used and the same best possible imaging quality is maintained. Preferably, transfusion may be performed to obtain e.g. to adapt to manufacturing tolerances. Typical cover glass thicknesses are in the range below 0.2 mm, typically 0.15 to 0.17 mm. The lateral dimensions of slides are, for example, 76 mm by 26 mm or 75 mm by 25 mm according to DIN ISO 8037-1 at one

Thickness in the range of 1 mm to 1.5 mm. The dimensions of the flow cell then arise advantageously accordingly.

In a preferred embodiment, the speed of movement of the cells in the flow cell in the imaging region of the microscope is adapted to the exposure time of the image acquisition systems used. The movement is typically smaller than U pixels (pxl), preferably smaller than 1/5 px, particularly preferably smaller than 1/10 pxl. This has the advantage that motion blurring effects can be reduced or completely avoided.

In a further advantageous embodiment of the inventive analyzer, the analyzer comprises a sample feeding device for slides. This may be particularly advantageous when e.g. Tissue sections or other medicinal preparations should be examined.

Another object of the invention is a method for determining a two-dimensional or three-dimensional Information of a cell by means of an analyzer according to the invention, the method comprising the steps

a) feeding a sample with a cell to the microscope, b) recording the light beams emanating from the cell in the illuminated sample by means of the digital recording apparatus,

c) imaging of the light field imaged in the microscope from the object area by means of the light field camera, wherein the light field camera includes the digital recording device.

d) determination of a two-dimensional or three-dimensional and / or volumetric information of the cell from information recorded in step b) of the light beams and / or information of the light field imaged in step c).

Advantageously, steps b) and c) may also be combined into one step, the step then comprising

- Picture of the light field imaged in the microscope from the object area by means of the light field camera and

Recording the light rays emanating from the cell in the illuminated sample by means of the digital recording apparatus, wherein the light field camera comprises the digital recording apparatus

In step b), the light beams emanating from the cell in the illuminated sample are preferably first focused and then drawn by means of the digital recording device. A conversion of the photons into electrical charge with subsequent determination of a charge quantity and digitization of the charge quantity is preferably carried out.

Instead of the cell, a two-dimensional or three-dimensional information can preferably also be determined correspondingly from a medical preparation. The sample is preferably supplied to the analyzer in step a) by means of a flow cell, the object plane of the microscope lying in the flow cell. Preferably, the flow cell comprises means making it possible, by appropriate control of the flow cell, to enrich the cells in an excellent plane in the flow cell. Preferably, the object plane is set to the excellent plane, so that the object plane and the plane drawn out advantageously coincide.

Preferably, the sample is supplied in step a) by means of a sample feeding device for slides to the analyzer on a slide.

Preferably, the method further comprises a step in which an image is provided in an object plane in the object region by means of a further camera for taking an image in the object plane in the object region, wherein the further camera has a lateral resolution which is higher than the lateral resolution of Light field camera is, wherein the resolution of the wei direct camera is preferably twice, particularly preferably four times the resolution of the light field camera and wherein be preferably the other camera has a larger field of view than the light field camera.

Preferably, the further camera is a color camera, preferably a 3-chip color camera.

The cells and / or the medical preparation are preferably not stained. In this case, the extended contrasting methods, such as, for example, phase contrast and / or differential interference contrast, in particular for imaging cells, are particularly advantageous. For example, in the case of sediments or other samples, amplitude contrast may also be advantageous. In a preferred embodiment of the method, the method further comprises the step

e) digital refocusing or focus variation by means of the determined in step d) two-dimensional or dreidi dimensional and / or volumetric information of the cell along the optical axis of the microscope, wherein preferably the digital refocusing is computer-aided and / or nume risch.

This has the advantage that for the first time a telemetric examination in hematology is made possible by the refocusing or focus variation provided according to the invention. Thus, also consular findings, e.g. by experts at other locations or even a Nachbefundung allows.

In order to do this with the established high quality of standard of care diagnosis, it is indispensable for the follow-up or consular doctor to be provided with an opportunity to focus through the cellular level.

In the prior art, the corresponding object carrier with the questionable cells has been physically brought out of eg a magazine, a storage box or an archive out and then placed under a microscope and the image ever Weil focused on the respective cell. Based on relatively inaccurate location information with respect to the position of the respec gene cells on the slide was then tried to find each respective cells again. If the cells were found again, the diagnosis is made. To determine this, the doctor analyzes the optical image of the cells and usually focuses through the cell. Therefore, the previously known systems and methods for de waisted Nachbefundung are not or only very suitably appro net, as the doctors no longer taken once the cell image be able to focus and thus an important for the diagnosis depth information is not accessible.

One advantage of the present invention is therefore that the cell image can be recorded as a three-dimensional image in order to allow the doctor on the digitally stored image a freely adjustable and at any time, also subsequently selectable, focus on different focal planes. The doctor no longer has to sit on a microscope himself, but the process can be performed spatially and temporally independent of the sample. Since the image is purely digital, occurs over the time since the image was taken no deterioration of the data, in contrast to the hitherto customary procedures in which the archived samples age over time and deteriorate in their state. For each follow-up, the sample would be returned to the microscope and immersed in immersion oil. After diagnosis, the sample is then cleaned again, which can also lead to damage and overall represents a certain time-consuming effort.

Another advantage is that when multiple cells are on an image, the focus can be adjusted and varied sequentially with respect to each one of the respective imaged cells.

In a further preferred embodiment of the method, the method further comprises the step

f) assignment of the cell to a cell type based vorbe vorbe information and the determined in step d) two-dimensional, three-dimensional and / or volumetric information of the cell. This has the advantage that an automated assignment of the cell to a cell type can be made particularly reliable and less error-prone.

A further subject of the invention is a method for assigning a cell to a cell type, comprising a method according to the invention for determining a two-dimensional len and / or three-dimensional and / or volumetric Infor mation of the cell, wherein at a first location, the steps a) to d ), and wherein the information determined in step d) is transmitted digitally to a second location via a data and / or network connection, and wherein steps e) and f) are performed at the second location.

The invention also relates to a corresponding method for assigning a cell to a cell type, wherein, however, ever determining a two-dimensional or dreidimensio nal or volumetric information of the cell using any other suitable method for determining a zweidi dimensional or three-dimensional and / or volumetric information Cell is done.

This has the advantage of enabling telemedicine for hematology. For example, this makes it possible to obtain expert advice from other locations. Thus, a fine diagnosis with Fo kussieren can also be made remotely on a recorded cell. Furthermore, this allows a consultation of a secondary physician to the post- or Ge genklassifizierung to secure the diagnosis in the database. Thus, for example, a ground-truth record with special and in particular rare pathologies arise because of the remote functionality and the findings Using 3D images to make physicians around the world easier and more efficient Advantageously, reliable findings are used in cloud server data sets for the automatic extension of a training data set, in particular in pathological cases.

Advantageously, a worldwide learning system is etab lated. This makes it possible for larger data sets to be collected as quickly as possible, even for very rare disease pictures. These data sets can then be used advantageously also for automatic computer learning algorithms, so that ultimately also for the automated evaluation and / or assignment of the cells a broader and more secure basis is available.

Advantageously, the patient data are kept in a database. This has the advantage that even later onset of disease can be analyzed earlier records on blood images and can be examined for abnormalities, such. in the case of hematology, to detect or recognize very early stages of development or signs of leukemia. If these data are learned by a system, this analysis can be automated on the images and run without additional effort, since the cell images are advantageously pre-evaluated by computers.

The presentation of the cells and / or samples preferably takes place by means of a 3D display device, preferably e.g. an autostereoscopic 3D display. This allows the physician to give a novel 3D visual impression of the cells and / or samples to be examined. In particular, this advantage is also achieved in telemedicine according to the invention.

Advantageously, a compact data format for the image data from the light field camera is used. Preferably, who compresses the image data. This makes it possible to keep the required storage space as small as possible, which is only the Storage of many patient records allows, which can then be used as a ground truth, for example, for a computer-learning system in terms of automated medicine and / or as a valuable assistance system for doctors.

Advantageously, use of cloud solutions and / or server solutions for image storage and data of the diagnosis and advantageously also for the backup of extended data, which person, e.g. Doctor, when and on which system with which configuration or software version the cell in question has been diagnosed or also when and where the sample in question was taken from the patient and information about the transport to the laboratory. Alternatively, this information may preferably also be provided via a link to another storage system. Advantageously, the time of Befun training is stored accordingly.

Another object of the invention is a method for digital staining a cell, the method comprising a method according to the invention for determining a zweidimensio dimensional or three-dimensional information of a cell by means of an analyzer according to the invention and additionally the following steps

g) digitally staining the cell according to a predetermined association between the two-dimensional or three-dimensional and / or volumetric information of the cell and a staining of a corresponding cell and / or a structure within the cell by means of a staining protocol,

h) displaying an image of the digitally stained

Cell.

Preferably, instead of the cell, a medical preparation is stained accordingly. Advantageously, the two-dimensional or three-dimensional and / or volumetric information of the cell is geometric information of the structure of the cell or medical preparation.

Here, under two-dimensional information, e.g. understood an image information that maps an object area in a planar image with the lateral coordinates in the X and Y direction, which describe the area and spans nen. The individual pixels of a two-dimensional image are also referred to as pixels.

Under three-dimensional image information, e.g. ver stood that the image in addition to the areal Bildinforma tion additionally at least for one pixel also contains a fen fen information in the axial direction of the Bildstrahlengan ges, so for example. in the Z direction, which is linearly independent of the X and Y directions. An example of three-dimensional image information could be e.g. be a contour image that in the z-direction, the topography of the object as dergibt.

For example, volumetric information is used if the image also contains information over a large area in the X and Y directions, also in the Z direction for different Z values. For example, the X, Y and Z axes may each be divided with increments of equal size to produce small volume elements, also referred to as voxels. Image information described via voxels is a form of volumetric information. Volumetric image data are used eg in tomography. If the voxel information is present only for a few pixels or voxels, the information can also be represented in the form of so-called point clouds, where each of the points is, for example, over its X, Y and Z coordinates are represented. The volumetric information is an extended representation for three-dimensional image information, which is particularly advantageous in the case of complex structured objects.

For clear interpretation of the image data with two-dimensional, three-dimensional and volumetric information, it is generally necessary to know in which coordinate system, e.g. Cartesian or cylindrical, and with wel cher orientation of the coordinate axes, the underlying coordinate system is defined.

In further advantageous embodiments, the two-dimensional or three-dimensional and / or volumetric information is information relating to intracellular structures such as e.g. Cell organelles and / or geometrical structures of a tissue section. Information about optical path lengths generally does not readily conform to geometric information, and therefore optical path lengths are not geometric information in this regard.

The dyeing protocol is preferably the dyeing protocol May-Grünwald-Giemsa, Modified-Wright dyeing, Wright-Gimsa dyeing and / or Romanowsky dyeing.

In the case of the digital coloration of the cell, it is preferable to use a color distribution according to one of the above-mentioned dyeing protocols. For this purpose, images of the technical contrast of black and white and / or according to extended contrasting methods are preferably recolored by means of dyeing properties determined by computer learning. For this purpose, 3D and / or volumetric information are preferably used, since this is advantageous over the use of a pure 2D color mapping. The methods according to the invention for the digital staining of a cell have the advantage that the digital staining of the cells can be carried out subsequently in the digital image. This eliminates the additional steps of dyeing the south in the sample preparation. This leads to considerable time and cost savings. Furthermore, this avoids differences between different laboratories or regions.

As a result, it is only possible to take into account the digital differences in the laboratories in the implementation of a staining protocol for the desired color or the different regionally used staining protocols and these habits of the respective physician regardless of the location of the actual blood or tissue examination to implement accordingly.

Another advantage is that switching between different staining models is possible, e.g. to be more specific for certain pathologies.

Advantageously, the staining protocol is an artificial color model, e.g. emphasizes certain critical features in order to simplify the diagnosis for physicians, or in the first instance to enable the diagnosis of inexperienced doctors. The artificial color model can be designed similarly to a false color representation for technical images, e.g. similar to thermal or IR images, which are displayed as a color image. Thus, even within the findings of one and the same cell simply the staining models can be ge changed, which would not be possible with a classical chemical staining.

Preferably, cells assigned to different classes are colored differently digitally. Advantageously, all cells in the image that are currently not to be considered are hidden. Preferred is alternatively only a selected set of different cell groups are shown.

Preferably, the 3D information of the cells, possibly also in combination with the advanced contrasting methods such as phase contrast or DIC, used to replace the omitted Far binformation of attributable immediate staining by the 3D information. This makes it possible to implement the accordingly learned information regarding the uncolored image as robustly as possible and less susceptible to errors in the color information.

Preferably, the predetermined association between the two-dimensional or three-dimensional or volumetric information of the cell and staining of a corresponding cell and / or structure within the cell is determined by a staining protocol as follows in a learning phase.

For this purpose, first the cell or the cells without staining in a first image is displayed. Subsequently, the cells are stained in accordance with the respective desired staining protocol. Then the stained cell or cells are stained in a second image. In the first and second images, the cell (or cells) is then recognized and segmented, and the unstained and stained cells are assigned accordingly.

Advantageously, the cells are grouped by cell type, e.g. red blood cells (RBC), white blood cells (WBC) (optionally inclusive 5 part diff.) and / or classified, e.g. by means of computer learning and / or neural networks.

In the next step, computer learning of the characteristics of the staining for each of the cell groupings is done separately and then a specific digital staining procedure for unstained cells, preferably separately for each cell grouping.

To verify the digital staining procedure then apply the staining to the unstained cells and a comparison with an evaluation of the result of the digital staining with the sample actually dyed, with a lichst same appearance of the colored and digitally colored cells is sought. Advantageously, an evaluation of the result then takes place in accordance with predefined quality criteria and / or after maximum permitted residual deviations.

Because of the question of color fastness, which is always very critical in medicine, the color deviations can also be evaluated against an external reference standard, as described, for example, in US Pat. in DICOM for displays and display devices is described. Since the standard is not written for this application, evaluations may only be transferred and applied analogously. The staining model is accordingly set, checked and stored for each staining protocol and each cell grouping individually or specifically.

For digital staining of a cell, the procedure is preferably as follows. First, the desired staining protocol is selected and the unstained cells are displayed in a first image. The cell or cells are segmented in the first image. Then an assignment of the cells to cell grouping and / or a determination of the cell type for the selection of a cell-specific staining model. In the next step, the cell or cells are stained. Subsequently, the display of a digitally colored image, e.g. a blood smear, on a dispenser.

Advantageously, the same uncolored image in the pas send, dyed each regionally common color. This allows the common evaluation of a sample by experts who are used to different colorations.

Also, an exchange of the respective staining protocol at any time is possible, regardless of whether the original blood sample is still present, for example. to make more blood smears. Another big advantage of digital coloring is that one and the same cell can be colored differently, which is not possible with a chemical dyeing of the real cell. This allows e.g. also the cross-regional consultation beyond previous limits of dyeing protocol applications. Due to the completely digital information available, previous problems such as e.g. bleach the sample over time.

Furthermore, it is possible that in any follow-up to specific patients also staining protocols applicable to those who were not known or unavailable at the time of the original diagnosis. This makes it possible that the medical progress with respect to the standard of care for the existing samples or their images applies bar and can lead to correspondingly higher quality findings.

So far, the blood smears were stained in a preliminary, to-moderate step, to make the cells in the blood smear for the microscope and the observation by the doctor sufficiently well visible to analyze them with high-resolution optical microscopy can. This method has been in use and established for many decades. In this regard, various staining protocols such as May-Grünwald-Giemsa, Modified-Wright-Stain, Wright-Giemsa Stain, Romanowsky Stain are established world wide, whereby even within the protocols laboratory-specific differences may arise, as some laboratories slightly change the color after their Adapt wishes. As a result, the comparability of the analysis of blood smears is limited, especially since in the automated analysis and classification of blood smears only cells that have been stained according to the same protocol can be well compared. Another disadvantage of the previous procedure is that the dye requires an exposure to the struck blood sample of about 10 - 15 minutes. This can be very disadvantageous in analyzes of acute clinical pictures.

The digital staining of a cell according to the invention is preferably an image of a preferably uncolored cell taken digitally in an analyzer according to the invention. This image of the cell is then colored in a post-processing, as it is used to the doctor of the stained according to the different conventional staining protocols cells.

It should preferably be avoided as far as possible to supply color to the sample for staining the cell, at least before it has been imaged once.

According to the invention, different staining protocols can also be selected according to the invention with the new digital staining for different cell types, because, for example, their specifics for the particular cell type are particularly suitable. Thus, a doctor or hematologist or pathologist could create personal staining schemes where he assigns a specific type of staining to a cell type and completely breaks through and abolishes the previous restriction of just one staining for a slide. As an extension of this "individual" coloration, generic digital staining protocols can advantageously also be created, which continue to use the technical color space for better recognizability of features and structures Chemical dyeing works, in each dyeing step typically, essentially a dye which accumulates in un ferent amounts of suitable bindable cell structures. The more dye attaches, the more light is absorbed from the spectrum of the illumination light according to the absorption spectrum of the respective dye out biert. The transmission image thus contains a higher color saturation (S) for the color value (s) of the dye (V). If several dyes in a staining protocol are used simultaneously or serially, which bind at different points of the cell structure, the effects are superimposed. Each dye changes its saturation value S for its specific color values V depending on the amount of dye attached. The number of dyestuffs and their specific spectral characteristics limits the number of altered color values V. Typical stains in medicine are in the red-blue range, so that, for example, color values in the green or yellow spectral range remain virtually unchanged. The digital coloration can advantageously work both with sharper color separation and with wider coverage in the optical spectrum.

In some studies, however, it may be advantageous not to completely dispense with the addition of dyes and / or pharma ceutical and / or chemical additives. For example, In the case of reticulocytes, in addition to the pure color effect, the dye also has the desired side effect that the RNA still available in the cell clumps and only then can be imaged and contrasted in the micrososcope. The undigested RNA is smaller in dimension than it would be resolvable with conventional optical microscopes.

The object of the invention is based on an optical micro-microscope, which is advantageously equipped for image acquisition with th th extender optical contrasting methods. This Advanced contrasting methods may include, for example, phase contrast (PC), differential interference contrast (DIC), polarization contrast (POL), interferometry (preferred as a digital holographic microscope (DHM)), hyperspectral imaging (pure color contrast in an extended spectral range, eg UV, VIS, NIR, MIR and / or FIR or selected sub-areas thereof), and / or structured turierte lighting (eg, with predetermined intensity distributions and / or phase distributions, preferably in the Pu pillenebene set).

Different contrasting methods are preferably present in parallel on a microscope system, in particular if these are used, for example. mutually free of interaction usable. Otherwise, the different contrasting methods are preferably carried out in chronological succession. If the different contrasting methods are carried out in chronological succession, according to the invention a particel image velocimetry (PIF) is preferably provided for tracking the respective trajectories, since the cells move in a combination with a microfluidic system between the images Modalita obtained color information then again bring together for each cell correctly. Preferably, the PIV also includes the rotation of the cell and advantageously also the defocusing.

Via the extended contrasting methods, the samples are given e.g. a color contrast of interference, such as DIC or polarization, which then provides in the digital image for a good resolution of the structures in the cell and / o which allows a good contrasting for the cell.

According to the invention, it is particularly advantageous if, in addition to the amplitude or intensity information and possibly Also a color information in addition a 3D information is present. The images of the light field camera for the 3D image can also be present as volumetric and / or tomographic data. According to the invention, this 3D information also permits a subsequent purely digital focussing on the image taken. This is of particular advantage for the review of automatic classification by the physician and for the specific and reliable assessment of pathological or conspicuous cells in this context.

Then, according to the invention, new dyeing modes are also provided in digital dyeing, where the color currently displayed depends only on the area of the cell which is e.g. is located below the focal plane. Thus, the 3D depth perception of the doctor can be further improved. Alternatively, only areas above the current focal plane can be used or the areas in a certain adjustable

Layer thickness around the concrete focal plane around.

For digital staining according to the invention, therefore, a picture taken microscopically with one of the systems according to the invention is then postprocessed so that it looks as it would have looked like if it had been taken with a classical microscope of stained cells. For conversion, for each staining protocol, a certain number of unstained samples with a plurality of cells are taken up in the new system. The cells can be detected and segmented manually or automatically in the image for comparison. Thereafter, the same samples are stained with the desired staining protocol and then resumed with a microscope the same cells. It is important that the microscope has a good color fidelity and a color camera, particularly preferably a color camera optimized for the most color-fast image capture, which may also be equipped with special samples or Calibration procedures was calibrated, eg according to the DICOM standard in medicine.

Typical color cameras have a Bayer pattern in which 50 percent of the pixels are predominantly green, 25 percent of the pixels are predominantly blue, and the remaining 25 percent of the pixels are predominantly red. In order to obtain complete color information on the RGB values for each pixel, then e.g. for the color value to be determined, interpolate the color values from the neighboring pixel in the corresponding color. If necessary, continuity information from the pixels of other color values is also considered and used. Preferred color cameras are e.g. 3-chip color cameras that directly measure an intensity value for red, green and blue for each pixel. These preferred color cameras should therefore be cameras which directly measure the color values required for each pixel.

Since the information from undyed cell and colored cell is now available for each cell, the color information can be linked, preferably, for example via methods of computer learning, between the two samples. Preferably, this linking takes place separately for each class of cells in order to capture specific staining behavior as well as possible. Alternatively, to simplify the effort and scope of learning, it is advantageous to learn the same cell types that are in different development stages, for example. This could advantageously be done, for example, with red blood cells that exist in different states of aging, where they easily change the geometric shape. Likewise, it is advantageously possible, in white blood cells, the 5 main groups, who differentiated in the 5 Part differential diagnosis who train together. It may then be advantageous to still be within such a group in a second step to learn the colorations of specific features in order to obtain as good and comprehensive color contrasting as possible.

It is particularly preferred in learning besides the pure 2D information, as they are from normal camera images in today's systems, also the 3D information, e.g. in the form of topographic information or as volumetric or tomographic information. For learning, this 3D information and information advantageously derived therefrom, e.g. an effective 3D profile, both for the uncolored image and preferably for the colored image. For learning then all methods known from computer learning are in principle applicable, such as e.g. PLS, PLSDA, PCA or neural networks (CNN) or deep CNNs.

For a good learning result and thus the most realistic digital coloration can be achieved, it is advantageous if each class of cells to be learned and / or group of cells learned together, the classification takes place in at least as many color values as on the original sample under divisible and / or can be displayed on the display device. In computers color spaces are common for color representation, where for each pixel per color e.g. 256 color values, ie 8 bits or 1 byte color depth, or 65536 color values, ie 16 bits or 2 bytes or 1 double byte can be used.

In order to efficiently save data volumes, it may be advantageous to choose a different representation of the color space and / or the color values than RGB. Since the effective color values can differ between the cell types and ideally differ within a cell type only in the saturation or helicity, a color value and a brightness value are advantageously stored. This has the advantage that the full color information does not have to be stored in RGB for each pixel. This advantageous representation would be comparable eg with a HSV or HSL color representation. Due to the often comparatively similar colors in the images, the potential for saving storage space and / or data volume is particularly high here.

A great advantage of digital dyeing according to the invention is that samples can now be vergli chen between regions, which typically with other coloring Tocolepro and thus other color effect of the cells work ar. If an undyed sample to be examined has been taken up, this can optionally be dyed according to the invention with different dyed color structures which correspond to the respective regional dyeing protocol. On the one hand, this offers the advantage that it is now possible to use the preferred or advantageous staining protocol for different pathologies. On the other hand, a doctor may ask another doctor for a consiliatory opinion, as he can now show him the picture in the familiar color. Digital staining can circumvent the hurdles created by the staining protocols, because now the doctor can choose his customary staining to carry out the diagnosis. So then also, e.g. Doctors from Europe and the USA work better together for the benefit of the patient. This allows for a worldwide collaboration of doctors and / or hematologists in terms of telemedicine.

Thus, for telemedicine and / or telemedicine, with the digitally matched 3D images and / or datasets, every doctor worldwide can contribute to a secure diagnosis. Thus, for example, via a database, in particular for rare pathological cases of observed and / or measured cells over the world wide networking with telemedicine advantageously one A well-grounded "ground truth" dataset, which is a combination of image and / or 3D information and the findings backed up by at least two physicians, is an important component of this procedure, as well in the possibility according to the invention in the digital dataset for each cell taken Separate and adapted to be able to change the focus of the images without the doctor finding himself sitting on a microscope.

This data set, which is independently diagnosed by several physicians, can then, assuming a match in the findings, e.g. be used to further train computer models, which can then be played back on the globally installed devices, in order to further develop each device in the recognition quality for certain cells, or in the ability to safely identify special pathologies automatically.

Depending on the device, it may also be advantageous for the analyzer according to the invention to take pictures and then store them, e.g. in a cloud-based application, or more generally to another computing unit, where then the recognition of the cells, their segmentation and an automated evaluation, e.g. in the sense of an association with a class of cell type or of a diagnosis as a recognition of certain disease images and / or the determination of a suspicion on a specific clinical picture takes place.

For the display of the inventively recorded image data to a cell, there are different possibilities for the doctor.

Advantageously, for example, a conventional 2D monitor, a mobile terminal and / or a tablet PC who used the. Advantageously, the color representation is so adjusted that the doctor can assume a color-fast color rendering. This is particularly important in hematology because small color differences and structural differences in the image of the cell can give an indication of conspicuousness, especially in the case of pathological or suspicious cells. Advantageously, the color reproduction, in particular special with respect to the stable color reproduction such as the DICOM standard for medical devices (dicom.nema.org/).

Advantageously, the user and / or user can then focus on the software through the cell images and / or the flat image from the slide or from the flowcell by operation on a software. Advantageously, here for a correspondingly secure, or possibly pre given, data connection to the data on a local data memory, a superordinate data memory, e.g. in a hospital, and / or a cloud. Preference may be given to e.g. a consultation, the data will be made directly accessible to a doctor.

Preferably, a 3D-capable display is used, which can be viewed either with or more preferably without further optical Hilfsmit tel and generated for the user a 3D He heindruck. Preferably, the 3D capable display is part of a computer and / or a mobile terminal, e.g. a tray. Preferably, the 3D effect can be switched on and off.

Data glasses, also referred to as smart glasses or VR glasses, are preferably used for the 3D representation of the data set.

According to the invention, a working mode master-slave is preferably provided for a remote diagnosis in telemedicine, if simultaneous diagnosis of the images by several Doctors, for example, in a medical conference, as a Mas ter navigation over the images of the cells and / or from the slide and / or the flow cell (Flowcell) leads and the other doctors then this can follow without own controlling the system.

Preferably, each of the doctors can also focus on his image independently of the others and / or color the image according to his usual staining protocol.

The invention will be explained in more detail with reference to the accompanying drawings by a specific embodiment. The example shown represents a preferred embodiment of the invention. It shows:

1 shows an automatic analyzer for analyzing cells in a sample.

The automatic analyzer shown in FIG. 1 for analyzing cells in a sample comprises an optical microscope (1) comprising a light source (4) for illuminating a sample (2) and a converging lens for collecting and focusing light rays (6) generated by go out of the illuminated sample (2).

The sample (2) is a blood sample containing blood cells (3). The sample (2) is located in a microfluidic flow cell (10). Furthermore, the microscope (1) comprises a light field camera (8) comprising a digita les recording device, the recording device comprising egg nen CCD chip or a CMOS chip, for receiving the micro-scope (1) imaged light field. Further, the microscope (1) comprises another camera for taking an image in the object plane, which is designed as a high-resolution 3-chip color camera (9). The lateral resolution of the color camera (9) be four times the effective lateral resolution of the light field camera (8). The focus of the color camera (9) is on one Set the central range of the measuring range of the light field camera (8), for example to a virtual depth in the range of 3 to 5. The incoherent illumination sigma on the microscope (1) is 1.0. The microscope (1) is a micro-scope that can also be operated as a differential interference contrast (DIC) microscope (1).

LIST OF REFERENCE NUMBERS

1 microscope

2 samples

3 blood cell

4 light source 6 light beams 8 light field camera 9 color camera

10 flow cell

Claims

claims

An analyzer for analyzing a medical sample, the analyzer comprising an optical microscope for imaging a light field in an object area for imaging the sample, the microscope comprising a light source for illuminating the sample and a lens comprising a converging lens for collecting and focusing Lichtstrah len starting from the illuminated sample and a digi tales recording device for recording the Lichtstrah len, characterized in that on the microscope a light field camera for receiving the imaged in the microscope light field from the Objektbe is rich, preferably the Lichtfeldka mera includes the digital recording device.

2. Analyzer according to claim 1, wherein the light field camera comprises a microlens array with lenses of different focal lengths, wherein the microlens array can image an intermediate image of the light field imaged in the microscope on the digital recording device.

An analyzer according to any one of the preceding claims, wherein the analyzer is an automatic analyzer, preferably an automatic hematology analyzer, and wherein the sample is preferably a blood sample comprising blood cells.

4. Analyzer according to one of the preceding claims, wherein the microscope is an amplitude contrast, and / or a phase contrast, and / or a differential interference contrast (DIC) microscope.

5. Analyzer according to one of the preceding claims, wherein on the microscope, a further camera for receiving an image in an object plane in the object area is provided, wherein the further camera has a lateral resolution which is higher than the lateral resolution of the

Light field camera is, wherein the resolution of the other camera is preferably twice, particularly preferably four times the resolution of the light field camera and where in preferred the further camera has a larger field of view than the light field camera.

6. Analyzer according to claim 5, wherein the further camera is a color camera, preferably a 3-chip color camera.

7. Analyzer according to one of the preceding claims, wherein at the microscope, the incoherent illumination sigma is greater than 0.8, preferably greater than 0.9, particularly preferably 1.0, where sigma is given as the quotient of the numerical aperture of the illumination of Sample and the numerical aperture of the objective, and wherein the microscope is preferably a differential interference contrast (DIC) microscope.

8. An analyzer according to one of the preceding claims, wherein the analyzer comprises a flow cell for the supply of the sample and wherein the object plane of the microscope is located in the flow cell.

9. A method for determining a two-dimensional or three-dimensional information of a cell and / or a medical preparation by means of an analyzer according to one of claims 1 to 8, the method comprising the steps

a) supplying a sample with a cell and / or the medical preparation to the microscope, b) recording the light beams emanating from the cell and / or the medicinal preparation in the illuminated sample by means of the digital recording apparatus, c) imaging the light field imaged in the microscope from the object area by means of the light field camera, d) determining a two-dimensional and / or three-dimensional and / or volumetric information of the cell and / or medical preparation from

Step b) recorded information of Lichtstrah sources and / or information of the imaged in step c) th light field.

10. The method of claim 9, wherein the sample is supplied to the analyzer in step a) by means of a flow cell and wherein the object plane of the microscope is in the flow cell, or wherein the sample in step a) by means of a sample supply means for object carrier the analyzer on a Slide is supplied.

11. The method according to any one of claims 9 or 10, wherein the cells and / or medical preparations are not stained.

12. The method according to any one of claims 9 to 11, further comprising the step

e) digital refocusing or focus variation by means of the two-dimensional and / or three-dimensional and / or volumetric information of the cell determined in step d) along the optical axis of the microscope, the digital refocusing preferably being computer-based and / or numerical.

13. The method according to any one of claims 9 to 12, further comprising the step

f) assignment of the cell to a cell type based vorbe vorbe information and the determined in step d) two-dimensional and / or three-dimensional and / or volumetric information of the cell.

14. A method for assigning a cell to a cell type, comprising a method for determining a zweidimensi onal and / or three-dimensional and / or volumetri's information of the cell according to claim 13, wherein at a first location, the steps a) to d) running the who , and wherein the information obtained in step d) is digitally transmitted to a second location via a network connection, and wherein steps e) and f) are performed at the second location.

15. A method for digital staining of a cell and / or a medical preparation, the method comprising a method according to any one of claims 9 to 14 and additionally the following steps

g) digitally staining the cell and / or the medicinal preparation according to a predetermined relationship between the two-dimensional and / or three-dimensional and / or volumetric information of the cell and / or the medical preparation and a staining of a corresponding cell and / or medical prepa Council and / or a structure within the cell and / or the medical preparation by means of a Därbepro tokolls,

h) displaying an image of the digitally stained cell and / or the preparation.

16. The method according to claim 15, wherein the staining protocol is the staining protocol May-Grünwald-Giemsa, modified-Wright staining, Wright-Gimsa staining and / or Ro manowsky staining.