DE102017009804A1 - Method for evaluating microscopic samples and apparatus for carrying out this method - Google Patents

Method for evaluating microscopic samples and apparatus for carrying out this method

Info

Publication number
DE102017009804A1
DE102017009804A1 DE102017009804.2A DE102017009804A DE102017009804A1 DE 102017009804 A1 DE102017009804 A1 DE 102017009804A1 DE 102017009804 A DE102017009804 A DE 102017009804A DE 102017009804 A1 DE102017009804 A1 DE 102017009804A1
Authority
DE
Germany
Prior art keywords
carrier
images
combined image
method according
characterized
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
DE102017009804.2A
Other languages
German (de)
Inventor
Jiri Snaidr
Claudia Beimfohr
Axel Bonsen
Peter Mühlhahn
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
vermicon AG
Original Assignee
vermicon AG
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by vermicon AG filed Critical vermicon AG
Priority to DE102017009804.2A priority Critical patent/DE102017009804A1/en
Publication of DE102017009804A1 publication Critical patent/DE102017009804A1/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS, OR APPARATUS
    • G02B21/00Microscopes
    • G02B21/36Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
    • G02B21/365Control or image processing arrangements for digital or video microscopes
    • G02B21/367Control or image processing arrangements for digital or video microscopes providing an output produced by processing a plurality of individual source images, e.g. image tiling, montage, composite images, depth sectioning, image comparison
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K9/00Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
    • G06K9/00127Acquiring and recognising microscopic objects, e.g. biological cells and cellular parts
    • G06K9/00134Acquisition, e.g. centering the image field
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K9/00Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
    • G06K9/00127Acquiring and recognising microscopic objects, e.g. biological cells and cellular parts
    • G06K9/00147Matching; Classification
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K9/00Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
    • G06K9/20Image acquisition
    • G06K9/209Sensor details, e.g. position, configuration, special lenses
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K9/00Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
    • G06K9/36Image preprocessing, i.e. processing the image information without deciding about the identity of the image
    • G06K9/46Extraction of features or characteristics of the image
    • G06K9/4671Extracting features based on salient regional features, e.g. Scale Invariant Feature Transform [SIFT] keypoints
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K9/00Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
    • G06K9/62Methods or arrangements for recognition using electronic means
    • G06K9/6267Classification techniques
    • G06K9/6268Classification techniques relating to the classification paradigm, e.g. parametric or non-parametric approaches
    • G06K9/6269Classification techniques relating to the classification paradigm, e.g. parametric or non-parametric approaches based on the distance between the decision surface and training patterns lying on the boundary of the class cluster, e.g. support vector machines
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/001Image restoration
    • G06T5/003Deblurring; Sharpening
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/50Image enhancement or restoration by the use of more than one image, e.g. averaging, subtraction
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10016Video; Image sequence
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10056Microscopic image
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20212Image combination
    • G06T2207/20221Image fusion; Image merging
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30004Biomedical image processing
    • G06T2207/30024Cell structures in vitro; Tissue sections in vitro

Abstract

The invention relates to a method for the evaluation of microscopic samples and to an apparatus for carrying out this method. The method according to the invention comprises the steps of: providing a carrier 1 with microscopic samples 7; Arranging the carrier 1 on a microscope stage 9; Imaging enlarging a section 3 of the carrier 1 by means of a digital camera 5 and a magnifying optic 11; the same section 3 being mapped multiple times, shifting a focal plane of the images during the mapping of successive pictures; Creating a combined image from the plurality of consecutive mappings; and evaluating the combined image for evaluation of the microscopic samples.

Description

  • The application relates to a method for evaluating microscopic samples and an apparatus for carrying out this method.
  • To quantify bacteria in a larger volume of liquid, for example 100 ml or more, the liquid to be examined is filtered in microbiological tests. Usually filters with a side length or diameter of about 2.5cm are used. Subsequently, the microorganisms are analyzed in a known manner by culturing the filters on agar plates and subsequent optical determination of microcolonies.
  • As a modern method for the determination of microorganisms, preferably bacteria, on filters has now established a specific staining method, the so-called fluorescence in situ hybridization (Fish). Here, the cells are visualized on a carrier / filter by fluorescence-labeled nucleic acid probe molecules under a microscope. This procedure is in: Amman et. all Microbial. Rev. 59 (1995), pages 143-169 described.
  • In other embodiments, specific staining methods may be applied to the filter itself, and the fluorescently labeled microorganisms may be counted by fluorescence microscopy.
  • Since the filter in the area is much larger than the field of view of the microscope, a large number of fields of view or sections must be counted for complete quantification of the microorganisms on the filter. Depending on the magnification, the size of the cutout and the size of the filter, there may be some 10,000 cutouts that need to be evaluated individually.
  • With a high concentration of microorganisms, it has therefore been natural to count only a statistically representative number of 10 - 100 segments and to extrapolate the result accordingly to the entire area. However, this approach is inaccurate and fraught with large errors. Also, the manual counting of microorganisms in a section is often difficult.
  • To solve this problem, it is known to scan the filter using automatic microscopes that have a motorized XYZ table and to take a picture of each cutout via a digital camera, ie to take a digital image. This image can then be analyzed with a pattern recognition or object recognition in a known manner.
  • Current embodiments of these automatic microscopes still require more than twelve hours for the evaluation of a carrier / filter of the type mentioned above.
  • A problem with this method is that the filter is not completely flat. That is, for each shot, the focal plane must first be adjusted so that a sharp image is obtained.
  • This focusing process to a target level, in which the microorganisms to be recognized are sharply imaged, is achieved in the prior art by detecting the surface of the filter via an auxiliary system, for example a separately fed load beam, and then the Z motor of the microscope stage The procedure is such that the filter surface is in focus. Due to the lack of planarity of the filter mentioned above, this focusing process must be repeated for each section. This focusing process is the most time-consuming step in the entire process compared to the actual recording of the digital image and the process of the table in the XY direction.
  • The object of the present invention is to provide a novel method for the evaluation of microscopic samples and a corresponding device, which can reliably and quickly determine the number of microscopic samples to be examined on a support.
  • This object is achieved by a method according to claim 1 and an apparatus according to claim 10. The dependent claims relate to further advantageous embodiments of the invention.
  • In the method according to the invention for the evaluation of microscopic samples, a carrier with microscopic samples formed thereon is first provided. This carrier is on a Microscope table arranged. With the help of a digital camera and a magnifying optics a section of the carrier is shown.
  • The method according to the invention is characterized in that the same section is scanned several times, wherein a focal plane of the images is displaced during the imaging of successive images.
  • From the thus obtained several successive images a combined image is created. This combined image is evaluated to evaluate the microscopic samples.
  • Since according to the invention no focusing step is necessary for the individual images, the method is determined only by the exposure time of the individual images, and thus considerably faster than the known process.
  • In a preferred embodiment of the method, the combined image is obtained by determining, for each pixel of the combined image, the maximum value of the positionally corresponding pixels of the plurality of consecutive mappings.
  • It is assumed that a pixel of an image assumes the maximum value if this image has the focal plane in the plane of the object to be imaged, ie the microscopic sample. In the areas of the section in which no microscopic sample is present, in which therefore no fluorescence takes place, while the maximum value of the pixels is also used, but this is determined only by the noise, and will be significantly different from a positive finding.
  • In a further embodiment of the invention, the combined image is evaluated by means of automatic pattern recognition, in particular based on a histogram of orientated gradient (HOG) in combination with a learning algorithm by means of Support Vector Machine (SVM).
  • HOG in conjunction with SVM is a well known pattern recognition technique used, for example, for face recognition or handwriting recognition. Examples of such pattern recognition are approximately in Hamayun A. Khan, "MCS HOG Features and SVM Based Handwritten Digit Recognition Systems", Journal of Intelligent Learning Systems and Applications, 2017, 9, pages 21-33 or in Chandrasheka, TR et. "Face Recognition Based on Histogram of Oriented Gradients, Local Binary Patterns and SVM / HMM Classifiers", International Journal of Engineering Sciences and Research Technology, August 2014, pages 344-352 , described.
  • The combined image obtained according to the invention is not clear to the human eye. However, the pattern recognition described above, when corresponding parameter sets are stored in a specially prepared database, allows a reliable and rapid detection of microorganisms on the carrier / filter.
  • According to a further aspect of the invention, different sections of the carrier are sequentially brought into the field of view of the digital camera and the optics, wherein for each section a plurality of images are created with different focal planes, a combined image is created and evaluated for each section, and based on the evaluation all sections the microscopic specimens on the support overall are evaluated.
  • With the method according to the invention, the XYZ table of the microscope can be moved automatically from one cutout to the next until the entire filter / the entire carrier has been scanned. In this way a fast and complete evaluation of the filter / carrier is possible.
  • According to the invention, the carrier is a filter which has been used to filter a microorganism-contaminated liquid and colored. Instead of a filter, the surface of a dish filled with agar or another culture medium can also be scanned.
  • Furthermore, it is preferred that on average between every two successive images the distance between the carrier and the digital camera be changed by 1-10 μm, particularly preferably by 2 μm, in order thereby to change the focal plane.
  • The selection of the distance is to be chosen in dependence on the size of the microorganisms or microcolonies investigated, so that there is a sufficiently high probability of actually having each microorganism at least once in focus in one of the images of a stack of successive images. In addition, the distance should also be sufficiently small so as not to have a significant change in the section due to the enlargement or reduction of the distance between the carrier and the digital camera, even in the case of a plurality of images. That is, in the conventional digital mappings where the pixels are arranged in rows and columns, a pixel of a first image located on a particular line and column also corresponds to approximately the same location on the filter as any other image of the same section with changed distance between carrier and camera.
  • In this way, it is easily possible to change the focal plane in small steps, with only the microscope stage in the Z direction must be moved.
  • It is not necessary for the distance to remain constant during the exposure time of an image.
  • Alternatively, it is of course also possible to adjust the optics so that the focal plane changes, or to move the digital camera, and to hold the carrier stationary.
  • According to a further advantageous method of the invention, it is provided to first determine a Ausgangsfokusebene, wherein distributed over the filter several focus values for sharp images of individual sections are determined, and wherein a Ausgangsfokusebene for the successive images of the remaining sections based on the average of the plurality of focus values , the amount of change in the focal plane between successive images and the number of consecutive shots for a combined exposure is calculated.
  • If, for example, 16 images of a section with a focal plane spacing of 2 μm are to be made, it makes sense to set the first focal plane offset by 8 μm behind the plane determined by the averaging of the focus values, and then in 2 μm steps the distance between the digital camera and to enlarge the carrier. In this case, images with focal planes in a range of plus minus 8μm will be obtained around the above determined average.
  • Furthermore, it is advantageous that for each cutout between 5 and 50, preferably between 10 and 30, in particular 16 successive images are produced, and processed into a combined image.
  • It has been found that a range of 32 μm is sufficient to obtain a useful combined image in conventional filters of the type described above.
  • Furthermore, it is preferable to change the focal plane continuously or stepwise. Whether a continuous or stepwise change of the focus plane is selected depends on the exposure time on the one hand and above all on the adjustment speed of the focal plane. It has proven to be advantageous to move the focal plane in stages, to comply with a waiting time, and then to move the focal plane about a further step. This can be done by synchronizing the exposure time of the camera and the means for adjusting the focal plane. Alternatively, it is also possible to ensure that only a timing of the camera and the microscope stage that to obtain a plurality of images of the same section with different focal planes. This is particularly advantageous if a complicated structure and a complicated control to be avoided.
  • The device according to the invention for carrying out the above-mentioned method comprises a microscope stage for arranging a carrier, a digital camera and a magnifying optical system for imaging an enlargement of a section of the carrier, means for changing a focal plane of the digital camera and the magnifying optical system with respect to the carrier A controller for controlling the imaging by the digital camera and changing the focus plane so as to produce a plurality of images of a section of the carrier having different focal planes, and an evaluation unit configured to generate a combined image from the plurality of consecutive images and the to evaluate the combined image to determine the microscopic samples.
  • In the following the invention will be described by means of preferred embodiments and with reference to the accompanying figures.
    • 1 : is a schematic view of the device according to the invention;
    • 2 : is a schematic view of the carrier with the neck and the samples to be evaluated according to the invention;
    • 3 Fig. 3 is a flowchart for explaining the method of the invention.
  • Essentially, the invention relies on taking a fast sequence of images rather than attempting to take a sharp focused image, rather than the microscope stage, for each field of view / excerpt of the wearer 9 in 1 in the Z direction. Within this recorded image sequence (for example 16 images with a distance in the Z direction of 2 μm), there are several images in which the microorganisms / cells are sharply imaged. By means of a calculation method, an overall image is calculated from this image stack, which contains all the information of the successively recorded images, and thus of course also the information of the fluorescence-marked cells.
  • The representation of the cells naturally differs from that which would be found for a focused image of the cells. Since a learning algorithm is used for object recognition, the cell can still be recognized as a cell, since the recognition algorithm is trained on the representation in the overall picture, and does not focus on a focused image of the cell.
  • By this method, it is possible to dispense with an automatic focusing, to allow even simple microscopes without mechanical and electronic focusing in the shortest possible time of less than two hours to measure and evaluate a carrier of the type mentioned. By dispensing with a technically complex and expensive autofocusing inexpensive fluoroscopic microscopes can be upgraded for automatic evaluation.
  • As stated above, an image stack of individual images focused in different planes is recorded at an arbitrary position on a filter / carrier. The images of an image stack thus captured differ only from the focal plane in the Z direction, with the X or Y direction not being changed, i. the images of a stack belong to a fixed XY coordinate on the filter / carrier. The size of the stack is chosen such that the plane that provides a sharp image of the cells is always located throughout the images of the stack throughout the filter / carrier. The average difference in Z-direction over a typical filter / carrier is in the range of 20μm, preferably with a coverage of 32μm.
  • According to the invention, the movement of the microscope stage is preferred 9 decoupled in the Z direction from the pure exposure time of the figure. This means that the movement of the microscope stage 9 not with taking single pictures through the digital camera 5 is synchronized. That's what the digital camera needs 5 be parameterized and provide continuous images, and on the other hand at the same time the microscope stage 9 be moved accordingly.
  • By further decoupling the image and then processing the images of the stack into a combined image, the scan / carrier scan rate is essentially the same as the exposure time, the number of images to be taken, and the distance between the images in the Z direction and the time required for moving the microscope stage 9 ,
  • The time required to take an image batch then becomes: T stack = T exposure * n .
    Figure DE102017009804A1_0001
    where n is the number of mappings of the stack.
  • In a preferred embodiment of the method, 16 images are taken, the exposure time being determined as a function of the signal intensity, the sensitivity of the camera and the aperture setting.
  • The distance relative to the Z-direction of the images within the stack and thus the total travel during the Z-movement is calculated from: S stack = S Illustration * n
    Figure DE102017009804A1_0002
    Wherein S stack is the total distance in the Z-direction and S illustration of traversing a single figure.
  • The resulting speed is given in a known manner by: V = S stack / T stack ,
    Figure DE102017009804A1_0003
  • In practice, the resulting velocity V is for commercial z-motors of microscope stages 9 too low, ie the travel speed must be artificially restricted by additional waiting times to meet the requirements of the exposure time of the camera. The process of the microscope stage 9 in the Z direction, this can be replaced by a clocked operation. For this purpose, the microscope stage is not moved continuously during the creation of the stack of images at a constant speed in the Z direction, but the microscope stage 9 is moved in steps with intermediate waiting times. Thereby the microscope stage becomes 9 initially only by the desired distance of the focal planes of two consecutive images proceed. Subsequently, a defined time is waited and afterwards the engine is renewed around this distance procedure, while the camera 5 continuously created a picture after another. The waiting time between two traversing commands is calculated from: T Waiting = T exposure * n Number of pictures / m Number of steps ,
    Figure DE102017009804A1_0004
  • By m number of steps is meant the number of desired motor breaks of the motor. This may differ from the number of illustrations. In particular, the number of steps may be greater than the number of mappings of a stack. For example, it is possible to first start the exposure of an image while the microscope stage 9 stands still, then the microscope stage 9 to proceed while the exposure of the same picture by the Kamrea 5 will continue. Then, pause the procedure again while still exposing the same image through the camera. After the end of the imaging time, the motor can be moved further. In this case, two or more focal planes are overlaid in the figure in the sense of a "multiple exposure".
  • The step size of the microscope stage 9 in the Z-direction during the exposure of an image may not be greater than the focus range of a cell level or not greater than the cell size itself. Otherwise, there is the possibility that a cell is not shown in any of the pictures "sharp". It could be lost to detection. A preferred value for the Z-distance between two images is according to the invention 1 to 10 .mu.m, more preferably 2 .mu.m.
  • The entire travel time can be calculated by including the manufacturer-specific engine speed, for example, then calculated from: T Travel time per step = S Illustration / V engine ,
    Figure DE102017009804A1_0005
    It is assumed that no significant time is needed to accelerate and decelerate the engine.
  • Because creating pictures through the camera 5 not with the movement of the microscope stage 9 Synchronized in the Z direction, occur in practice additional deviations that can be corrected. The correction factor is: K = n Total number of pictures / n Number of pictures already taken
    Figure DE102017009804A1_0006
  • To compensate for the additional deviations, the waiting time is dynamically adjusted using the correction factor during the current sampling: T Waiting corrected = T Waiting * K
    Figure DE102017009804A1_0007
  • Moreover, it is useful according to the invention to determine a start position of the focal plane for the creation of the images.
  • At the beginning of automated detection of a filter / carrier 3 Therefore, according to the invention, in a first step, the starting position of the focal plane of the stack is determined such that the cells are located at least in a region that is determined by the method of the microscope stage 9 is covered in the Z direction. For this purpose, in the simplest embodiment, three optional points are manually optimally focused on the filter edge and the resulting mean value for the focal plane (Z value) is used as the starting point.
  • In another embodiment, multiple points on the filter are manually optimally focused and then used to calculate a topology level of the filter. Based on this topology level, the starting point in the Z direction for the image stack is dynamically corrected based on the XY coordinates of the selected section on the filter / carrier.
  • The position for the start of the image stack then results, for example, from: P Figure pile = P Average - ( ( S images * n Number of steps ) / 2 )
    Figure DE102017009804A1_0008
  • In one example of the invention, a Leica DMRB fluoroscopic microscope with a 40x objective and a 14 megapixel color camera (Basler acA4600-10uc USB3 camera) was used. The microscope was extended by an automatic XYZ stage OptiScanII ® from Prior Scientific, Inc..
  • At an average shutter speed of the camera 5 of 100 milliseconds and a distance of the selected 20 images from each other by 2 microns at 16 steps results in the following values: Entity for a stack : 20 * 100 ms = 2000 ms Total travel for the stack : 2 μ m * 16 = 32 μ m waiting period : 100ms * 20/16 = 125ms
    Figure DE102017009804A1_0009
  • In this example, the number of images is greater than the number of traversing steps. Although two mappings should have the same focus plane, this does not matter in the proposed evaluation because only the maximum value of one pixel among the mappings of the stack is taken into account when creating the combined image.
  • So the motor drive command for 2μm is sent to the motor control and at the same time the timer for the waiting time is started. The motor stops automatically after 2μm and gets a drive command again when the timer for the 125ms has expired. The important thing is that the engine takes no longer than 125 ms for the track, which is certainly the case in reality.
  • In the following case the motor speed is 200μm per second
  • Then: T Travel time per step = 2 μ m / 200 μ m / s = 0 , 01s = 10ms
    Figure DE102017009804A1_0010
  • In this example, taking a picture batch takes a while 2 Seconds. Within this time, the camera takes 20 pictures at an exposure time of 100ms. At the magnification used of 40 is for the entire filter / the carrier 1 a sampling time of 1h 54min. The entire scanning time is additionally extended by the much smaller proportion for the process of the microscope stage 9 in XY direction to a maximum of 2 hours.
  • As shown at the beginning, a combined image containing the information of all the individual images of the stack is calculated on the individual images of an image stack. For this purpose, the maximum pixel value (PW) over all images of the stack is determined at each position and entered at the same pixel position in the combined image. This will be for each pixel of the images of the stack carried out. All images of the stack have the same pixel number given by the camera in the XY direction. PW ( X , Y ) combined picture = PW ( X , Y ) Max of all frames ,
    Figure DE102017009804A1_0011
  • In this way, a blurred image or image results for the human eye, since the pixel values, preferably brightness values, of blurred images / images-including the motion blur-lead to corresponding pixels in the combined image. However, an evaluation of these combined images can take place in that an adaptive recognition algorithm is trained specifically for these "new" cell-specific imaging objects. For this purpose, in the illustrated method, a combination of the object recognition based on HOG (Histogram of Oriented Gradients) together with learning algorithm SVM (Support Vector Machine) is trained on these imaging objects, so that these representatives are recognized for a fluorescence-marked cell. Details of the pattern recognition will not be further elaborated here, but they are known to those skilled in the in the above-mentioned documents for other purposes, such as handwriting recognition or face recognition.
  • In 1 you can see how the device in principle from the microscope stage 9 , the filter or carrier attached to it 1 , the optics 11 and the digital camera 5 is constructed. The microscope optics 11 and the camera 5 are shown here only schematically as a single lens and simple box. However, those skilled in the art are well-versed in the design of these components.
  • 2 shows a view of the carrier 1 with -in this marked section 3 on which the microorganisms 7 are arranged.
  • As described in the introduction, the method according to the invention requires that first, after the filter / carrier 1 on the microscope stage 9 and a section has been selected, an output focus level is set.
  • Subsequently, a section 7 of the filter / carrier 1 with the digital camera 5 Shown and saved this picture. In a next step - but preferably simultaneously with the exposure of the image - the focal plane is changed, for example in which the microscope stage 9 in the Z direction is moved by 2 microns.
  • The procedure then checked whether the maximum change in the focus plane has already been reached or whether the desired number of images of the stack has been created. If this is not the case, another image of the same section is produced and stored. The formation of an image and the method of the focal plane can take place simultaneously or in a coordinated manner, so that the method takes place in the pauses between two images. However, this is not preferred because it can extend the time for creating the image stack and there is synchronization between the microscope stage 9 and camera 5 must be set up.
  • Once the maximum change in the focal plane / maximum number of mappings of the stack has been achieved, a combined map is created from the stored mappings by selecting, for the value of a pixel of the combined map, the maximum value of the corresponding pixels of the individual mappings, as described.
  • On the combined image thus obtained, a pattern recognition method is used and the number of microorganisms / cells is determined.
  • In a next step, the microscope stage 9 moved in the XY direction to set a new section. It is conceivable, already during this movement in the XY direction, the microscope stage 9 also to move in the Z direction to the new output focus level. Alternatively, the Z-direction of movement can also be reversed when creating successive image stacks, so that the focal plane of the last image of a first stack is the output focal plane of the next stack.
  • It is checked whether all sections of the filter have already been scanned. If this is the case, the device terminates the process. If this is not the case, the output focus level is set again and a new batch of images is created.
  • Although the invention has been described above with reference to a preferred embodiment, the invention is not limited thereto. Various modifications and modifications can be made which are familiar to those skilled in the art.
  • The pattern recognition described here is just one example. Other pattern recognition techniques will be apparent to those skilled in the art and may be used.
  • With the invention, it is possible to reduce the usual processing times for automatically scanning a filter from 12 hours to approximately 2 hours. This considerably increases the throughput and the productivity in the evaluation of such filters and lowers the costs.
  • The invention has been described with reference to the scanning of a filter. It is not limited to this. It is also possible to microscopically examine the surface of a dish, for example a petri dish with agar medium.
  • Fluorescence was used in the invention. Again, this is not mandatory if other optical possibilities for detecting microorganisms or microcolonies are given, for example in transmitted light.
  • In the example described, the brightness value was selected as the pixel value. However, it is also possible to use the brightness value of a particular color to achieve better selectivity.
  • QUOTES INCLUDE IN THE DESCRIPTION
  • This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.
  • Cited non-patent literature
    • Amman et. all Microbial. Rev. 59 (1995), pages 143-169 [0003]
    • Hamayun A. Khan, "MCS HOG Features and SVM Based Handwritten Digit Recognition Systems", Journal of Intelligent Learning Systems and Applications, 2017, 9, pages 21-33 [0020]
    • Chandrasheka, T.R. et. "Face Recognition Based on Histogram of Oriented Gradients, Local Binary Patterns and SVM / HMM Classifiers", International Journal of Engineering Sciences and Research Technology, August 2014, pp. 344-352 [0020]

Claims (10)

  1. A method of evaluating microscopic samples comprising the steps of: providing a support (1) with microscopic samples (7); Arranging the carrier (1) on a microscope stage (9); Imaging enlarging a section (3) of the carrier (1) by means of a digital camera (5) and a magnifying optical system (11); characterized in that the same section (3) is mapped multiple times, wherein during the imaging of successive images a focal plane of the images is shifted; Creating a combined image from the plurality of consecutive mappings; and evaluating the combined image for evaluation of the microscopic samples.
  2. Method according to Claim 1 , characterized in that the combined image is obtained by determining, for each pixel of the combined image, the maximum value of the positionally corresponding pixels of the plurality of consecutive images.
  3. Method according to Claim 1 or 2 , characterized in that the combined image is evaluated by means of automatic pattern recognition, in particular based on a histogram of orientated gradient (HOG) in combination with a learning algorithm by means of Support Vector Machine (SVM).
  4. Method according to one of Claims 1 to 3 characterized in that different sections of the carrier (1) are sequentially brought into a field of view of the digital camera and the optics (11), wherein for each section several images are created with different focal planes, for each section a combined image is created and evaluated, and the microscopic samples are evaluated on the basis of the evaluation of all sections.
  5. Method according to one of Claims 1 to 3 , characterized in that the carrier (1) is a filter which has been used to filter a microorganism-contaminated liquid and colored.
  6. Method according to Claim 5 , characterized in that between each two consecutive images, the distance between the carrier (1) and the digital camera by 1-10 microns, preferably changed by 2 microns, so as to change the focal plane.
  7. Method according to Claim 5 or 6 characterized in that for determining an output focus plane distributed across the filter (1), a plurality of focus values for sharply displaying individual sections are obtained, and wherein the output focus plane for the successive maps of the remaining sections is based on the mean of the plurality of focus values, the extent of the change Focus plane between successive pictures and the number of consecutive shots for a combined recording is calculated.
  8. Method according to one of Claims 1 to 7 , characterized in that for each section 5 to 50, preferably 10 to 30, particularly preferably 16 consecutive images are processed into a combined image.
  9. Method according to one of Claims 1 to 8th , characterized in that the focal plane is changed stepwise.
  10. Apparatus for carrying out a method according to one of Claims 1 to 9 characterized in that the apparatus comprises: a microscope stage (9) for arranging a carrier (1); a digital camera (5) and magnifying optics for imaging enlarging a section (3) of the carrier (1), means for varying a focal plane of the digital camera (5) and magnifying optics with respect to the carrier (1); a controller (11) for controlling the image by the digital camera (5) and changing the focal planes so as to produce a plurality of images of a section of the carrier (1) having different focal planes; and an evaluation unit configured to generate a combined image from the plurality of successive images and to evaluate the combined image to determine the microscopic samples.
DE102017009804.2A 2017-10-20 2017-10-20 Method for evaluating microscopic samples and apparatus for carrying out this method Pending DE102017009804A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
DE102017009804.2A DE102017009804A1 (en) 2017-10-20 2017-10-20 Method for evaluating microscopic samples and apparatus for carrying out this method

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE102017009804.2A DE102017009804A1 (en) 2017-10-20 2017-10-20 Method for evaluating microscopic samples and apparatus for carrying out this method
PCT/EP2018/078834 WO2019077157A1 (en) 2017-10-20 2018-10-22 Method for evaluating microscopic samples and device for carrying out said method

Publications (1)

Publication Number Publication Date
DE102017009804A1 true DE102017009804A1 (en) 2019-04-25

Family

ID=64172451

Family Applications (1)

Application Number Title Priority Date Filing Date
DE102017009804.2A Pending DE102017009804A1 (en) 2017-10-20 2017-10-20 Method for evaluating microscopic samples and apparatus for carrying out this method

Country Status (2)

Country Link
DE (1) DE102017009804A1 (en)
WO (1) WO2019077157A1 (en)

Citations (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE19502472A1 (en) * 1995-01-27 1996-08-01 Fraunhofer Ges Forschung Method and device for picking up an object
US5745239A (en) * 1997-04-07 1998-04-28 Taiwan Semiconductor Manufacturing Company Multiple focal plane image comparison for defect detection and classification
US6055097A (en) * 1993-02-05 2000-04-25 Carnegie Mellon University Field synthesis and optical subsectioning for standing wave microscopy
WO2000025113A1 (en) * 1998-10-28 2000-05-04 Innovationsagentur Ges Mbh Device for visualizing molecules
DE10149357A1 (en) * 2000-10-13 2002-04-18 Leica Microsystems Imaging Sol Optical object surface profile measurement involves comparing contents of all images per point to determine plane using defined criteria, associating plane number, storing in mask image
EP1258766A2 (en) * 2001-05-14 2002-11-20 Robert Bosch Gmbh Optical system for shape measurement
DE10241290A1 (en) * 2001-09-11 2003-04-24 Leica Microsystems Method and device for the optical examination of an object
DE10237470A1 (en) * 2001-10-22 2003-04-30 Leica Microsystems Method and device for generating light microscopic, three-dimensional images
DE19944516B4 (en) * 1999-09-16 2006-08-17 Brainlab Ag Three-dimensional shape detection with camera images
DE10359780B4 (en) * 2003-12-19 2007-02-15 Pentacon Gmbh Foto- Und Feinwerktechnik Method for optical image acquisition
EP1929939A2 (en) * 2006-09-28 2008-06-11 JenLab GmbH Method and assembly for microscopic high-resolution reproduction or processing the laser endoscopy
WO2008125605A2 (en) * 2007-04-13 2008-10-23 Michael Schwertner Method and assembly for optical reproduction with depth discrimination
DE102007045897A1 (en) * 2007-09-26 2009-04-09 Carl Zeiss Microimaging Gmbh Method for the microscopic three-dimensional imaging of a sample
US7646482B2 (en) * 2007-05-31 2010-01-12 Genetix Limited Methods and apparatus for optical analysis of samples in biological sample containers
DE102004047928B4 (en) * 2004-10-01 2011-02-24 Carl Mahr Holding Gmbh Optical 3D measuring method and measuring device
DE102013006994A1 (en) * 2013-04-19 2014-10-23 Carl Zeiss Microscopy Gmbh Digital microscope and method for optimizing the workflow in a digital microscope
DE102015107367A1 (en) * 2015-05-11 2016-11-17 Carl Zeiss Ag Evaluation of Fluorescence Scanning Microscopy Signals Using a Confocal Laser Scanning Microscope
WO2017013054A1 (en) * 2015-07-17 2017-01-26 Leica Microsystems Cms Gmbh Light sheet microscope for simultaneously imaging a plurality of object planes
DE102015115615A1 (en) * 2015-09-16 2017-03-16 Technische Universität München Apparatus and method for chromatic-confocal examination of a sample
DE202017003181U1 (en) * 2016-06-21 2017-07-12 Cytena Gmbh Device for detecting cells or particles in a fluid container
DE102016116311A1 (en) * 2016-05-02 2017-11-02 Carl Zeiss Microscopy Gmbh Angle selective lighting
DE102016116620B3 (en) * 2016-09-06 2017-11-02 Stiftung Caesar Center Of Advanced European Studies And Research Beam guidance unit and system of beam guidance units and their use
DE102016125255A1 (en) * 2016-12-21 2018-06-21 Carl Zeiss Jena Gmbh Wavefront manipulator and optical device

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE10053202A1 (en) * 2000-10-26 2002-05-16 Gsf Forschungszentrum Umwelt Method for image acquisition of samples and optical viewing device for carrying out the method
AU2003230069B2 (en) * 2002-05-14 2008-11-13 Ge Healthcare Bio-Sciences Company System and methods for rapid and automated screening of cells
EP2936116B8 (en) * 2012-12-19 2020-04-01 Koninklijke Philips N.V. System and method for classification of particles in a fluid sample

Patent Citations (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6055097A (en) * 1993-02-05 2000-04-25 Carnegie Mellon University Field synthesis and optical subsectioning for standing wave microscopy
DE19502472A1 (en) * 1995-01-27 1996-08-01 Fraunhofer Ges Forschung Method and device for picking up an object
US5745239A (en) * 1997-04-07 1998-04-28 Taiwan Semiconductor Manufacturing Company Multiple focal plane image comparison for defect detection and classification
WO2000025113A1 (en) * 1998-10-28 2000-05-04 Innovationsagentur Ges Mbh Device for visualizing molecules
DE19944516B4 (en) * 1999-09-16 2006-08-17 Brainlab Ag Three-dimensional shape detection with camera images
DE10149357A1 (en) * 2000-10-13 2002-04-18 Leica Microsystems Imaging Sol Optical object surface profile measurement involves comparing contents of all images per point to determine plane using defined criteria, associating plane number, storing in mask image
EP1258766A2 (en) * 2001-05-14 2002-11-20 Robert Bosch Gmbh Optical system for shape measurement
DE10220824B4 (en) * 2001-05-14 2010-08-05 Robert Bosch Gmbh Optical measuring device
DE10241290A1 (en) * 2001-09-11 2003-04-24 Leica Microsystems Method and device for the optical examination of an object
DE10237470A1 (en) * 2001-10-22 2003-04-30 Leica Microsystems Method and device for generating light microscopic, three-dimensional images
DE10359780B4 (en) * 2003-12-19 2007-02-15 Pentacon Gmbh Foto- Und Feinwerktechnik Method for optical image acquisition
DE102004047928B4 (en) * 2004-10-01 2011-02-24 Carl Mahr Holding Gmbh Optical 3D measuring method and measuring device
EP1929939A2 (en) * 2006-09-28 2008-06-11 JenLab GmbH Method and assembly for microscopic high-resolution reproduction or processing the laser endoscopy
WO2008125605A2 (en) * 2007-04-13 2008-10-23 Michael Schwertner Method and assembly for optical reproduction with depth discrimination
US7646482B2 (en) * 2007-05-31 2010-01-12 Genetix Limited Methods and apparatus for optical analysis of samples in biological sample containers
DE102007045897A1 (en) * 2007-09-26 2009-04-09 Carl Zeiss Microimaging Gmbh Method for the microscopic three-dimensional imaging of a sample
DE102013006994A1 (en) * 2013-04-19 2014-10-23 Carl Zeiss Microscopy Gmbh Digital microscope and method for optimizing the workflow in a digital microscope
DE102015107367A1 (en) * 2015-05-11 2016-11-17 Carl Zeiss Ag Evaluation of Fluorescence Scanning Microscopy Signals Using a Confocal Laser Scanning Microscope
WO2017013054A1 (en) * 2015-07-17 2017-01-26 Leica Microsystems Cms Gmbh Light sheet microscope for simultaneously imaging a plurality of object planes
DE102015115615A1 (en) * 2015-09-16 2017-03-16 Technische Universität München Apparatus and method for chromatic-confocal examination of a sample
DE102016116311A1 (en) * 2016-05-02 2017-11-02 Carl Zeiss Microscopy Gmbh Angle selective lighting
DE202017003181U1 (en) * 2016-06-21 2017-07-12 Cytena Gmbh Device for detecting cells or particles in a fluid container
DE102016116620B3 (en) * 2016-09-06 2017-11-02 Stiftung Caesar Center Of Advanced European Studies And Research Beam guidance unit and system of beam guidance units and their use
DE102016125255A1 (en) * 2016-12-21 2018-06-21 Carl Zeiss Jena Gmbh Wavefront manipulator and optical device

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
Amman et. all Microbial. Rev. 59 (1995), Seiten 143 - 169
Chandrasheka, T.R. et. all. „Face Recognition Based on Histogram of Oriented Gradients, Local Binary Pattern and SVM/HMM Classifiers", International Journal of Engineering Sciences and Research Technology, August 2014, Seiten 344-352
Hamayun A. Khan, „MCS HOG Features and SVM Based Handwritten Digit Recognition Systems", Journal of Intelligent Learning Systems and Applications, 2017, 9, Seiten 21-33

Also Published As

Publication number Publication date
WO2019077157A1 (en) 2019-04-25

Similar Documents

Publication Publication Date Title
Fahrbach et al. Rapid 3D light-sheet microscopy with a tunable lens
US9645378B2 (en) Microscope and method for SPIM microscopy
Jonkman et al. Any way you slice it—a comparison of confocal microscopy techniques
US20180307005A1 (en) Multifunction Autofocus System and Method for Automated Microscopy
CA1272285A (en) Dynamic microscope image processing scanner
US8749627B2 (en) Method and apparatus for acquiring digital microscope images
JP2017194700A5 (en)
JP2017194699A5 (en)
US7885447B2 (en) Image acquiring apparatus including macro image acquiring and processing portions, image acquiring method, and image acquiring program
US9829691B2 (en) Microscope with at least one illuminating beam in the form of a light sheet
JP4917331B2 (en) Image acquisition apparatus, image acquisition method, and image acquisition program
US9189677B2 (en) Recording medium having observation program recorded therein and observation apparatus
US10462351B2 (en) Fast auto-focus in imaging
EP2256534B1 (en) In-vivo examination apparatus
US7576307B2 (en) Microscope with dual image sensors for rapid autofocusing
CN102298206B (en) Microscope and focus method
EP0834758B1 (en) Continuous volume imaging system for scanning microscopy
Yazdanfar et al. Simple and robust image-based autofocusing for digital microscopy
US8588504B2 (en) Technique for determining the state of a cell aggregation image processing program and image processing device using the technique, and method for producing a cell aggregation
US10241314B2 (en) Microscope apparatus and storage medium storing microscope apparatus control program
US8150137B2 (en) Image processing device, image processing program, and observation system
JP2009522604A (en) Automatic focusing method and system of automatic microscope
KR20120004991A (en) System and method for enhanced predictive autofocusing
EP1893971B1 (en) Laser-micro-dissection method and device for laser-micro-dissection
AU2007333324B2 (en) Method for assessing image focus quality

Legal Events

Date Code Title Description
R012 Request for examination validly filed