WO2007144386A1

WO2007144386A1 - Method for taking and evaluating image sequences

Info

Publication number: WO2007144386A1
Application number: PCT/EP2007/055850
Authority: WO
Inventors: Martin Daffertshofer; Silke Hinnah; Jürgen Müller; Jürgen Kupper
Original assignee: Evotec Technologies Gmbh
Priority date: 2006-06-14
Filing date: 2007-06-13
Publication date: 2007-12-21

Abstract

The invention relates to a method for taking and evaluating a plurality of image sequences comprising the following steps: a first reference image from a first sample which is to be analyzed is taken, the reference image comprising reference image data values which are arranged in a reference coordinate system, a selection of coordinate values is determined in the reference coordinate system taking account of reference image data values, a first sequence of examination images from the first sample is taken, the examination image comprising examination image data values which are arranged in an examination coordinate system, image properties in the first sequence of examination images are ascertained from examination image data values, with the ascertaining being effected at least at those coordinates in the examination coordinate system which essentially correspond to the previously determined selection of coordinate values in the reference coordinate system.

Description

Method for recording and evaluating biopsy sequences

The present invention relates to a method and a device for recording and evaluating image sequences.

Cameras suitable for capturing sequences of fast successive images are commercially available. In particular, a number of CCD or CMOS image sensors are available which allow image rates of about 100 / s to over 100 / s at typical image resolutions in the range of about 500 * 500 up to 2000 * 2000 image dots (pixels) ( eg Micron Technology Ine, MT9 M413, Fairchild CCD456, Cypress LUPA-1300). Cameras based on such sensors are usually used for monitoring one-off, fast processes, for example in materials testing, ballistics or automotive engineering. The large amounts of data accumulating during the recording of the image sequence are stored in the form of the complete image data in quickly addressable buffers and then transferred to an evaluation unit and evaluated there. The transmission and evaluation usually require a multiple of the actual observation time.

The use of such a camera on a microscope for recording BiSdsequenzen in lms cycle is described in T. Tanaami et al., Applied Optics Vol. 22, August 2002, p. 4704. Again, these authors use the known arrangement of camera and fast-addressable Cache for the entire image sequence (see Fig. 3 and Section 6 of the cited paper).

The analysis of kinetic processes in biological systems _/ especially in cells by the acquisition of a sequence of images is also common in the art. On the whole! In this case, relatively low frame rates, up to a maximum of typical video rates of 25 or 30 Hz, are used. Regardless of the frame rate, the entire image data of the sequence is generally stored and later evaluated, usually after completion of a whole series of measurements.

EP 1 348 124 Bl describes methods for analyzing kinetic processes in cells in the screening by recording image sequences. Specifically, it aims to account for the spatial motion of cells, as occurs at long observation times for the analysis of relatively slow cellular processes. The approaches of image processing disclosed in EP 1 348 124 51 require separate, relatively complex object recognition in each recorded individual image; they are therefore not suitable for the real-time analysis of rapid processes.

The use of imaging, in particular (fluorescence) microscopic, methods has become very important in the field of life sciences in recent years. In particular, the study of biological and biochemical processes in rows or cell networks benefits from the significantly differentiated information provided by imaging techniques, for example compared to measurements of total fluorescence of a reporter in a cell population.

On the other hand, there is a strong interest in the observation of dynamic processes in cells. For example, the family of FLIPR readers (manufacturer: Molecular Devices Inc.) is specific to the observation of temporal changes in membrane potential or intracellular Ionenkoπzentrationen designed. However, such devices intended for high-time-resolved observations only provide one total fluorescence intensity per well and measurement time

It is desirable to use the high information content of spatially resolved images also in the observation of dynamic processes. Generally, suitable detectors are commercially available (e.g., CMOS sensors that allow the acquisition of image sequences with clock rates less than 1 ms). The use of such sensors has so far been limited to the observation of short periods of time. This limitation is based on the need to capture and process very large amounts of data. For example, the following data rate is obtained when using a camera with 1000 * 1000 pixels, digitizing the brightness values at 8 bit / pixel and a frame rate of 1000 / s: 1,000,000 pixels / frame * 1000 frames / s * 1 byte / pixel = 1 GByte / s. In the prior art, this problem is solved by incorporating into the camera a data buffer (e.g., 1 to 4 GB of memory size) that allows capturing image sequences for a limited time (until the memory is full). Since already the

Transmission interface to the downstream PC, which takes over the image analysis or long-term storage, has a much smaller bandwidth, the subsequent transmission of image data to the PC typically requires a multiple of the recording time, the subsequent evaluation of the image data generally requires a much longer computing time. This is acceptable in the established applications of such cameras where one-time events (for example, Automobii crash tests) are observed. Examining numerous samples in a row, however, results in a massively reduced sample throughput, since only a fraction of the time can actually be used to record measurement data.

It is the object of the present invention to overcome the disadvantages known in the prior art and in particular a method and a - A -

To provide apparatus, which e.g. capture and process many sequences of images in rapid succession or a sequence of sequences of biases over a singer period in a time acceptable to the user. Preferably, in this case, the total duration required for acquisition and evaluation of the biid data should not significantly exceed the sum of the actual acquisition times.

This object is achieved by the methods and devices set forth in the independent claims. The dependent claims relate to preferred embodiments of the same.

The invention relates to a method for recording and evaluating a plurality of subject sequences, comprising the following steps:

Receiving a first reference image of a first sample to be analyzed, wherein the reference image comprises reference image data arranged in a reference coordinate system,

Determining a selection of coordinate values in the reference coordinate system, taking into account

Reference image data values,

Recording a first sequence of examination images of the first sample, the examination image comprising examination image data values arranged in an examination coordinate system,

Determining image properties in the first sequence of examination images from examination image data values, wherein the determination takes place at least at those coordinates in the examination coordinate system which are predetermined Selection of coordinate values in the Referenzkoordinateπsystem substantially correspond.

It is preferable that the reference coordinate system is identical to the examination coordinate system.

Further, it is preferable that a predetermined relationship exists between the reference coordinate system and the examination coordinate system, which assigns a coordinate value in the reference coordinate system to a coordinate value in the examination coordinate system.

It is also preferred that the determination of biofilm properties in the first sequence of examination subjects from examination biographical data values is essentially limited to those coordinates which essentially correspond to the previously determined selection of coordinate values in the reference coordinate system.

It is also preferred that groups are formed within the previously determined selection of coordinate values in the reference coordinate system.

It is preferable that the groups are formed on the basis of coordinate values in the reference coordinate system and / or reference image data values, preferably brightness values.

In particular, it is preferred that a mathematical operation is performed on those examination image data values whose coordinates in the examination coordinate data system correspond to the coordinates in

Reference coordinate system in at least one of the previously formed groups correspond, wherein the mathematical operation preferably comprises an averaging or integration of the examination image data values In a preferred embodiment, a first examination image is recorded within the first sequence of examination images of the first sample and image properties are determined there before a second examination image is recorded within the same sequence.

It is preferred that within the first sequence of examination images of the first sample a Mehrzah! taken and stored intermediately before image properties are determined on these examination images.

The selected coordinate values preferably serve as a mask for the determination of image properties of the examination images. In this case, each mask can have a plurality of objects, in particular cell membranes, cell nuclei and / or cytoplasm. It is preferred here for each mask to have a plurality of identical object types, for example a plurality of cell membranes, for later examination

In a particularly preferred embodiment of the invention, several, in particular all, objects of the first sequence of examination images are examined with regard to the temporal course of at least one image property, such as the image brightness. In this case, the time profiles are preferably linked to one another using mathematical methods.

For example, to determine the behavior of individual cell populations, the individual responses of all cells assigned to a cell population are summed up. In addition, the availability of single-cell responses opens up the possibility of different types of data optimization using mathematical methods. For example, one can divide a single cell response by the number of pixels associated with that cell's mask method. This allows a standardization and thus a better one Comparability of the individual single-cell responses, In this way, for example, a cell, which would normally respond with a strong change in brightness attribute this property even if the cell was only slightly lit / excited due to an error and therefore only weakly lit.

Alternatively, one can use threshold methods to determine at what time within the measurement the responses of the individual cells change significantly. In this way, for example, one can correct the responses of individual cells that were inadvertently excited too late, and thus compare with the responses of lines that were stimulated at the right time, etc.

It is preferred that a second sequence of examination images of the first sample or a second sample is taken.

It is also preferred that the second sequence of examination images is recorded, while Bsldeigenschaften are determined on the first sequence of examination images.

It is also preferred that the selection of coordinate values in the reference coordinate system is determined while the first sequence of examination images of the first sample is taken.

The invention furthermore relates to a device for carrying out the method according to the invention comprising:

Means for receiving a first reference sample of a first sample to be analyzed, the reference picture comprising reference picture data values arranged in a reference coordinate system, Means for determining a selection of coordinate values in the reference coordinate system, taking into account

Reference image data values,

Means for receiving a first sequence of examination subjects of the first sample, wherein the examination image

Includes examination image data values that are included in a

Are arranged examination coordinate system,

Means for determining Büudeigenschaften in the first sequence of examination images from examination image data values, wherein the determination is carried out at least at those coordinates in the examination coordinate system, which correspond to the previously determined selection of coordinate values in the reference coordinate system substantially.

It is preferred that the means for recording reference image and / or examination images comprise a camera, preferably a CCD or CMOS camera.

It is also preferred that the means for determining image properties comprise microprocessors, signal processors or gate arrays, preferably field-programmable gate arrays.

It is further preferred that it comprises means for secondary analysis of the determined image properties, wherein this center! preferably comprise microprocessors.

It is also preferred that the means for recording the examination images and the means for determining image properties are integrated in a unit which has a data interface to the means for secondary analysis of the determined biofilaments. In particular, it is preferred that the data interface is designed in such a way that it transmits the image properties determined from a sequence of examination images to the center within a period of time. transmits to the secondary analysis, which does not significantly exceed the time required to record the sequence of examination images.

Likewise preferred is that the data interface is designed such that a possible transmission of the complete

Examining data values of a sequence of examination images to the means for secondary analysis would require a period of time which substantially exceeds the time required for the acquisition of the sequence of examination images,

The method according to the invention and the associated device make it possible to store the data volumes, i. The examination image data, already immediately during the recording of the examination subject (hereinafter also referred to as "image sequence") significantly reduce., They allow in particular a substantially continuous image recording at high sampling rates and a corresponding processing of the data and thus the use of location and time resolved Observation in the automated (high-throughput) analysis of a large number of samples.

The basic idea of the invention is to first receive a single image ("reference image") for each sample to be examined and to subject it to image processing, in particular one or more masks for objects recognized in the image is then used in the evaluation - or already during the recording of the image data (such as the read out of the camera) - the examination sequence recorded in the following, the data present in each frame (also referred to as "examination image") initially on pixel level data directly on a Compacting object level. The individual frame can be discarded immediately after this operation.

The arithmetic operations that are desirable for evaluating the unit images of the library sequence are typically very simple. For example, object recognition can be performed in the reference image, the result of which is a "mask image" that contains an object number of the object to which this pixel is assigned in each pixel instead of a brightness value To calculate the object number from the mask image as well as the sacred value from the frame, and to compute for each object an averaged or integrated sanctity of all associated pixels in the frame.Similarly effective implementations can be performed so quickly that imaging and image analysis are comparable take a long time.

Preferably, the object recognition also takes place in the reference image in real time during the current measurement, particularly preferably before the recording of the image sequence is started. This makes it possible to compress the information from each image of the image sequence already during the current measuring operation and to discard the respective image. Thus, preferably no buffer memory is required for the entire image sequence recorded on a sample. Ideally, only the last read-out image is buffered and evaluated in real time, while the subsequent image is exposed.

In a preferred application, the method according to the invention and the associated device are used to investigate kinetic processes in biological cells. Preferably, a plurality of cells are present in one or more samples to be examined for this purpose. The subject matter of the analysis may be, in particular, cellular potentials or ion concentrations which are transmitted via appropriate environmental sensors Dyes (eg the Ca indicator Fura-2) are optically detectable. In this case, the optical read-out parameter may in particular be the fluorescence intensity at each predetermined excitation and observation wavelength, or the ratio of two fluorescence intensities which are recorded alternately or simultaneously at different excitation and / or observation wavelengths ("ratiometric imaging").

In addition, preferred applications are the detection of changes in the electrical signals or morphological changes in rows or cell networks, such as cardiac muscle cells, as well as the signal propagation on neurons. The realization of some special applications will be discussed in detail below.

The processes to be examined can occur spontaneously, or triggered and / or synchronized by external stimulation (triggering). For example, stimulation may be by coupling in electrical signals, irradiating optical radiation, adding chemical substances via a dispensing device, or releasing previously added but encapsulated substances, e.g. via UV radiation (photochemical release).

In a preferred embodiment, an optical structure is used, which essentially corresponds to a (fluorescence) microscope known in the prior art (cf. FIG. 1): the sample to be examined is irradiated with illumination light whose illumination wavelength is preferably in the visible range of the Spectrum, in the ultraviolet or infrared range. As a light source, for example, one or more lasers, light emitting diodes or lamps, if necessary, with downstream color filters, are used. A microscope objective detects radiation emitted by the sample and images it onto a camera. Optionally, this lens can also be used to direct the illumination radiation onto the sample be used (incident light microscopy); Alternatively, it is possible to direct the illumination from the lens direction opposite spatial direction to the sample (transmitted light microscopy). Optionally, the camera may detect the light at the time of illumination (scattered light microscopy), or be arranged by upstream filters to capture only light of other wavelengths. It is particularly preferred that fluorescent markers present in the sample emit light of a longer wavelength when excited by the moistening light, and the camera is set up to record this longer wavelength of light (fluorescence microscopy),

Preferably, a camera is used as the detector, which can accommodate several hundred BÜder per second. The camera already contains analogue / digital converters, which convert the brightness values of the individual pixels into digital numerical values. Optionally, this camera may contain a cache that can buffer the image sequence first. Preferably, the camera is equipped with a DatenschnittsteMe that can transmit the image data at the full speed with which they are incurred when taking a picture sequence to a downstream evaluation unit.

For example, the camera CMC 1300 / C from VDS Vosskühler, Osnabrück, is suitable. This camera captures images at a resolution of 1280 * 1024 pixels at a rate of up to 500 fps. It is equipped with a "Camera Link" interface, which can transmit the image data in real time to a subordinate unit, eg a frame grabber, via several serial transmission channels.There are considerable data rates, for example in the specified operating mode: 500 fps * 1280 * 1024 pixels / image * IByte / Pixel = 655 MB / s. Example: Preferred measuring procedure

In a preferred embodiment, the arrangement described above is used in the following measuring sequence:

(a) reference picture recording

After the sample to be examined has been positioned in front of the objective, a reference bit is first recorded. This is transmitted to an evaluation unit, preferably a commercial personal computer (PC), and subjected there to an image processing step. The aim of the image processing step is to recognize the individual objects present in the image field and to delimit them both from the background (intermediate spaces) and from each other. In this case, objects can be in particular the individual lines in the sample, but optionally also substructures within the cells; this can be further explained below. For object recognition, image processing methods known per se, such as the segmentation by means of heterogeneity or the watershed algorithm, as well as downstream filtering and classification on the basis of shape or size criteria, can be used. The objects recognized in the image are numbered (in arbitrary order). As a result of this image processing, it is known for each pixel which object it belongs to or whether it belongs to the background. This information will be used in the processing of the image sequence to be subsequently captured, hereinafter referred to as "object mask." This corresponds to a selection of such coordinate values in the reference coordinate system that designate pixels on which objects (as opposed to the image background) were observed.

Optionally, it is possible to limit the number of objects in the object mask. To get a sufficiently good statistical quality of the following As a rule, it is desirable to analyze a certain minimum number of objects (eg cells) in order to obtain kinetic measured quantities. This compensates for fluctuations in the biological properties of the individual cells, and in addition reduces the statistical noise of the optical measuring process itself. On the other hand, with an increasing number of considered cells, the computational outlay in image processing increases. Due to tolerances in the sample preparation, the number of objects present in the image field can only be predicted with a large fluctuation range. However, when analyzing the reference image, it is possible to specify an upper limit for the number of objects to be considered, or for the number of pixels belonging to objects. In this way it can be achieved that measurement data of comparable statistical quality are determined in different samples and on the other hand that the computational outlay for image processing does not exceed a predetermined upper limit. The latter is particularly important if the evaluation is to be carried out in real time, as described below as a preferred embodiment.

The reference image may preferably be recorded at the same excitation and observation wavelength at which the subsequent image sequence is also recorded. In this case, the same fluorescent labels used in the observation of the kinetic processes are also used to localize the cells. However, an exposure time longer than the exposure time for the individual images of the image sequence is particularly preferably used. This is possible because the high time resolution in the reference image is not required, and offers the advantage that the captured image can be converted by integrating the photocurrent can have a much better signal-to-noise ratio for a long time, allowing for reliable cell localization. Alternatively, the cells can be localized at different excitation and / or observation wavelengths, for example in scattered light microscopy or, using an additional fluorescence markers, which specifically serves to localize the cells, in fluorescence microscopy at other wavelengths. In particular, the use of the numerous dyes known to those skilled in the art for the specific staining of cell nuclei, cytoplasm or cell membranes is preferred.

(b) Recording the test sequence

Following the acquisition of the reference image, one or more image sequences are taken for analyzing the cellular kinetics. In rapid succession, a variety of Biidern is added. The time intervals between the shots of the individual images are preferably chosen to be constant. But it is also possible to choose a variable time frame for the recordings. In particular, a preferred possibility is the use of a substantially logarithmic or geometric time-lapse, in which the time interval between successive shots progressively increases by a constant factor. Such a time frame is particularly suitable for effectively detecting decay events, which essentially have an exponential profile. The same applies to the exposure times of the individual images: Here too, either a constant or different exposure times can be selected for the individual images; In particular, correspondingly varied exposure times can be reasonably combined with a variable grid of the recording times.

In a preferred embodiment, the fluorescence intensity at two different illumination and / or observation wavelengths is detected alternately in successive individual images. By a later ratio formation of the determined fluorescence intensities per cell, a so-called "ratiometric imaging" can be carried out with which, for example, spectral shifts in the fluorescence emission of a suitably selected, ambient-sensitive dye can be detected. Depending on the application detail presented at the beginning, a trigger event may also be triggered (reagent addition, electrical or optical stimulation). This can be done immediately or at a defined time interval before the beginning of the image sequence recording. However, it is also preferable in particular to start recording the image sequence first and to trigger the trigger event during the current recording. In this way, a starting value (zero value) is recorded for the later evaluation of the kinetic sequence. In addition, this procedure tolerates uncertainties in the temporal synchronization between image sequence start and trigger event, since in each case the trigger time is recorded in the Biidsequenz.

The recording of the image sequence is preferably carried out over a predetermined total duration or a predetermined duration after triggering the Triggerereigπisses. If the evaluation of the Biidsequenz is performed in real time (s, below), it is alternatively possible to continue the recording until a predetermined state of the sample is reached - such as the decay of a variable parameter to a predetermined percentage of the initial value,

Optionally, several trigger events can also be triggered within the recording duration, for example at equidistant times. This can on the one hand serve to increase the signal / noise ratio by triggering a series of similar stimulations, respectively detecting the reaction of the sample and then averaging over the Plural of measured curves is performed. In other cases, the response of the sample upon repeated stimulation may change systematically, and it may be desirable to detect these changes by observing a series of stimulus / response cycles. (c) evaluation

The evaluation of the recorded images of the BÜdsequenz done on the basis of the previously determined object mask. In each individual subject, an integrated or averaged intensity value is preferably determined for each object by averaging or integrating over all of the object points associated with the object, and then stored. The respective individual image can be discarded immediately after this determination of the object values. By storing the object values for all individual images, which each correspond to an observation time, a time series of intensity values is available for each object. This procedure is shown in FIG. 3 as a flowchart. Continuous arrows indicate the flow of the algorithm (control flow), dashed arrows indicate the flow of data. This notation is also used in the following flowcharts.

The evaluation of the individual images happens particularly preferably in each case immediately after taking the individual image. This optimizes the storage requirements, since in the ideal case only a single frame at a time must be stored for processing. During the exposure of the subsequent image, the list of object values is determined and the image is discarded in order to release the memory for the subsequent image. Alternatively, however, it is possible to make available buffer memories for a plurality of images, up to the complete image sequence. This allows the temporal decoupling of image acquisition and image processing. Thus, on the one hand temporal fluctuations in the image processing can be absorbed, if it can not be ensured that each frame can be processed completely within the exposure time of the next image. On the other hand, when using a sufficiently large buffer memory, a measurement is possible even if the image processing regularly takes longer than the recording. In this case, after recording a BÜdsequenz, the next image sequence will be until the image processing of all individual images has been completed, and thus the buffer memory is released again (cf. FIG. 4),

The use of a large buffer memory also allows a particularly preferred embodiment, in which the total duration of the measurement and evaluation process is less critically dependent on the duration of the image processing of the reference son. In the above-described Abiauf, it was initially assumed that the object mask determined from the reference image already exists at the beginning of the recording of the image sequence. This is necessary to be able to process the individual images directly on the basis of the object mask. This approach has the disadvantage that between the recording of reference image and image sequence, the object recognition in the reference image must be awaited. This object recognition usually involves complex arithmetic operations and takes some 100 ms to several seconds on typical PCs. This time can not be used in the measuring process and extends the total duration of the measurement. Therefore, it is particularly preferred according to the invention to start recording the bios sequence immediately after the reference image has been recorded and transmitted to a first evaluation unit (for example a PC). The image sequence is first stored in a buffer memory. The complete recording time is available for the analysis of the reference image in the first evaluation unit. After the recording of the image sequence has been completed, the then determined object mask is transferred to a second evaluation unit, which uses its help to analyze the individual images in the buffer memory. Parallel to this, the recording of a new reference image in another sample and its evaluation in the first evaluation unit can already begin. This paralleled Vorgeheπsweϊse is shown in Fig. 5.

The processing of the Biidsequenz happens in all the above-described constellations particularly preferably in the camera itself (Fig. 2a), This is possible if the camera includes a logic unit with sufficient computing power, such as a signal processor or a field programmable gate array (FPGA), as in the case of the above-mentioned CMC 1300 / C. Furthermore, this procedure requires that the object mask is transferred to the camera and held there in a suitable memory. The advantage of this method is that only considerably smaller data sets (one time series per object) are transferred from the camera to the downstream data acquisition unit instead of the extensive image data. This requires a data interface with lower bandwidth and relieves the downstream unit, which can then be eg a standard PC. In FIGS. 2a to 2c, arrows show the data flow within the system comprising camera and evaluation unit / PC. Thick arrows correspond to high data rates, thin arrows to lower data rates.

It is particularly preferred to already limit the completion of the image data from the detector to the areas in which objects are arranged in accordance with the object mask. Such a procedure is possible with suitable detectors, in particular CMOS bit sensors, which allow the targeted readout of individual image areas. An example is the CMOS image sensor MT9M413 from Micron Technology Ine. This procedure according to the invention makes it possible, for a given bandwidth of the readout interface of the detector, to achieve a higher sampling rate of the image sequence since the data of all the pixels of the detector need not be transmitted via the readout interface. It is particularly advantageous to use in combination with the optimization of the object mask described above: When determining the object mask from the reference image, it can already be ensured that the pixel areas specified by the object mask can be read out in a defined limited time. This can be done particularly preferably by limiting the number of image lines in which objects to be considered are located. For this example, such objects from the object mask are subsequently deleted again, which are located in line areas in which there are only a few objects, so that these lines can be omitted when reading the detector. Alternatively, it is possible to transmit the complete image data via a broadband interface to a downstream unit and to evaluate it on the basis of the object mask. Preference is given here to the use of a so-called frame grabber, ie a unit with data interface, buffer memory and own locomotive unit for carrying out the evaluation (FIG. 2b). However, it is also possible to use a less complex unit, which comprises only data interface and buffer memory and is operated in a PC, which takes over the evaluation of the individual images (FIG. 2c). In such a solution, the required evaluation time will generally be greater than the measurement time, so that it is only suitable for high-throughput applications to a limited extent.

Preferred application

Action potentials are typical changes in the membrane potential of excitable cells and tissues such as heart cells of the heart muscle or nerve cells of the central nervous system. It is known that drug candidates or their unwanted side effects in some cases affect the rate of propagation of action potentials i) within cells or tissues or ii) between cells. In a particular application, the method of the invention and the associated apparatus are for measuring changes in the rate of propagation of a membrane potential change. For example, to detect these in nerve cells, subcellular structures such as axons or dendrites (collectively called ninths) must be detected during object recognition. With the help of object recognition, the spatial arrangement of individual neurite segments in relation to the cell body is determined. This allows the spatially and temporally offset membrane potential change in the neurite segments and thus the propagation velocity to be determined. - Z l -

In addition to the rate of propagation, in some cases the kinetics of membrane potency change, i. For example, the rate of membrane potential increase (also called depolarization) or membrane potential drop (also called hyperpolarization) change. In a further specific application, the method and the associated device for measuring changes in the kinetics of a membrane potential change are used. For example, heart muscle cell cultures stained with a voltage-sensitive, fluorescent dye are incubated with a drug candidate. By means of object masks one can detect the fluorescence intensity changes in the cell membrane of the heart muscle cells. From this, the depolarization or hyperpolarization rate of the

Determine cardiac muscle impact potentials and compare them with the action potential kinetics of untreated cardiac myocytes.

The use case and the basic assumption are accordingly opposite to the boundary conditions of the present invention, which involves the analysis of very fast processes under the basic assumption of spatially stationary cells.

Application: Ion channel screening

In drug research, ion channels are considered an extremely important target class. This is due to the fact that the drugs currently on the market are the second most frequently used after the G-protein-coupled receptors (GPCRs) and are therefore the second most frequently used to counteract the superfamily of ion channels, thus having an effect. On the other hand, individual ion channel types are modulated by a wide variety of substances, which in some cases results in undesirable side effects

Ion channels are membrane proteins that, by momentarily opening a pore, temporarily disrupt the otherwise nonpermeable cell membrane for individual ion species make it permeable. The concomitant change in the ion concentration on both sides of the membrane changes the membrane potential drop across the cell membrane,

Fast voltage-sensitive dyes are fluorescent dyes that can stain cell membrane and whose properties (e.g., fluorescence intensity or excitation wavelength) change by changing the membrane potential. It is known that it is possible to measure membrane potential changes and thus, inter alia, the modulating effect of substances on ion channel by means of voltage-sensitive dyes.

In a preferred application case! the method according to the invention and the associated device for measuring membrane potential change in cells, in whose membrane ion channels are embedded. As shown in FIG. 12/17, an image with high spatial resolution is first recorded for this purpose and a "mask" of the individual cells or line regions, for example the cell membrane, is created with the aid of image analysis methods. In a second step, only the fluorescence intensities of the individual cell masks with high temporal resolution are recorded and evaluated. (Figure 13/17)

The method according to the invention and the associated device make it possible to acquire kinetic data using an imaging method. A significant advantage of imaging techniques compared to "whole wave measurements" is that unwanted signals of the sample that are not derived from the cells (eg cell breakage, "dirt", dust, etc) can be filtered out, increasing the signal - to - noise ratio and thus the data quality , Further advantages are shown in FIG. 7/17. Variations in cell size, degree of staining, illumination or excitation can lead to differences in the individual cell responses. Using data analysis methods such as curve fitting, temporal Aligning, normalizing these variations can be "eliminated". By setting threshold values, the mixed response of an inhomogeneous cell population can also be broken down into the contributions of the individual cell groups, as shown in the lower part of the figure.

Instead of ion channels, the method and apparatus of the present invention may also be used to determine the modulations of other membrane proteins that affect membrane potential. As an example, let us mention the group of transporters. The same applies to substances which can directly change the membrane potential, e.g. kanban-forming ionophores (e.g., valinomycin or gramicidin) or carrier ionophores (e.g., dinitrophenol or FCCP).

As an alternative to the determination of changes in the membrane potential, there are many test methods in which the change in an ion concentration by means of so-called ion sensitives or pH-sensitive dyes are determined. As ion-sensitive dyes can be, inter alia, known, commercially available calcium dyes (eg fluo-calcium indicators, fura indicators, indo indicators, Calcium Green ™ or Oregon Green ™, Molecular Probes) or sodium or potassium dyes (eg SBFI, PBFI, Sodium Green Na + indicator, CoroNa Green Na + indicator, CorolMa Red Na + indicator; Molecular Probes).

In a further Anwendungsfail the inventive method and the associated device for the measurement of Ionenenkonzeπtrationsänderungen. Essentially, one pursues a very similar approach, as pursued above for the determination of Membranpotentiaiänderungen, with the difference that instead of the voltage-sensitive dyes here ions or pH-sensitive dyes are used and for the masks another ZeÜbereich such the cytoplasm is identified. This is indicated in Figure 12/17 by the green colored cytoplasm of the cells,

Multiplextng I;

In the field of test systems / data analysis, there is a trend not only to determine the primarily interesting parameter (e.g., fluorescence changes as an indicator of membrane potential changes) but at the same time as much as possible! Additional information about the sample e.g. to get the effect of an applied substance. In this connection it is known that by means of imaging techniques e.g. Statements about the vitality of a cell or cell population can make. In a further application, the method and the associated device according to the invention make it possible to determine both a kinetic parameter (possibly with high temporal resolution) and, for example, a morphological parameter (possibly with high spatial resolution and longer measurement time), this is shown in Rg, 8/17. First, the procedure described above for determining changes in ion concentrations is analogous to that described above. In the following case, the temporal change of the cytoplasmic calcium concentration is determined with the aid of the "mask method." In a subsequent step, a high-resolution image of the sample is taken and a detailed bath analysis is carried out to obtain a morphological parameter such as, for example, the nuclear fractionation If necessary, this detailed Bold analysis can also be performed on the image originally taken to create the cell mask, and the high-resolution image may contain additional information (eg from a different staining).

The method according to the invention and the associated device are suitable for continuously changing the application just described ~~ ZD ^~

increasing number of kinetic parameters (possibly with multiple masks) and morphological parameters.

Multiplexing II:

Another trend in the field of test systems / data analysis is to capture several processes that occur in a timely manner. With this approach, e.g. determine whether and / or how the inhibition of an enzyme in a signaling cascade affects the different downstream reactions. Thus, the method according to the invention and the associated device allow, in a further application, e.g. It is important to point out that the two processes do so in parallel, but not necessarily at the same rate, and consequently with the same temporal progression Resolution must be measured.

To determine the time course of the two parameters, the mask method described above is again used. It may be sufficient to detect and evaluate the signals at different wavelengths using a single mask, as shown in the middle part of the figure. If possible, however, in the first step not only a cell mask but also a cell mask for the cytoplasm and the nuclei of the individual cells should be created in order to capture both parameters in parallel. This is illustrated in Figure 15/17.

The inventive method and the associated device also allow the parallel determination of two processes that run asynchronously or with a time offset. Another application is shown in Figure 16/17. For example, in addition to the change in the membrane potential and the time course of a morphological To determine change, one can again work with two different cell masks. In order to determine the change in the morphological parameter, for example, in the case shown here, it is necessary to calculate the ratio of the fluorescence intensities of the two cell masks.

The method according to the invention and the associated device are suitable for the application just described on both a multiplicity of cell masks and, if appropriate, a wide variety of mathematical ones

Expand calculations.

Multiplexing III:

An essential challenge of many test strategies is to determine the selectivity of an observed effect. So you want, for example to know as early as possible whether an observed effect caused by the interaction of a substance with a certain type of protein but not by the interaction of the same substance with one of the protein species closely related isoform. Furthermore, one would like to e.g. ensure that the observed effect occurs only in cells expressing the protein under investigation and not in cells in which the protein under investigation is not or only to a very limited extent present

In the established processes for determining selectivity, the associated tests are performed either simultaneously or at a later time in a separate sample chamber. This approach has the disadvantage that the conditions for the selectivity tests in some cases deviate greatly from the conditions for the primary tests, so that the interpretability of the data is limited.

In a further application, the method and the associated device according to the invention make it possible to determine kinetic parameters in different cells with the aid of a plurality of masks, as in FIG Fig "9/17 and 17/17 shown. In the case illustrated in FIG. 9/17, two different cell masks can be based on different staining with the aid of image analysis methods for the two different cell populations. Alternatively, it is shown in FIG. 17/17 that different cell populations, possibly based on different morphological features, can be assigned to different groups. Subsequently, the temporal course of the same parameter for the two cell populations can be determined separately, as it has already been summarized in the applications described above.

The method according to the invention and the associated device are suitable for expanding the application just described to a multiplicity of cell populations, so that, for example, the selectivity of a substance-protein interaction with a plurality of isoforms of the protein can be determined.

The method and the associated device according to the invention are suitable for freely combining the applications described in the sections Multipiexing I, Multipiexing II and Multipiexing III. For example, e.g. It is quite possible to use several groups of cell masks to determine the temporal course of several possibly asynchronous processes in parallel in several cell types and to record a spatially highly resolved image of the sample and to determine a series of further parameters by means of detailed image analysis.

Claims

claims

1. A method for recording and evaluating a plurality of image sequences of chemical and / or biological samples, in particular cells, which have been investigated in particular in screening plants, in particular cells, comprising the following steps:

Recording a first reference image of a first sample to be analyzed, the reference image comprising reference image data values arranged in a reference coordinate system,

Determining a selection of coordinate values in the reference coordinate system taking into account reference image data values,

Recording a first sequence of examination images of the first sample, the examination image comprising examination image data values arranged in an examination coordinate system, and

Determining image properties in the first sequence of examination subjects from examination data values, the determination taking place at least at those coordinates in the examination coordinate system which essentially correspond to the previously determined selection of coordinate values in the reference coordinate system.

2. The method according to claim 1, characterized in that the reference coordinate system is identical to the examination coordinate system.

3. The method according to claim 1, characterized in that there is a predetermined relationship between the reference coordinate system and the examination coordinate system, which assigns a coordinate value in the reference coordinate system a coordinate value in the examination coordinate system.

4. Method according to claim 1, characterized in that the determination of image properties in the first sequence of examination images from examination image data values is essentially limited to those coordinates which essentially correspond to the previously determined selection of coordinate values in the reference coordinate system.

5. Method according to one of the preceding claims, characterized in that groups are formed within the previously determined selection of coordinate values in the reference coordinate system,

6. The method according to claim 5, characterized in that the groups are formed on the basis of coordinate values in the reference coordinate system and / or Referenzbilddateπwerten, preferably brightness values.

7. Method according to claim 5, characterized in that a mathematical operation takes place on those examination image data values whose coordinates in the examination coordinate system correspond to the coordinates in the reference coordinate system in at least one of the previously formed groups, wherein the mathematical operation preferably comprises averaging or integration of the examination image data values .

8. The method according to any one of claims 1 to 7, wherein an image processing of the reference image, in particular by means of an evaluation, such as a data processing device for particular spatial delimitation of objects contained in the sample, in particular cells is performed.

The method of claim 8, wherein after performing the image processing, each BUD point is associated with an object such as a cell or the background.

10. The method according to any one of the preceding claims, characterized in that within the first sequence of Untersuchungsbifdem the first sample a first examination biography is taken and this Bildigeπππschaften be determined before within the same sequence, a second examination image is taken.

11. The method according to any one of the preceding claims, characterized in that taken within the first sequence of examination images of the first sample, a plurality of examination images and zwischeπgespeichert before image properties are determined on these examination images.

12. The method according to any one of the preceding claims, wherein the selected coordinate values serves as a mask for the determination of image properties of the examination images.

13. The method of claim 10, wherein each mask have a plurality of objects, in particular different cell areas such as cell membranes, cell nuclei and / or cytoplasm.

14. The method of claim 12 or 13, wherein for several, in particular all objects of the first sequence of examination images, the time course of an image characteristic, in particular the brightness is determined.

15. The method according to claim 14, wherein in particular for determining the overall response of a cell population or for correction purposes, the temporal courses of individual image properties of different objects are linked to one another by means of mathematical methods.

16. The method according to any one of claims 12 to 15, wherein the selection of the coordinate values due to the image processing takes place.

17. The method according to any one of claims 12 to 16, wherein in each frame for each object an integrated and / or gemsttelter intensity value is determined.

18. Method according to claim 16, wherein by storing the object values for all individual images, which possibly correspond to an observation point, a time series of intensity values is made available for each object,

19. The method according to any one of the preceding claims, characterized in that a second sequence is taken by investigators of the first sample or a second sample.

20. The method according to claim 19, characterized in that the second sequence of examination images is recorded, while on the first sequence of examination images image properties are determined.

21. The method according to any one of the preceding claims, characterized in that the selection of coordinate values in the reference coordinate system is determined while the first sequence is taken by investigators of the first sample.

22. The method according to any one of claims 1 to 21, wherein the recording of the images by means of a detector, in particular, only the Büddaten be read from areas in which objects are arranged due to the detected object mask.

23. An apparatus for carrying out the method according to any one of claims 1 - 22 comprising:

Means for receiving a first reference image of a first sample to be analyzed, the reference image comprising reference image data values arranged in a reference coordinate system,

Means for determining a selection of coordinate values in the reference coordinate system taking into account reference image data values,

Center! for recording a first sequence of examination subjects of the first sample, wherein the examination image

Includes investigation budget data values that are contained in a

Are arranged examination coordinate system,

Means for determining image properties in the first sequence of examination subjects from examination image data values, wherein the determination takes place at least at those coordinates in the examination coordinate system which are predetermined Selection of coordinate values in the reference coordinate system substantially correspond.

24. The device according to claim 23, characterized in that the means for receiving Referenzbüd and / or investigators include a camera, preferably a CCD or CMOS camera,

25. The apparatus of claim 23 or 24, characterized in that the means for determining image characteristics include microprocessors, signal processors or gate arrays, preferably field-programmable gate arrays.

26. Device according to one of claims 23 to 25, characterized in that it comprises means for secondary analysis of the determined image properties, said middle! preferably comprise microprocessors,

27. The device according to claim 26, characterized in that the means for receiving the examination images and the middle! for the purpose of determining office properties, they are integrated in a unit which has a data section to the means for secondary analysis of the determined image properties

28. The device according to claim 27, characterized in that the Datenschnϊttsteile is designed such that it transmits the determined from a sequence of examination images Buuteigenschaften within a period of time to the means for secondary analysis, which does not wesentiich the time required for recording the sequence of examination images exceeds.

29. Device according to claim 23, characterized in that the data interface is designed in such a way that any transmission of the complete

Exam image data values of a sequence of examination images to the secondary analysis means would require a period of time substantially exceeding that required to acquire the sequence of examination images