WO2001001109A1

WO2001001109A1 - Device for automatically analysing and counting objects and data processing method

Info

Publication number: WO2001001109A1
Application number: PCT/FR2000/001784
Authority: WO
Inventors: Christophe Dubessy; Jean-Louis Merlin; Emmanuel Laidelli
Original assignee: Centre Alexis Vautrin (Etablissement Prive)
Priority date: 1999-06-28
Filing date: 2000-06-27
Publication date: 2001-01-04
Also published as: FR2795516A1; FR2795516B1

Abstract

The invention concerns a device for liquid, semi-liquid or solid media comprising in combination a sample feeding automaton (1), a sample identifying system, a sample illuminating set (3), an image sensing optical assembly (2), an electronic module controlling (5) said assemblies, the sample to be analysed being set in a box (6) arranged on a mobile support (7) of the automaton (1). The invention is characterised in that the mobile support (7) displaces the box (6) in the illuminating zone along a first direction (X) and a post (12) bearing the optical assembly (2) moves along a second horizontal direction (Y) perpendicular to (X). The invention also concerns a method for controlling the device and processing the data.

Description

Automatic object analysis and counting device and data processing method

The present invention relates to a device for automatic analysis and counting of objects in transparent or translucent liquid, semi-solid or solid media, and a method for controlling and managing the device and for processing data.

More particularly, this device is intended for research laboratories, for example in oncology and the pharmaceutical industries practicing the analysis of spheroids and the technique of clonogenic tests.

Spheroids are spherical cellular clusters whose structure is close to that of micro-tumors. They therefore constitute a model of choice in oncology for radio and chemotherapeutic studies. A sensitivity study therefore consists in measuring the survival rate of the spheroids as well as their size after treatment. In order for the information to be precise and reproducible, the analysis must be carried out on as many spheroids as possible.

The technique of clonogenic tests, another method of assessing sensitivity to chemotherapy drugs and radiation, consists of counting thousands of colonies which develop in a semi-solid medium or at the bottom of the dish.

These two techniques which require microscopic analysis, therefore require a great deal of time and cause considerable visual fatigue for the operator. The first objects of the invention are to propose an analysis and counting device the resolution of which is sufficient to apply to the aforementioned fields, which must moreover be controlled by a computer so as to fully automate the analysis protocols. and counting. To achieve this control, it is necessary to create appropriate software; in fact, the known devices do not have sufficient resolution and / or are not automated.

To achieve these goals, the applicant encountered several problems. First of all, for a shooting system to scan the entire counting surface comprising the samples, it is necessary to carry out a relative displacement between shooting of the sample in two horizontal directions X and Y. A first solution would consist in make the table carrying the sample mobile, but this would result in a very large table surface (nine times that of the box containing the sample) and would generate vibrations in the liquid sample.

To overcome this drawback, another solution would consist in placing on a single table a group of fixed samples (for example one hundred for application to spheroids) with an arm moving above each of them. This solution would be very cumbersome.

To overcome all these drawbacks, the applicant has chosen a compromise by separating the two movements, thus the box containing the sample is movable in a first horizontal direction X and the imaging system is movable in a second horizontal direction Y and perpendicular to X.

The vibrations of the box are reduced and do not hinder the acquisition of measurements, even on liquid media.

The plaintiff has had to solve another problem of another order: that of lighting because the samples to be analyzed have little contrast with the background; they can be white and the outline of the spheroids is not detectable by the shooting.

Known laser lighting systems destroy cells and cannot be used. Known contrast or coloring treatments cannot be used because they can modify or even destroy the samples.

A frosted glass system is not satisfactory because it gives a whitish appearance and without relief to the cells.

A direct lighting system cannot be suitable for lack of contrast and relief, and the presence of cast shadows distorting the calculations

All these problems are solved by the invention which consists of a device for analyzing objects contained in liquid, semi-solid or solid media, of the type comprising in combination: a sample-passing automaton - a system for identifying samples ,

- a set of lighting of the sample,

- an optical assembly for image taking,

an electronic module for controlling the aforementioned assemblies, the sample to be analyzed being placed in a transparent cell culture box placed on a mobile support of the automated device, characterized on the one hand in that the mobile support moves the box in the lighting area according to a first horizontal direction (X) and in that a mast carrying the optical assembly moves in a second horizontal direction (Y) perpendicular to (X) and on the other hand in that the lighting assembly is designed specifically to provide a significant contrast to the objects contained in semi-solid or solid white liquid samples and in that it does not destroy the cells of the sample.

Preferably, the Applicant had the idea, to solve the problem, of using pots with a black background in combination with lateral and radial lighting making it possible to accentuate the contrasts necessary for the shooting without destroying the samples.

These objectives are achieved by a lighting system comprising a light generator and an optical fiber for transporting the light distributing the light radially and in a homogeneous manner to remove the shadows cast, a black background being placed under the box.

According to a first variant, an optical fiber opens out laterally into a black-bottomed pot placed under the box, a ring of optical fiber being placed inside and at the periphery of the pot and being connected to the optical fiber.

According to a second variant, an optical fiber conducts the light to an annular illumination placed above the box.

To automatically manage the entire device and process the data, the method according to the invention comprises the following steps: configuration steps with saving of the settings, - image analysis steps comprising at least the actual analysis - processing results

In a nonlimiting manner, the image analysis steps also include a separation step making it possible to activate or not the separation of the contiguous objects then their reconstruction.

If the separation of the contiguous objects is activated, one carries out an extraction of the connected components starting with a simple extraction, followed by an ultimate erosion determining the number of objects composing the sub-image, followed by a thickening to find the image initial with unjoined objects, finally followed by a Hough transform to reconstruct the missing part of the spheroid. The invention will be better understood using the following description made with reference to the following appended figures:

• Figure 1: schematic side view of an analyzer device according to the invention; • Figure 2: schematic front view of an analyzer device according to the invention;

• Figure 3: flow diagram of the functions of a spheroid analysis software according to the invention.

• Figure 4: sketch showing a first alternative embodiment of the lighting device. First, reference is made to FIGS. 1 and 2 showing the analyzer device integrating:

- a sample changer (1),

- a sample identification system (not shown)

- a sample lighting assembly (3) designed specifically for solid or semi-solid white samples

- an optical assembly (2) for taking images allowing a resolution of the order of 5 μm (or even better).

- A computer (4) comprising control software controlling the electronic control module (5) of the above-mentioned assemblies, image analysis and data processing software.

A sample to be analyzed is contained in a box (6) (for example standard type P6 / P96, Petri dishes) placed on a mobile support (7) of the automatic device.

(1) -

The carpet (7) raised relative to a chassis (9) and actuated by a motor (8) moves the box (6) in the lighting area in a first direction (X).

The samples are identified by any known system, for example by barcode or others (for example ...).

The optical assembly (2) consists of a sensor usually known by the name of CDD sensor (10) and a macro zoom (11) allowing a resolution of the order of at least 5 μm to be expected.

The sensor (10) and zoom (11) assembly is guided vertically along a mast (12) by a fixing means (13) adjustable in height for example by means of knobs (14).

The mast (12) is moved by a mobile support (15) driven by a stepping motor (16) in a second horizontal direction (Y) perpendicular to the first horizontal direction (X). The combination of movements along X and along Y sweeps the entire surface of the box containing the samples. Optionally, one or two loading towers (19) can be provided for depositing the samples on the mobile support (7).

The lighting assembly (3) is specifically designed to provide significant contrast to the objects contained in samples of transparent or translucent media which must not be destroyed or modified by analysis.

According to a first alternative embodiment (Figure 4), the light generated by the light generator (17) is brought by an optical fiber (18) opening laterally in a pot (21) then is distributed radially and homogeneously in this pot by means of a ring (20) of optical fiber arranged inside and at the periphery of the pot, and connected to the optical fiber (18). The box (6) containing objects to be counted is placed above this pot which must have a black background so as to create a contrast, the objects standing out in white on the black background.

The lateral and radial lighting creates a grazing light visualizing the reliefs, distributed homogeneously over the entire surface of the sample so that the binarization described below is homogeneous.

Point or directional lighting would not be suitable; indeed it would produce for each object a drop shadow whose surface would be added to that of the object, thus distorting the results.

According to a second alternative embodiment (not shown), the box (6) can be lit from above, for example by annular lighting surrounding the macro-zoom (11), the ring (20) being deleted, but the black background being held to create contrast.

The electronic module (5) and its control software are within the reach of those skilled in the art; it controls the motors and other elements of the device, according to the steps of the process below.

The method for analyzing samples, more particularly spheroids, and for processing data has been specifically designed for this application; it is preferably automated by corresponding software.

Its flowchart is shown in Figure 3 and it has three groups of steps: A: the steps for configuring the process and its software:

Before counting the spheroids, it is necessary to configure. All the settings are saved, thus avoiding complete reconfiguration with each use. However, it is recommended to perform the calibration before each use, in particular to take into account variations in light.

A.1. camera-configuration We chose a camera from those connected and managed by the software (for example four). A specific comment and a calibration coefficient (returned by the calibration function) are assigned to each camera.

A .2. configuration-calibrations Accessible by the configuration of the choice of camera. A conversion factor establishes a correlation between the number of pixels counted and the actual diameter of the object. For this, three calibers of different sizes (example: 200, 400, 600 μm) are analyzed three times. The regression line is then calculated and the slope (therefore the multiplying coefficient between the number of pixels and the actual diameter of the object) is automatically displayed in the "coefficient" area of the chosen camera. This coefficient is affected by the amount of light, so it seems necessary to calibrate the process for each use.

A3. configuration-settings Elimination of the results table of objects whose size is less than the desired threshold.

Example: if a diameter of 20 μm is entered, all objects whose diameter is less than 20 μm will not be taken into account in the results (although they are identified and then counted). This allows not to take into account cellular debris, isolated cells, small spheroids, etc.

A.4. configuration-histogram

A histogram makes it possible to view in real time the distribution of the population analyzed. It is possible to give the values of the minimum and maximum limits as well as the pitch. The histogram is readjustable at any time, allowing you to target only a population of objects. The ordinate axis is automatically readjusted according to the number of objects counted.

B - Image analysis steps

B.1. - the actual analysis:

The image analysis takes place in several parts:

Acquisition

The acquisition of an image is done via the library of the Matrox Météor acquisition card (registered trademark) which is an acquisition procedure specific to the software. The image is then converted so as not to depend on a proprietary API (Application Programming Interface belonging to a company) and therefore to improve the scalability and conversion of the software on other platforms.

Acquisition can be simple; it makes it possible to carry out acquisitions on the fly, or series: it makes it possible to carry out acquisitions associated with a comment chosen from among a plurality (for example 10) stored in memory.

binarization

It is the conversion of the grayscale image to black and white. This implies automatically finding the optimal binarization threshold.

This is done by smoothing the histogram of distribution of the gray values of the image in order to identify the two gray level peaks. The first maximum corresponds to the “dark” values therefore the pixels constituting the background, the second peak consists of the pixels of the spheroids therefore “light”.

The threshold is chosen as the minimum between these two peaks. That is, the gray level values lower than this level will take the value 0 (black), the higher values will take the value 1 (white). The image then consists only of black (= 0) and white (= 1) pixels.

Simple extraction

The image on which the software works is not noised by erosion / expansion filters so as not to alter the image. The image is scanned so as to locate the white pixels. For this, each pixel is analyzed successively. When a white pixel (therefore a possible spheroid) is located, it is placed in a storage matrix with the adjacent pixels of the same color. The image is therefore broken down into a succession of sub-images, each containing an object which will be processed so as to determine its surface and its perimeter.

B.2. - separation This is a special image processing which activates the separation of contiguous objects and then their reconstruction via the transform of

Hough.

If the separation is not activated, the software goes directly to processing the results (see C below). If the separation function is activated, go to B3

B.3. - extraction of related components

It is possible to activate this function allowing to separate the contiguous objects. In return for better accuracy in the counts, the calculation time is extended.

Extraction

Image processing begins with a simple extraction returning a set of sub-images to be processed (see above).

Ultimate erosion Erosion is not used here for the purpose of eliminating background noise but to determine the number of objects making up the subimage.

The image is scanned and the neighborhood of each pixel is analyzed, if it has a black neighbor, it becomes black; if it has only white neighbors, it retains its original value

(black = 0, white = 1). Mathematically, it is a "logical AND". The operation is repeated n times until the components disappear (we then have a black image).

The software works on the sub-image n-1 and counts the number of isolated pixels remaining, which indicates the initial number of related objects.

Sub-images with a single object are not processed and the original image is used to perform the extraction. Only the sub-images having connected components are processed this in order to lighten the processing of the image. Thickening

Dilation is the opposite of erosion and works on the following principle: if a pixel has a white neighbor, it becomes white. If it has only black neighbors, it is kept in its original color. Mathematically, it is a "logical OR". The thickening also makes it possible to expand the components without the edges coming into contact again thanks to a collision detection algorithm.

The selected sub-images are thickened n-1 times which makes it possible to find the initial image with unjoined objects (unlike dilation).

Hough transform

This set of treatments and functions makes it possible to “virtually reconstruct” the missing part of the spheroid in order to determine the geometric characteristics of the extracted objects, and consists of: Extraction of the contour

The sub-image is scanned, as in a simple extraction, to extract the objects which are now separated.

Vote for the center

For each of the pixels constituting the periphery of the object, the tangent is calculated by the method of least squares. This makes it possible to find the line perpendicular to the tangent passing through the pixel concerned.

The operation is repeated for all the points around the object. The set of drawn lines identifies a set of points in the central area of the object where the majority of the lines pass. Each of the points identified can pretend to be the center.

Department vote

Several centers per object are then available. The best radius for each center is sought and only the best is selected.

Reconstruction The radius being available, it is simple to calculate the diameter, the surface, the perimeter and the estimated volume of each of the components considered.

C - Processing of results

A prior configuration of the camera is necessary (see above in the description and see also path (a) in Figure 3). C.1. Calculation of results

The real surface (simple extraction) or calculated (reconstruction by extraction of the connected components) in pixels of each object is converted into μm. From the surface are calculated the diameter, the perimeter and the volume.

The preliminary paths (b) and (c) are necessary and already explained previously.

C.2. - Display

The results are displayed in the form of a histogram whose parameters have already been configured (path (d) figure 3).

D - Save-print

D.1. Impression

Printing of results is possible as well as configuration of printing

D.2. safeguard

The saving of the results tables is carried out on all the data or only on the data useful for the spheroids, or the clonogenic tests this in order to simplify the results and save space.

The backup takes place directly if the acquisition was not a simple acquisition. Otherwise (serial acquisition), the software returns to the simple image analysis function. Throughout the application, it will be understood that the term object is synonymous with particles.

Claims

1. Device for analyzing objects contained in liquid, semi-solid or solid media, of the type comprising in combination: an automatic sample changer (1)

- a sample identification system, a lighting assembly (3) for the sample, - an optical assembly (2) for taking an image,

- an electronic control module (5) of the aforementioned assemblies, the sample to be analyzed being placed in a box (6) arranged on a mobile support (7) of the automaton (1), characterized on the one hand in that the mobile support (7) moves the box (6) in the lighting area in a first horizontal direction (X) and in that a mast (12) carrying the optical assembly (2) moves in a second horizontal direction (Y) perpendicular to (X) and on the other hand in that the lighting assembly (3) is designed specifically to bring a significant contrast to the objects contained in semi-solid or solid liquid white samples and in that 'it does not destroy the cells of the sample.

2. Device according to claim 1, characterized in that the movable support (15) is driven by a stepping motor.

3. Device according to one of claims 1 to 2, characterized in that it further comprises at least one loading tower.

4. Device according to one of claims 1 to 3, characterized in that the lighting assembly (3) comprises a light generator (17) and an optical fiber (18) for transporting the light distributing the light radially homogeneously to remove the shadows cast, a background not placed under the box (6) creating contrast.

5. Device according to claim 4, characterized in that the optical fiber (18) opens laterally into a pot (21) with a black background placed under the box (6), a ring (20) of the optical fiber being disposed at the inside and at the periphery of the pot and being connected to the optical fiber (18)

6. Device according to claim 4, characterized in that the optical fiber (18) conducts light to an annular lighting placed above the box (6).

. Method for the analysis of liquid, semi-solid or solid media, characterized in that it is designed to manage a device according to one of claims 4 to 6 and characterized in that it comprises at least the following steps: - configuration steps (A) with saving of settings,

- image analysis steps (B) comprising at least the actual analysis (B1)

- processing of results (C)

8. Method according to claim 7, characterized in that the configuration steps (A) include a camera configuration step (A1) in which a comment and a calibration coefficient are assigned to each camera.

9. Method according to one of claims 7 to 8, characterized in that the calibration steps (A) include a configuration-calibrations step with a conversion factor between the number of pixels counted and the actual diameter of the object .

10. Method according to one of claims 7 to 9, characterized in that the calibration steps (A) include a configuration-settings step eliminating objects whose size is less than a threshold.

11. Method according to one of claims 7 to 10, characterized in that the calibration steps (A) include a configuration step-histogram readjustable at any time.

12. Method according to one of claims 7 to 11, characterized in that the actual analysis (B1) takes place in several parts: acquisition, binarization, extraction.

13. The method of claim 12, characterized in that the acquisition of an image is done via an acquisition procedure specific to the software and is then converted.

14. Method according to one of claims 12 to 13, characterized in that the acquisition is chosen from the set (simple acquisition, serial acquisition).

15. Method according to one of claims 12 to 14, characterized in that the binarization converts the grayscale image into black and white, the image then being only made up of black pixels and white pixels.

16. Method according to one of claims 13 to 15, characterized in that in simple extraction, the image is broken down into a succession of sub-images each containing an object which will be treated in order to determine its surface and its perimeter.

17. Method according to one of claims 7 to 16, characterized in that the image analysis steps (B) further include a separation step (B2) allowing to activate or not the separation of the attached objects then their reconstruction.

18. The method of claim 17, characterized in that if the separation of the attached objects is activated, an extraction of the related components is carried out (B3) starting with a simple extraction, followed by an ultimate erosion determining the number of component objects the sub-image, followed by a thickening to find the initial image with unjoined objects, finally followed by a Hough transform to reconstruct the missing part of the spheroid.

19. Method according to one of claims 7 to 18, characterized in that the processing of the results (C) comprises a step of calculations (C1) of the diameter, the perimeter and the volume followed by a display (C2).

20. Method according to one of claims 7 to 19, characterized in that it further comprises a save-print step D with return to the simple image analysis function in the event of serial acquisition.