EP4268120A1

EP4268120A1 - Method for characterising sperm cells

Info

Publication number: EP4268120A1
Application number: EP21839575.4A
Authority: EP
Inventors: Cédric ALLIER; Olivier CIONI; Ondrej MANDULA; Agnès CAMUS; Eric Schmitt
Original assignee: Commissariat a lEnergie Atomique CEA; Commissariat a lEnergie Atomique et aux Energies Alternatives CEA
Current assignee: Commissariat a lEnergie Atomique et aux Energies Alternatives CEA
Priority date: 2020-12-22
Filing date: 2021-12-20
Publication date: 2023-11-01
Also published as: US20240044771A1; WO2022136325A1; FR3118169A1

Abstract

Method for characterising a travelling particle in a sample (10), the method comprising: a) acquiring at least one image (I o,n) of the sample during an acquisition period, using an image sensor (20), and forming a series of images, the series of images comprising at least one image; b) using each image in the series of images resulting from a) as the input image (I in,n) of a convolutional neural network for detection (CNNd), the convolutional neural network for detection being configured to detect the particles and to produce, from each image, an output image (I out,n) in which each particle detected is assigned an intensity distribution which is centred on the particle and extends around the particle; c) for each particle detected, from each output image (I out, n) resulting from b), estimating a position (xi, yi) of each particle detected in each image in the series of images; d) characterising each particle detected from the estimate of the position resulting from c), based on each image in the series of images.

Description

Title: Method for characterizing spermatozoa

TECHNICAL AREA

The technical field of the invention is the observation of mobile or motile microscopic particles in a sample, with a view to characterizing them. A targeted application is the characterization of spermatozoa.

PRIOR ART

The observation of motile cellular particles, such as spermatozoa, in a sample is usually carried out using a microscope. The microscope comprises a lens defining an object plane, extending into the sample, as well as an image plane, coinciding with a detection plane of an image sensor. The microscope images the sperm cells in a focused configuration. The choice of such a modality supposes a compromise between the spatial resolution, the observed field and the depth of field. The higher the numerical aperture of the objective, the better the spatial resolution, at the expense of the size of the observed field. Similarly, a high numerical aperture decreases the depth of field.

It is understood that the observation of motile particles supposes an optimization of the objective, knowing that, in such a configuration, it is not possible to obtain at the same time a good spatial resolution, a wide observed field and a great depth. of field. However, these properties are particularly important when observing microscopic motile particles, in particular when the latter are numerous: the size of the particles requires good spatial resolution; their number justifies an extended observed field, so as to be able to maximize the number of particles observed on the same image; their movement requires a significant depth of field, so that the particles appear clearly on the image.

Given these requirements, it is usual to use a high magnification microscope. The small size of the observed field is compensated by the use of a translation stage. The latter allows image acquisition by moving the lens relative to the sample, parallel to the latter. The shallow depth of field is compensated by limiting the thickness of the sample: the latter is for example placed in a thin fluidic chamber, typically less than 20 μm, so as to limit the movement of the particles in a direction perpendicular to the object plane. Furthermore, the objective can be brought closer or further from the sample, so as to move the object plane in the sample, according to its thickness. This results in a complex and expensive device, requiring precise displacement of the lens.

The document WO2019/125583 describes general principles relating to the analysis of the motility or the morphology of spermatozoa, by mentioning the use of an artificial intelligence algorithm with supervised learning.

The publication Amann R. et al “Computer-assisted sperm analysis (CASA): Capabilities and potential developments”, describes the use of a conventional, focused imaging device, using a thin fluidic chamber.

An alternative to conventional microscopy, as described in the aforementioned publication, has been proposed by lensless imaging. It is known that lensless imaging, coupled with holographic reconstruction algorithms, allows cell observation while maintaining a high field of view, as well as a large depth of field. Patents US9588037 or US8842901 describe for example the use of lensless imaging for the observation of spermatozoa. Patents US10481076 or US10379027 also describe the use of lensless imaging, coupled with reconstruction algorithms, to characterize cells.

It is known that the use of digital reconstruction algorithms makes it possible to obtain clear images of particles. Such algorithms are for example described in US10564602, US20190101484 or US20200124586. In this type of algorithm, from a hologram acquired in a detection plane, an image of the sample is reconstructed in a reconstruction plane, remote from the detection plane. It is usual for the reconstruction plane to extend through the sample. However, this type of algorithm may require a relatively long calculation time.

Such a constraint is acceptable when the particles are considered to be immobile in the sample. However, when one wishes to characterize mobile particles, and in particular motile particles, the calculation time can become too long. Indeed, the characterization of mobile particles requires the acquisition of several images, at high frequencies, so as to be able to characterize the movement of the particles in the sample. The inventors propose an alternative to the previously cited patents, making it possible to characterize motile particles, and in particular spermatozoa, using a simple observation method. The method designed by the inventors allows the characterization of a large number of particles without requiring movement of an objective relative to the sample.

DISCLOSURE OF THE INVENTION

A first object of the invention is a method for characterizing at least one mobile particle in a sample, the method comprising: a) acquisition of at least one image of the sample during an acquisition period, at the using an image sensor and forming a series of images from the acquired images; b) using each image of the series of images resulting from a) as an input image of a detection convolutional neural network, the detection convolutional neural network being configured to detect the particles and to produce , from each image, an output image on which each detected particle is assigned an intensity distribution centered on the particle and extending around the particle; c) for each detected particle, from each output image resulting from b), estimation of a position of each detected particle in each image of the series of images; d) characterization of each particle detected from the estimation of the position resulting from c), established from each image of the series of images.

According to one possibility, steps a) to d) can be performed with a single image. In this case, the series of images comprises a single image.

According to one possibility, on the output image, each particle can be represented in the form of a point.

The particle (or each particle) can in particular be a spermatozoon.

Step d) can comprise a characterization, in particular morphological, of each spermatozoon detected. Step d) then comprises: for each of the spermatozoon detected, from each image resulting from a), and from the positions resulting from c), extraction of a thumbnail comprising the detected spermatozoon, the position of the spermatozoon detected in the thumbnail being predetermined, so as to obtain, for detected sperm, a series of thumbnails, the size of each thumbnail being smaller than the size of each image acquired during step a); - for each spermatozoon detected, use of the series of thumbnails as input data for a classification neural network, the classification neural network being configured to classify the spermatozoon among predetermined classes. These may in particular be morphological classes.

The method can be such that each detected sperm is centered with respect to each thumbnail.

Step d) may comprise a characterization of the motility of the spermatozoon. Step d) can then comprise, from the positions of the spermatozoon resulting from step c):

- a determination of a trajectory of the spermatozoon during the acquisition period;

- a calculation of a speed of the spermatozoon from the trajectory;

- a classification of the spermatozoon according to the speed.

The process may include:

- a determination of a distance, in a straight line, between a first point and a last point of the trajectory, and a calculation of a linear trajectory speed from said distance;

- And/or a determination of distances traveled between each acquired image, and a calculation of a speed of the curvilinear trajectory from said distances;

- And/or a determination of a smoothed trajectory, and a calculation of a speed of the average trajectory from the smoothed trajectory.

According to one embodiment,

- the sample extends along a plane of the sample;

- the image sensor extends along a detection plane;

- an optical system extends between the sample and the image sensor, the optical system defining an object plane and an image plane;

the object plane is offset from the sample plane by an object defocusing distance and/or the image plane is offset from the sample plane by an image defocusing distance.

In such an embodiment, the method can be such that: the sample is placed on a sample support, resting on at least one spring, the spring being configured to push the sample support towards the optical system;

- the optical system is connected to at least one abutment, extending, from the optical system, towards the sample holder; - so that during step a), under the effect of the spring, the sample support rests on the stop.

According to one embodiment, no imaging optics extend between the sample and the image sensor.

According to one embodiment, step a) comprises a normalization of each image acquired by an average of said image or by an average of images of the series of images. In this case, the series of images is made from each acquired image, after normalization.

According to one embodiment, step a) comprises an application of a high-pass filter to each acquired image. In this case, the series of images is performed from each image acquired, after application of the high-pass filter.

Advantageously, during step b), the intensity distribution assigned to each particle is decreasing, such that the intensity decreases as a function of the distance relative to the particle.

Advantageously, during step b), the intensity distribution assigned to each particle can be a two-dimensional parametric statistical distribution.

A second object of the invention is a device for observing a sample, the sample comprising mobile particles, the device comprising:

- a light source, configured to illuminate the sample;

- an image sensor, configured to form an image of the sample;

- a holding structure, configured to hold the sample between the light source and the image sensor;

- a processing unit, connected to the image sensor, and configured to implement steps a) to d) of a method according to the first object of the invention from at least one image acquired by the sensor of picture.

According to one embodiment, no imaging optics extend between the image sensor and the sample. The holding structure can be configured to maintain a fixed distance between the image sensor and the sample.

According to one embodiment,

- the image sensor extends along a detection plane; the device comprises an optical system, extending between the image sensor and the support plane, the optical system defining an image plane and an object plane, the device being such that: • the image plane is offset from the detection plane by an image defocusing distance;

• and/or the support plane and offset with respect to the object plane by an object defocusing distance .

The invention will be better understood on reading the description of the embodiments presented, in the remainder of the description, in connection with the figures listed below.

FIGURES

FIG. 1A represents a first embodiment of a device enabling the invention to be implemented.

Figure IB is a three-dimensional view of the device shown schematically in Figure IA.

Figure IC shows an assembly making it possible to maintain the sample at a fixed distance from an optical system.

Figure 1D shows the assembly shown in Figure IC in an observation position.

FIG. 2 represents a second embodiment of a device enabling the invention to be implemented.

Figure 3 shows the main steps of a method for characterizing mobile particles in the sample.

FIGS. 4A, 4B and 4C respectively represent an acquired image, a reference image and an image resulting from the detection neural network.

FIG. 5A is an example of an image of a sample obtained by implementing a device according to the first embodiment of the invention. Figure 5B shows the image depicted in Figure 5A after processing by a detection neural network. FIG. 5C shows an example of obtaining trajectories of particles detected on image 5B.

Figure 6 shows vignettes centered on a single spermatozoon, the vignettes being extracted from a series of nine images acquired using a defocused imaging modality.

FIGS. 7A, 7B and 7C respectively represent an acquired image, a reference image and an image resulting from the detection neural network.

FIGS. 8A, 8B and 8C respectively represent an acquired image, a reference image and an image resulting from the detection neural network. In this series of images, FIG. 8A has been processed by a high-pass filter.

FIGS. 9A and 9B represent performance of particle detection by the detection neural network, under different conditions. The performances are the sensitivity (FIG. 9A) and the specificity (FIG. 9B). Figures 10A to IOC represent confusion matrices, expressing the classification performance of spermatozoa, using images acquired according to the defocused modality, the classification being carried out respectively: by implementing a holographic reconstruction algorithm from an image obtained by holographic reconstruction and processed by a neural network; by implementing a classification neural network whose input layer comprises a single image acquired according to a defocused configuration, without holographic reconstruction; by implementing a classification neural network, the input layer of which comprises a series of images of a particle, each image being acquired according to a defocused configuration, without holographic reconstruction.

DESCRIPTION OF PARTICULAR EMBODIMENTS

There is shown in Figure IA, a first embodiment of a device 1 for implementing the invention. According to this first embodiment, the device allows the observation of a sample 10 interposed between a light source 11 and an image sensor 20. The light source 11 is configured to emit an incident light wave 12 propagating up to 'to the sample parallel to a propagation axis Z.

The device includes a sample holder 10s configured to receive the sample 10, such that the sample is held on the holder 10s. The sample thus maintained extends along a plane, called the sample plane Pw. The plane of the sample corresponds for example to a mean plane around which the sample 10 extends. The sample support can be a glass slide, for example 1 mm thick.

The sample notably comprises a liquid medium 10 _m in which mobile and possibly motile particles 10i are immersed. The 10 _m medium can be a biological liquid or a buffer liquid. It may for example comprise a bodily fluid, in the pure or diluted state. By bodily fluid is meant a fluid generated by a living body. It may in particular be, without limitation, blood, urine, cerebrospinal fluid, semen, lymph.

The sample 10 is preferably contained in a fluidic chamber 10 _c . The fluidic chamber is for example a fluidic chamber with a thickness of between 20 μm and 100 μm. The thickness of the fluidic chamber, and therefore of the sample 10, along the axis of propagation Z, typically varies between 10 μm and 200 μm, and is preferably between 20 μm and 50 μm.

One of the objectives of the invention is the characterization of moving particles in the sample. In the embodiment described, the mobile particles are spermatozoa. In this case, the sample contains sperm, possibly diluted. In this case, the fluidic chamber 10 _c can be a counting chamber dedicated to the analysis of the mobility or the concentration of cells. It may for example be a counting chamber marketed by Leja, with a thickness of between 20 μm and 100 μm.

According to other applications, the sample comprises mobile particles, for example microorganisms, for example microalgae or plankton, or cells, for example cells in the process of sedimentation.

The distance D between the light source 11 and the sample 10 is preferably greater than 1 cm. It is preferably between 2 and 30 cm. Advantageously, the light source 11, seen by the sample, is considered as point. This means that its diameter (or its diagonal) is preferably less than one tenth, better still one hundredth of the distance between the sample and the light source.

The light source 11 is for example a light-emitting diode. It is preferably associated with a diaphragm 14, or spatial filter. The aperture of the diaphragm is typically between 5 μm and 1 mm, preferably between 50 μm and 1 mm. In this example, the diaphragm has a diameter of 400 µm. According to another configuration, the diaphragm can be replaced by an optical fiber, a first end of which is placed facing the light source and a second end of which is placed facing the sample 10. The device can also comprise a diffuser 13 , placed between the light source 13 and the diaphragm 14. The use of a diffuser/diaphragm assembly is for example described in US10418399

The image sensor 20 is configured to form an image of the sample according to a detection plane P20. In the example represented, the image sensor 20 comprises a matrix of pixels, of CCD or CMOS type. The detection plane P20 preferably extends perpendicular to the axis of propagation Z. Preferably, the image sensor has a high sensitive surface, typically greater than 10 mm ² . In this example, the image sensor is an IDS-UI-3160CP-M-GL sensor comprising pixels of 4.8×4.8 μm ² , the sensitive surface being 9.2 mm×5.76 mm, ie 53 mm ² . In the example shown in Figure 1A, the image sensor 20 is optically coupled to the sample 10 by an optical system 15. In the example shown, the optical system includes an objective 15i and a tube lens 152. The latter is intended to project a formed image on the sensitive surface of the image sensor 20 (53 mm ² surface).

In this example: the 15i objective is a Motic CCIS EF-N Plan Achromat lOx objective, with a numerical aperture of 0.25; lens 152 is a Thorlabs LBF254-075-A lens - 75 mm focal length.

Such an assembly provides an observation field of 3 mm ² , with a spatial resolution of 1 μm. The short focal length of the lens makes it possible to optimize the bulk of the device 1 and to adjust the magnification to the size of the image sensor.

The image sensor is configured to acquire images of the sample, according to an acquisition frequency of a few tens of images per second, for example 60 images per second. The sampling frequency is typically between 5 and 100 frames per second.

The optical system 15 defines an object plane P _o and an image plane Pi. In the embodiment represented in FIG. 1A, the image sensor 20 is configured to acquire an image according to a defocused configuration. The image plane Pi coincides with the detection plane P ₂ o, while the object plane P _o is offset by an object focusing distance 6 of between 10 μm and 500 μm, relative to the sample. The focusing distance is preferably between 50 μm and 100 μm, for example 70 μm. The object plane P _o extends outside the sample 10. According to another possibility, the object plane extends into the sample, while the image plane is shifted with respect to the detection plane, by a distance image defocus. The image focusing distance is preferably between 50 μm and 100 μm, for example 70 μm. According to another possibility, the object plane P _o and the image plane Pi are both offset respectively with respect to the plane of the sample and with respect to the detection plane. Whatever the configuration chosen, the defocusing distance is preferably greater than 10 μm and less than 1 mm, or even 500 μm, and preferably between 50 μm and 150 μm. The observation of a cellular sample according to a defocused configuration has been described in patent US10545329.

According to such a modality, the image sensor 20 is exposed to a light wave, called exposure light wave. The image acquired by the image sensor comprises figures interference, which can also be designated by the term "diffraction patterns", formed by: a part of the light wave 12 emitted by the light source 11, and having passed through the sample without interacting with the latter; diffraction waves, formed by the diffraction of part of the light wave 12 emitted by the light source in the sample. These include the diffraction formed by the particles.

A processing unit 30, comprising for example a microprocessor, is capable of processing each image acquired by the image sensor 20. In particular, the processing unit comprises a programmable memory 31 in which is stored a sequence of instructions for perform the image processing and calculation operations described in this description. The processing unit 30 can be coupled to a screen 32 allowing the display of images acquired by the image sensor 20 or resulting from the processing carried out by the processing unit 30.

The image acquired by the image sensor 20, according to a defocused imaging modality, is a diffraction figure of the sample, sometimes called a hologram. It does not make it possible to obtain an accurate representation of the observed sample. In the usual way in the field of holography, it is possible to apply, to each image acquired by the image sensor, a holographic reconstruction operator so as to calculate a complex expression representative of the light wave to which the sensor is exposed. image, and this at any point of coordinates (x, y, z) in space, and in particular in a reconstruction plane corresponding to the plane of the sample. The complex expression returns the intensity or phase of the exposure light wave. Such a holographic reconstruction is described in connection with the prior art, as well as in US10545329.

However, for reasons of processing speed, the inventors followed a different approach from that suggested by the prior art, without recourse to a holographic reconstruction algorithm applied to the images acquired by the image sensor. The process implemented by the sensor is described below, in connection with Figure 3.

Figure IB is a representation of an example of a device as shown schematically in Figure IA. Figures 1C and 1D represent a detail of the arrangement of the sample 10 facing the optical system 15, and more precisely facing the lens 15i. The sample support 10 _s is connected to elastic return means 16 _a , for example springs. The springs tend to Bring the _10s sample holder closer to the 15i objective. The device also includes abutments 16b, integral with the lens 15i. The objective 15i and the stops 16b are fixed relative to the image sensor 20. The stops 16b and the return means 16a form _a holding structure 16, intended to hold the sample between the light source 11 and the image sensor 20.

Figures 1C and 1D represent the maintenance of the sample on the sample support respectively during the positioning of the sample and during its observation. A rigid link 16c connects the lens _15i to the stops 16b. During the positioning of the sample (FIG. IC), the springs _16a are compressed. During the observation of the sample (figure ID), under the effect of a relaxation of the springs 16a _, the support of the sample _10s comes to bear against the stops 16b. This makes it possible to control the distance A between the objective 15i and the sample 10, independently of the thickness of the sample support 10 _s . This makes it possible to overcome variations in the thickness of the sample support. For example, when the latter is a glass slide with a thickness of 1 mm, the thickness can fluctuate over a relatively wide range, for example ±100 μm. The assembly described in connection with FIGS. 1C and 1D makes it possible to control the distance A between the objective 15i and the sample 10, to within ± 5 μm, independently of such fluctuations in the thickness of the plate 10 _s . The inventors believe that such an assembly makes it possible to avoid having to resort to an autofocus system to produce images

Figures 1C and 1D also show a heating resistor 19 connected to a temperature controller 18. The function of these elements is to maintain a temperature of the fluidic chamber 10 _c at 37°C.

FIG. 2 represents a second embodiment of device 1' suitable for implementing the invention. The device 1' comprises a light source 11, a diffuser 13, a diaphragm 14, an image sensor 20, a holding structure 17 and a processing unit 30 as described in connection with the first embodiment. The holding structure 17 is configured to define a fixed distance between the sample and the image sensor. According to this embodiment, the device does not include an image forming lens between the image sensor 20 and the sample 10. The image sensor 20 is preferably brought closer to the sample, the distance between the image sensor 20 and sample 10 being typically between 100 μm and 3 mm. According to this embodiment, the image sensor acquires images according to a lensless imaging modality. The sample is preferably contained in a fluidic chamber 10 _c , for example a “Leja” chamber as described in connection with the first embodiment. The advantage of such an embodiment is that it does not require precise positioning of an optical system 15 with respect to the sample 10, and that it confers a high field of observation. The disadvantage is obtaining images of lower quality, but which remain usable.

Preferably, the holding structure is arranged so that the distance between the sample, when the sample 10 is placed on the holding structure 17, and the image sensor 20, is constant. In the example represented in FIG. 2, the holding structure 17 is connected to a base 20', on which the image sensor 20 extends. - Printed circuit), on which the image sensor 20 is placed. The sample 10, contained in the fluidic chamber 10 _c , is held, by the sample holder 10 _s , on the holding structure 17. The support sample is for example a transparent slide. The sample extends between the _10s sample holder and the image sensor. Thus, the distance between the sample 10 and the image sensor 20 is not affected by a fluctuation in the thickness of the support 10s.

FIG. 3 schematizes the main steps of a method for processing several images acquired by an image sensor according to the defocused imaging modality (first embodiment) or imaging without lens (second embodiment). The method is described in connection with the observation of spermatozoa, it being understood that it can be applied to the observation of other types of motile particles.

Step 100: Acquisition of a series of images I _{O n}

During this step, a series of images I _{O n} are acquired according to one of the methods previously described, n is a natural number designating the rank of each image acquired, with 1<n<N, N being the total number images acquired. The images acquired are images acquired either according to a defocused imaging modality or according to a lensless imaging modality. The number N of images acquired can be between 5 and 50. As previously indicated, the images can be acquired according to an acquisition frequency of 60 Hz.

In the results presented below, the images were acquired using a defocused imaging modality, as described in connection with Figures 1A to 1D.

Step 110: Preprocessing

The objective is to perform a pre-processing of each image acquired, so as to limit the effects of a fluctuation in the intensity of the incident light wave or in the sensitivity of the camera. The pre-processing consists in normalizing each image acquired by an average of the intensity of at least one image acquired, and preferably of all the images acquired.

Thus, from each image I _{O n} , the normalization makes it possible to obtain a normalized image I _n , such that I _n = x 100 where I _{O n} is an average of one or more images of the series of images, and preferably each image in the series of images.

According to one possibility, the pre-processing can include an application of a high-pass filter to each image, possibly normalized. The high pass filter eliminates low frequencies from the image.

According to one possibility, a Gaussian filter is applied, by carrying out a convolution product of the image I _n by a Gaussian kernel K. The width at mid-height of the Gaussian kernel is for example 20 pixels. The preprocessing then consists in subtracting, from the image I _n , the image I _n * K resulting from the application of the Gaussian filter, so that the image resulting from the preprocessing is: I' _n ⁼ In ~ In * . * is the convolution product operator.

The effect of such filtering is described below, in connection with FIGS. 7A to 7C and 8A to 8C.

Step 120: Particle detection

This step consists in detecting and precisely positioning the spermatozoa, and this on each image I _{O n} , resulting from step 100 or from each image I′ _n preprocessed during step 110. This step is carried out using of a CNNd detection convolutional neural network. The CNNd neural network comprises an input layer, in the form of an input image I _{in n} . The input image I _{in n} is either an acquired image I _{O n} , or a preprocessed image I _n , I′ _n . From the input image I _{in n} , the detection neural network CNNd generates an output image I _or t, _n - The output image I _outin is such that each spermatozoon 10i detected on the image d input I _{in n} , in a position appears as an intensity distribution centered around said position. That is, from an input image the CNNd neural network allows: detection of spermatozoa; a position estimate of each 10i sperm detected; a generation of an output image/ _{out n} comprising a predetermined intensity distribution around each position.

Thus, I _or t _n — CN NdUin.n) Each position (x^y^ is a two-dimensional position, in the detection plane P20. The distribution is such that the intensity is maximum at each position (%j,yi) and that the intensity is considered negligible beyond a neighborhood Vt of each position. By neighborhood is meant a region extending according to a predetermined number of pixels, for example between 5 and 20 pixels, around each position (x^y^). The distribution D _t can be of the niche type, in which case in the neighborhood each pixel is of constant intensity is high, and beyond the neighborhood each pixel has zero intensity.

Preferably, the distribution D _t is centered on each position (x^y^) and is strictly decreasing around the latter. It may for example be a two-dimensional Gaussian intensity distribution, centered on each position (Xj,yj). The width at mid-height is for example less than 20 pixels, and preferably less than 10 or 5 pixels. Any other form of parametric distribution can be considered, knowing that it is preferable that the distribution be symmetrical around each position, and preferably strictly decreasing from position (xj,yj). Assigning a distribution at each position (xj,yj) makes it possible to obtain an output image / _{out n} in which each spermatozoon is simple to detect. In this, the output image I _{out n} is a detection image, based on which each sperm can be detected.

The output image of the neural network is formed by a resultant of each intensity distribution D _t defined respectively around each position (x^yj).

In the example implemented by the inventors, the convolutional detection neural network comprised 20 layers comprising either 10 or 32 characteristics (more usually designated by the term “features”) per layer. The number of features was determined empirically. The transition from one layer to another is performed by applying a convolution kernel of size 3x3. The output image is obtained by combining the characteristics of the last convolution layer. The neural network was programmed in a Matlab environment (publisher: The Mathworks). The neural network has previously been the subject of learning (step 80), so as to parameterize the convolution filters. During the learning, learning sets were used, each set comprising: an input image, obtained by the image sensor and preprocessed by normalization and possibly filtering, using animal semen. an output image, on which the position of each sperm was manually annotated. Learning was performed using 10000 or 20000 annotated positions (i.e. between 1000 and 3000 annotated positions per image). The effect of the size of the training set (10,000 or 20,000 annotations) is discussed in connection with Figures 9A and 9B.

FIGS. 4A, 4B and 4C are a set of test images, comprising respectively: an image of a sample of bovine semen acquired by the image sensor and having undergone normalization and high pass filtering according to step 110; a manually annotated image, which corresponds to a reference image, usually designated by the term “ground truth” (reality); an image resulting from the application of the detection convolutional neural network.

The image of FIG. 4C is consistent with the reference image (FIG. 4B), which attests to the detection performance of the CNNd neural network. The inventors have noted that FIG. 4C shows a position not shown on the reference image: a check showed that it was an annotation omission on the reference image.

Thus, on the basis of a hologram, and subject to possible pre-processing of the normalization or high-pass filter type, to bring out the high frequencies, the application of the convolutional neural network of detection CNNd allows precise detection and positioning of sperm. And this without resorting to an image reconstruction implementing a holographic propagation operator, as suggested in the prior art. The input image of the neural network is an image formed in the detection plane, and not in a reconstruction plane remote from the detection plane. In addition, the detection neural network generates low-noise output images: the signal-to-noise ratio associated with each sperm detection is high, which facilitates subsequent operations.

Step 120 is repeated for different images of the same series of images, so as to obtain a series of output images I _tool I _O ut,N-

During step 120, from each output image I _{out l} I _0U t,N> a local maximum detection algorithm ^is applied, so as to obtain, for each image, a list of 2D coordinates, each coordinate corresponding to a position of a spermatozoon. Thus, from each output image I _tool I _O ut,N> a list L _{out l} L _{0Ut w} ^is established -Each list L _{out n} comprises corresponds to the 2D positions, in the detection plane, of spermatozoa detected in a picture I _outin . FIG. 5A represents an image acquired during step 100. More precisely, it is a detail of an image acquired according to a defocused mode. This is an image of a sample containing bovine semen. Such an image is difficult to interpret by a user. Figure 5B shows an image resulting from the application of the detection neural network in Figure 5A, after normalization and application of a high-pass filter. The comparison between images 5A and 5B shows the gain provided by the detection convolutional neural network in terms of signal-to-noise ratio.

Step 130: position tracking

During this step, from the lists L _{out l} L _outN respectively established from the output images I _tool I _O ut,N> resulting from the same series of images I _{0 1} ... I _{0 N} , a position tracking algorithm is applied, usually designated by the term “tracking algorithm”. For each spermatozoon detected following step 120, an algorithm is implemented allowing tracking of the position of the spermatozoon, parallel to the detection plane, between the different images of the series of images. The implementation of the position tracking algorithm is efficient because it is performed from the lists L _{out l} L _{0Ut N} resulting from step 120. As previously indicated, these images have a signal to noise ratio high, which facilitates the implementation of the position tracking algorithm. The position tracking algorithm may be a “nearest neighbor” type algorithm. Step 130 allows a trajectory of each sperm to be determined parallel to the detection plane.

In FIG. 5C, output images I _tool I _O ut,N have been superimposed ^by considering an image stack of 30 images. It is observed that the trajectory, parallel to the detection plane, of each spermatozoon can be determined precisely.

Step 140: characterization of the motility

During step 140, for each spermatozoon, each trajectory can be characterized on the basis of metrics applied to the trajectories resulting from step 130. Knowing the frequency of acquisition of the images, it is possible to quantify displacement speeds of each spermatozoon detected, and in particular: a speed of the linear trajectory VSL , usually designated by the term "velocity straightline path", which corresponds to the speed calculated on the basis of a distance, in a straight line, between the first and last points of the trajectory (the first point is determined from the first acquired image of the series of images, and the last point is determined from the last acquired image of the series of images) a speed of the curvilinear path VCL, usually referred to by the term “velocity curvilinear path”: this is a speed established by summing the distances traveled between each image, and by dividing by the duration of the acquisition period; a speed of the average trajectory VAP, usually designated by the term “velocity average path”: this is a speed established after smoothing the trajectory of a particle: the distance traveled according to the smoothed trajectory (or average trajectory) is divided by the length of the vesting period.

From the calculated velocities, it is possible to define indicators making it possible to characterize the motility of the spermatozoa, known to those skilled in the art. These are, for example, indicators of the type: straightness indicator STR, obtained by a ratio between VSL and VAP, usually designated by the term “straightness”. This indicator is all the higher as the sperm moves in a straight line; LIN linearity indicator, obtained by a ratio between VSL and VCL, usually designated by the term “linearity”. This indicator is also all the higher as the sperm moves in a straight line. WOB oscillation indicator, obtained by a VAP/VCL ratio, usually designated by the term wobble.

The quantification of the speeds or parameters listed above makes it possible to categorize the spermatozoa according to their motility. For example, a spermatozoon is considered to be: motile if the length of the trajectory is greater than a first threshold, for example 10 pixels, and its displacement along the average trajectory (VAPx At, At being the acquisition period) is greater than a predefined length, corresponding for example to the length of a sperm head; progressive if the length of the trajectory is greater than the first threshold, and if the straightness STR and the speed of the average trajectory VAP are respectively greater than two threshold values STRth and VAPthi; slow if the length of the trajectory is greater than the first threshold and if the speed of the linear trajectory VSL and the speed of the average trajectory VAP are respectively less than two threshold values VSL _t h2 and VAP _t h2; static if the length of the trajectory is greater than the first threshold and if the speed of the linear trajectory VSL and the speed of the average trajectory VAP are respectively lower than two threshold values VSL _t h3 and VAP _t h3- not categorized if the length of the trajectory is below the first threshold.

The threshold values STR _t h, VSL _t h2, VSL _t h3 are determined beforehand, with VSL _t h2^ VSL _t h3- The same applies to the values VAP _t hi, VAP _t h2 and VAP _t h3 with VAP _t hi> VAP _t hi> VAP _t h3-

Step 150: Extract thumbnails for each sperm.

During this image, for each spermatozoon detected by the detection network CNNd, a thumbnail V _in is extracted, and this in each image I _{O n} acquired by the image sensor during step 100.

Each thumbnail V _in is a portion of an image I _{O n} acquired by the image sensor. For each detected spermatozoon 10i, from each image I _{O n} , a vignette V _in is extracted around the position assigned to the sperm. The position of the spermatozoon 10d, in each image I _{O n} , is obtained following the implementation of the position tracking algorithm (step 130).

The size of each thumbnail V _in is predetermined. Relative to each thumbnail V _in , the position (%j, yj) of the spermatozoon 10j considered is fixed: preferably, the position (x^ y^) of the spermatozoon 10i is centered in each thumbnail V _in .

A thumbnail V _in may for example comprise a few tens or even a few hundreds of pixels, typically between 50 and 500 pixels. In this example, a thumbnail is 64 x 64 pixels. Due to the size and concentration of spermatozoa in the sample, a V _in thumbnail may include several spermatozoa to be characterised. However, only the spermatozoon 10j occupying a predetermined position in the thumbnail V _in , for example at the center of the thumbnail, is characterized using said thumbnail.

Figure 6 shows nine V _in thumbnails extracted from I _outin images resulting from step 100, using the positions (%j, yj) resulting from trajectory tracking of a spermatozoon resulting from step 130.

Step 160 Classification of the morphology of each spermatozoon

During this step, a CNN _c classification neural network is used, so as to classify each 10i spermatozoon previously detected by the convolutional neural network detection CNNd. For each 10i sperm, the neural network is fed with thumbnails ... V _{i N} extracted during step 150.

The convolutional neural network of classification CNN _c can for example comprise 6 convolutional layers, comprising between 16 and 64 characteristics: 16 characteristics for the first four layers, 32 characteristics for the fifth layer, and 64 characteristics for the sixth layer. The passage from one layer to another is carried out by applying a convolution kernel of size 3 by 3 The output layer comprises nodes, each node corresponding to a probability of belonging to a morphological class. Each morphological class corresponds to a morphology of the analyzed sperm. It can for example be a known classification, the classes being:

0: undefined

1: standard

2: stump

3: distal droplet

4: decapitated

5: microcephalus

6: proximal droplet

7: DMR (Distal Midpiece reflex, usually referred to as Curvature of the distal end of the intermediate piece)

8: head abnormality

9: flagellum abnormality: curved or coiled flagellum

10: aggregates.

The output layer can thus have 11 classes.

The neural network is programmed in a Matlab environment (publisher: The Mathworks). The neural network has previously been the subject of learning (step 90), so as to parameterize the convolution filters. During the learning, learning games were used, each game comprising: thumbnails, extracted from images acquired by the image sensor; a manual annotation of the morphology of each sperm analyzed. The annotation was carried out on the basis of thumbnails having been the subject of a holographic reconstruction.

Experimental trials Different tests were performed to examine the performance of the CNNd detection neural network. The sensitivity of the detection has been tested with respect to the application of a high-pass filter during step 110. FIGS. 7A, 7B and 7C represent respectively an image acquired by the image sensor, an image reference, manually annotated, and an image resulting from the detection neural network. Image 7C was obtained without implementing the filter during step 110.

FIGS. 8A, 8B and 8C respectively represent an image acquired by the image sensor, a reference image, annotated manually, and an image resulting from the detection neural network. Image 8C was obtained by implementing a high pass filter during step 110.

It is observed that image 8C includes detected spermatozoa, marked by arrows, which do not appear on image 7C. Applying the filter improves the detection performance of the classification neural network.

FIGS. 9A and 9B respectively represent the detection sensitivity and the detection specificity of the CNNd detection neural network (curves a, b and c), in comparison with a conventional algorithm (curve d). The classical algorithm was based on morphological operations of erosion/dilation type image processing.

In each of FIGS. 9A and 9B, curves a, b and c correspond respectively: to the neural network trained with 10,000 annotations; the trained neural network with 20,000 annotations; to the neural network trained with 20,000 annotations, each image acquired by the sensor having been subject to high-pass filtering.

Sensitivity and specificity (ordinate axes) were determined on 14 different samples (abscissa axis) of diluted bovine semen (factor 10). It is observed that the performances of the neural network, whatever the configuration (curves a, b and c) are superior to that of the classical algorithm, which is remarkable. Furthermore, the best performance is obtained in configuration c, according to which the neural network is trained with 20,000 annotations and is fed with an image that has been preprocessed with a high-pass filter. It is recalled that the sensitivity corresponds to the rate of true positives over the sum of the rates of true positives and false negatives, and that the specificity corresponds to the rate of true positives over the sum of the rates of true and false positives.

Figure 10A is a confusion matrix representing the classification performance of a CNN _c classification neural network parameterized to receive, as an input image, an image resulting from a holographic reconstruction. The neural network is a convolutional neural network, as previously described, trained using images acquired using a defocused device, as previously described. Using this neural network, each acquired image undergoes a holographic reconstruction, in a sample plane extending through the sample, using a holographic reconstruction algorithm as described in US20190101484. From the reconstructed image, a thumbnail centered on the sperm to be characterized is extracted. The thumbnail forms an input image of the neural network.

Figure 10B is a confusion matrix representing the classification performance of a classification neural network parameterized to receive, as an input image, a single thumbnail, centered on the spermatozoon to be characterized, and extracted from a image acquired in a defocused configuration. The neural network is a convolutional neural network, structured as described in step 160, except that the neural network is powered by a single tile.

Figure 10C is a confusion matrix representing the classification performance of a classification neural network as described in step 160, fed by a series of 5 vignettes extracted from images acquired in defocused configuration.

The axes of each confusion matrix correspond to classes 1 to 10 previously described.

It is noted that the classification performances of the algorithms described in connection with FIGS. 10B and 10C are superior to that described in FIG. 10A. For example, with regard to class 6 (“proximal droplet” anomaly), the classification performance goes from 31% (FIG. 10A), to 61% (FIG. 10B), and to 96.9% (FIG. 10C).

It is also noted that a single input image makes it possible to obtain a satisfactory classification performance. Also, for morphological classification, it is not necessary to have a series of images comprising several images. A single image may suffice. However, the classification performance is better when using multiple images.

The invention allows an analysis of a sample comprising spermatozoa without resorting to a translation stage. Furthermore, it allows characterization directly from the image acquired by the image sensor, that is to say on the basis of diffraction figures of the different spermatozoa, and this without requiring the use of algorithms of digital reconstruction. Currently, the use of such an algorithm results in a processing time of 10 seconds per image. That is 300 seconds for a series of 30 images. Furthermore, as represented in FIGS. 10A to 10C, the invention allows a more precise classification than a classification based on a neural network of identical structure, fed by reconstructed images.

Another advantage of the invention is the tolerance of defocusing. The inventors estimate that the invention tolerates shifts of ±25 μm between the optical system and the sample.

Claims

23

CLAIMS. Method for characterizing at least one mobile particle (10j) in a sample (10), the method comprising: a) acquisition of at least one image (I _{O n} ) of the sample during an acquisition period, at using an image sensor (20) and forming a series of images, the series of images comprising at least one image, each image of the sample being acquired according to a defocused imaging modality or according to a lensless imaging modality, so that each particle forms, on each image, a diffraction figure; b) using each image of the series of images resulting from a) as the input image (n,n) of a detection convolutional neural network (CNNd), the detection convolutional neural network being configured to detect the particles and to produce, from each image, an output image (I _or t, _n ) ^on which each detected particle is assigned an intensity distribution, centered on the particle and extending around the particle; c) for each detected particle, from each output image (/ _{out n} ) resulting from b), estimation of a position of each particle detected in each image of the series of images; d) characterization of each particle detected from the estimation of the position resulting from c), established from each image of the series of images. . A method according to claim 1, wherein the particle is a spermatozoon. . Method according to claim 2, in which step d) comprises a morphological characterization of each spermatozoon detected, step d) comprising: for each of the spermatozoon detected, from each image resulting from a), and positions resulting from c), extraction of a thumbnail ( j _n ) comprising the detected spermatozoon, the position of the detected spermatozoon in the thumbnail being predetermined, so as to obtain, for detected spermatozoon, a series of thumbnails, the size of each thumbnail being less than the size of each image acquired during step a);

- for each sperm detected, use of the series of thumbnails as input data for a classification neural network (CNN _c ), the neural network classification being configured to classify the spermatozoon among predetermined morphological classes.

4. Method according to claim 3, in which each detected spermatozoon is centered with respect to each thumbnail.

5. Method according to claim 3 or claim 4, in which step d) comprises a characterization of the motility of the spermatozoon, step d) comprising, from the positions of the spermatozoon resulting from step c):

- a calculation of a speed of the spermatozoon from the trajectory;

- a classification of the spermatozoon according to the speed.

6. Method according to claim 5, comprising:

7. Method according to any one of the preceding claims, in which:

- the sample extends along a plane of the sample (Pio);

- the image sensor extends along a detection plane (P20);

- an optical system (15) extends between the sample and the image sensor, the optical system defining an object plane (P _o ) and an image plane (Pi);

- the object plane is offset from the sample plane by an object defocus distance and/or the image plane is offset from the sample plane by an image defocus distance, so that when step a), each image of the sample is acquired according to a defocused imaging modality.

8. Method according to claim 7, in which: the sample is placed on a sample holder (10s), resting on at least one spring (16a), the spring being configured to push the sample holder towards the system optical; - the optical system is connected to at least one abutment (16b), extending, from the optical system, towards the sample holder;

- so that during step a), under the effect of the spring, the sample support rests on the stop.

9. Method according to any one of claims 1 to 6, in which no imaging optics extend between the sample and the image sensor, so that during step a), each Sample image is acquired using a lensless imaging modality.

10. Method according to any one of the preceding claims, in which step a) comprises a normalization of each image acquired by an average of said image or by an average of images of the series of images.

11. Method according to any one of the preceding claims, in which step a) comprises an application of a high-pass filter to each acquired image.

12. Method according to any one of the preceding claims, in which during step b), the intensity distribution assigned to each particle is decreasing, such that the intensity decreases as a function of the distance with respect to the particle.

13. Method according to any one of the preceding claims, in which during step b), the intensity distribution assigned to each particle is a two-dimensional parametric statistical distribution.

14. Device (1, the) for observing a sample, the sample comprising mobile particles, the device comprising:

- a light source (11), configured to illuminate the sample;

- an image sensor (20), configured to form an image of the sample;

- a holding structure (16, 17), configured to hold the sample between the light source and the image sensor;

- a processing unit (30), connected to the image sensor, and configured to implement steps a) to d) of a method according to any one of claims 1 to 13 from at least one image acquired by the image sensor. 26

15. Device according to claim 14, in which no imaging optics extend between the image sensor and the sample.

16. Device according to claim 15, in which the holding structure (17) is configured to maintain a fixed distance between the image sensor and the sample.

17. Device according to claim 14, in which:

- the image sensor extends along a detection plane; the device comprises an optical system (15), extending between the image sensor and the support plane, the optical system defining an image plane and an object plane, the device being such that:

• the image plane is offset from the detection plane by an image defocusing distance;

• and/or the support plane and offset with respect to the object plane by an object defocus distance.

18. Device according to claim 17, in which:

- the holding structure (16) comprises a sample support (10s), mounted on a spring ( _16a );

- the optical system (15) is mechanically connected to at least one stop (16b);

- the spring is arranged so that the sample support rests against the abutment, so as to control a distance between the optical system and the sample.