FR3050833A1 - PLENOPTIC CAMERA WITH CORRECTION OF GEOMETRIC ABERRATIONS - Google Patents

Abstract

The invention relates to a plenoptic camera (600) comprising: - an input optical system (610); a matrix optical sensor (630); and a matrix (620) of optical elements (621) disposed between the input optical system and the matrix optical sensor, each optical element (621) being associated with a plurality of pixels of the optical sensor, and the optical centers. optical elements (621) being distributed according to a distribution grid, at the points of intersection thereof. According to the invention, the distribution grid (650) comprises a plurality of curved lines, and an object grid (640), the distribution grid (650) of which is the image by the input optical system (610), consists of two sets of intersecting straight lines. The invention makes it possible at least partially to correct the effect of a distortion aberration of the input optical system.

Description

PLENOPTICAL CAMERA WITH CORRECTION OF GEOMETRIC ABERRATIONS

DESCRIPTION

TECHNICAL FIELD The invention relates to the field of plenoptic cameras.

STATE OF THE PRIOR ART

Plenoptic cameras are imaging systems.

Advantageously, but not limitatively, a plenoptic camera can be used as a three-dimensional vision system.

The patent application FR-1558338 describes a particular example of such a camera. A plenoptic camera is configured to acquire a two-dimensional image in which several pixels refer to a same point in the object space.

One can thus find, by triangulation, the coordinates in three dimensions of this point.

Figures IA to IC schematically illustrate the principles implemented in a plenotic camera 100.

The plenoptic camera 100 comprises an optical input system 110 configured to receive light rays from an object 200 to be imaged.

The input optical system 110 may consist of a single lens.

Alternatively, the input optical system 110 may consist of a set of lenses located one behind the other, along an axis of propagation of light from the object 200, to the input optical system.

The optical input system 110 carries out the optical conjugation between an object plane π, and an image plane π 'receiving a matrix 120 of optical elements. Advantageously, the plane π is located at a finite distance from the camera.

The optical elements 121 are for example microlenses, or pinholes, distributed coplanarly in the image plane π '.

Microlenses 121 may each have a non-circular section, for example to reduce a distance between neighboring microlenses. It can be plane-convex, biconvex, aspherical microlenses, etc.

Each of the optical elements 121 receives light rays coming from the object 200, and having passed through the input optic 110.

After passing through an optical element 121, the light rays propagate to a matrix optical sensor 130.

The matrix optical sensor 130 is a photosensitive sensor, for example of the CCD sensor type, configured to convert an incident photon flux into an electrical signal, to form an image.

The detection surface of the matrix optical sensor 130 consists of a plurality of pixels, preferably arranged in rows and columns.

It is located near the matrix 120, for example between 0.4 and 0.6 mm behind the rear faces of the optical elements 121 (distance measured along the optical axis of the input optical system).

The matrix 120 of optical elements is configured to distribute, on the pixels of the matrix optical sensor 130, light rays coming from the object to be imaged and having passed through the optical input system. Each optical element 121 corresponds to a selection of pixels of the matrix optical sensor. A light ray coming from the object 200 is directed towards one of these pixels, as a function of its angle of incidence on the optical element 121.

Said selection of pixels, associated with the same optical element, forms a macropixel 131.

Preferably, there is no overlap between the different macro-pixels. This overlap is avoided, in particular by an adequate choice of the focal lengths of the input optical system 110 and the optical elements 121.

In a perfect plenoptic camera, that is to say without optical aberration, light rays coming from the same point on the object 200, propagate through the optical input system 110, and the matrix of elements. 120, up to different pixels of the matrix optical sensor 130.

In this case, the position of this point in the object space can be determined by triangulation.

Throughout the text, the object space designates the field of view of the plenoptic camera, located upstream of the optical input system according to the direction of propagation of the light, from the object to the input optical system.

In particular, we identify the pixels associated with the same point, and we deduce the three-dimensional coordinates of this point.

This deduction can be done by calculation. For each pixel thus identified, an image ray, propagating in a straight line between this pixel and the optical center of the optical element 121 associated with this pixel, is calculated. An optical conjugate ray of this image ray is then calculated by the input optical system 110. The point of the object is at the intersection of the at least two object rays thus calculated.

In addition or alternatively, a prior calibration of the plenoptic camera, by moving a light point in the object space, and by associating for each of a plurality of points of the object space, the corresponding pixels on the optical sensor matrix. We can thus take into account imperfections of the real system that forms the plenoptic camera. Then, we can associate a set of pixels corresponding to the same point, and the position of this point in the object space.

When the optical elements 121 are pinhole, the optical center designates the geometric center of the hole forming a pinhole.

When the optical elements 121 are microlenses, the optical center is located on the optical axis of a microlens, and corresponds to the point of the microlens, such that an incident light ray at this point is not deflected by this microlens .

FIG. 1A illustrates ray traces from a point 200i of the object 200, located in the object plane π. The image of this point lies in the image plane π ', on the optical center of an optical element 121. As a function of their angle of incidence on this optical element 121, the light rays propagate to the one or the other of the pixels of the macropixel 131 corresponding.

FIG. 1B illustrates ray traces from a point 2002 of the object 200, located outside the object plane π, upstream of this plane.

A first optical element 121i receives a first image ray R'i coming from point 2002. This first image ray R'i passes through the optical center of this first optical element 121i, and propagates up to a pixel of the macro-pixel 131i. corresponding to this first optical element 12h.

A second optical element 1212 receives a second image ray R'2 from the same point 2ΟΟ2. This second image ray R'2 passes through the optical center of this second optical element 1212, and propagates to a pixel of the macro-pixel 1312 corresponding to this second optical element 1212.

As explained above, from the image rays R'i and R'2, we can go back to the corresponding object rays, to determine the position of the point 2ΟΟ2 in the object space.

Figure IC illustrates ray traces from a point 2ΟΟ3 of the object 200, located outside the object plane π, downstream of this plane.

A first optical element 1213 receives a first image ray R'3 from point 2ΟΟ3. This first image ray R'3 passes through the optical center of this first optical element 1213, and propagates to a pixel of the macro-pixel 1313 corresponding to this first optical element I2I3.

A second optical element 12U receives a second image ray R'4 from the same point 2ΟΟ3. This second image ray R'4 passes through the optical center of this second optical element 1214, and propagates to a pixel of the macro-pixel 1314 corresponding to this second optical element I2I4.

As explained above, from the image rays R'3 and R'4, we can go back to the corresponding object rays, to determine the position of the point 2ΟΟ3 in the object space.

The points 200ι, 2ΟΟ2, 2ΟΟ3 form a sampling of the points of the object 200, making it possible to reconstruct a three-dimensional representation of the object 200.

Experimentally, we note that when the input optical system 110 has geometric aberrations, including distortion, the three-dimensional representation of the object 200 loses accuracy.

An object of the present invention is to provide a plenoptic camera, in which the loss in accuracy on the three-dimensional reconstruction of an object, due to the distortion aberrations of the input optical system, is at least partially corrected, preferably completely corrected. .

More generally, an object of the present invention is to provide a plenoptic camera, having a correction of the effect of the distortion aberrations of the input optical system.

STATEMENT OF THE INVENTION

This objective is achieved with a plenoptic camera comprising: an optical input system configured to receive light rays from an object to be imaged; a matrix optical sensor comprising a plurality of pixels; and an array of optical elements, arranged between the input optical system and the matrix optical sensor, each optical element being associated with a plurality of pixels of the matrix optical sensor, and the optical centers of the optical elements being distributed according to a grid of optical elements. distribution, at the points of intersection between a first series of lines and a second series of lines of this distribution grid.

According to the invention, the lines of the first and second series of rows of the distribution grid comprise a plurality of curved lines.

In addition, an object grid, whose distribution grid is the image by the input optical system, consists of a first series of straight lines, parallel to each other, and of a second series of parallel straight lines. between them and secant (preferably perpendicular) with the lines of the first series of lines.

In other words, the optical elements according to the invention are not distributed in rows and columns, but according to a distribution figure which depends directly on the optical characteristics of the input optical system.

Therefore, this distribution figure takes into account the geometric distortion aberrations of the input optical system.

This distribution figure makes it possible to limit, or even cancel, a loss in precision in the three-dimensional reconstruction of an object, due to these geometric aberrations.

Throughout the text, the term "grid" is not limited to a square or rectangular grid formed of a first series of horizontal lines and a second series of vertical lines.

According to the invention, the term "grid" can in particular designate a distorted square or rectangular grid. In this case, the grid is formed of a first series of lines that do not intersect, and a second series of lines that do not intersect. Each line of the first series of lines crosses all the lines of the second series of lines. Lines of the first and / or second series of lines are curved lines.

According to a first embodiment of the invention, the input optical system has a barrel-type optical aberration, and the curved lines of the distribution grid are curved towards the outside of said grid.

According to a second embodiment of the invention, the input optical system has a pincushion-type optical aberration, and the curved lines of the distribution grid are curved towards the inside of said grid.

The lines of the first series of lines of the object grid can be distributed at regular intervals according to a first grid step, and / or the lines of the second series of lines of the object grid can be distributed at regular intervals according to a second step. grid may be equal to the first grid step.

Alternatively, the lines of the first series of lines of the object grid may be distributed at irregular intervals, and / or the rows of the second series of lines of the object grid may be distributed at irregular intervals.

Preferably, the distances between two lines directly adjacent to the first series of lines of the object grid are each between 95% and 105% of a first average value, and / or the distances between two lines directly adjacent to the second series rows of the object grid are each between 95% and 105% of a second average value that may be equal to the first average value.

According to this same variant, the plenoptic camera advantageously has a random distribution of the distances between two lines directly adjacent to the first series of lines of the object grid, and / or by a random distribution of the distances between two lines directly adjacent to the second series of lines of the object grid.

The array of optical elements may be a microlens array or a pinhole array.

The plenoptic camera advantageously comprises a first array of optical elements, and a second matrix of optical elements coplanar with the first matrix, the optical elements of the first matrix being distributed in a first distribution grid, image by the optical system of input of a first object grid, and the optical elements of the second array being distributed according to a second distribution grid, image by the input optical system of a second object grid.

According to an advantageous embodiment, each optical element is mounted on a mobile support connected to an actuator, the mobile support and the actuator being configured together to move the optical element independently of the other optical elements.

In a variant, the matrix optical sensor may be mounted in a plane.

According to another variant, the plenoptic camera according to the invention may comprise an array of individually controllable shutters, upstream of the matrix of optical elements, each shutter being aligned with an optical element of said matrix to block or pass radii light having passed through the optical input system.

According to another variant, the plenoptic camera according to the invention may comprise several sets of an array of optical elements and a matrix optical sensor, and at least one splitter plate, for separating a light beam having passed through the optical input system. in several contributions each directed to one of said sets.

According to another variant, the plenoptic camera according to the invention may comprise at least two matrices of optical elements juxtaposed or nested in the same plane, each dedicated to a distinct wavelength, an optical filter being arranged upstream of each element. optical. The invention also relates to a method for positioning the optical elements of a plenoptic camera according to the invention, the method comprising the following steps: definition of the object grid, located in an object plane, conjugated by the optical input system a plane for receiving the optical centers of said optical elements; calculating the image of the object grid by the input optical system, to obtain the distribution grid; and positioning the centers of the optical elements on the points of intersection of the distribution grid.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will be better understood on reading the description of exemplary embodiments given purely by way of indication and in no way limiting, with reference to the appended drawings in which: FIGS. 1A to 1C schematically illustrate a plenoptic camera according to FIG. prior art; FIG. 2 diagrammatically illustrates a plenoptic camera according to the prior art, and sampling points of the object space; Figure 3 illustrates in another view, the plenoptic camera of Figure 2; FIG. 4 illustrates an example of distribution of the optical elements, in a plenoptic camera according to the prior art; FIG. 5 illustrates a plenoptic camera according to the prior art, in which the input optical system exhibits distortion; FIG. 6 illustrates a plenoptic camera according to the invention; FIGS. 7A and 7B respectively illustrate an object grid and a distribution grid, according to a first embodiment of the invention; FIG. 7C illustrates the distribution of the optical elements according to the invention, according to the distribution grid of FIG. 7B; FIG. 8 illustrates a second example of distribution of the optical elements according to the invention; FIGS. 9A and 9B respectively illustrate an object grid and the corresponding distribution of the optical elements, according to a third embodiment of the invention; and Figures 10A and 10B illustrate a fourth embodiment of the invention

DETAILED PRESENTATION OF PARTICULAR EMBODIMENTS

As detailed in the introduction, in order to determine the three-dimensional coordinates of a point of the object space using a plenoptic camera, this point must be imaged on at least two different pixels of the matrix optical sensor 130. The point is then defined by the intersection of the corresponding object rays, in the object space, if this intersection exists in the object space.

Therefore, the three-dimensional coordinates can only be determined by a finite number of points, forming sample points of the object space.

The more sample points in the object space, the more likely the object will pass through many of these sample points. In other words, the more sample points in the object space are numerous, the more likely it is that the sampling points of the object are numerous.

Now, the more a large number of points of the object are sampled, the better the accuracy of the three-dimensional reconstruction of this object.

An idea underlying the invention is to understand the origin of the loss in accuracy of the three-dimensional reconstruction, observed when the input optical system has distortion.

It is shown that this loss in precision results from the reduction of a number of sampling points of the object space, by reducing a number of intersection of rays in the object space.

Figure 2 schematically illustrates a plenoptic camera 100 according to the prior art. Each optical element 121 corresponds to a macro-pixel 131, and a plurality of image rays R 'propagating in a straight line through the optical center of this optical element 121 (see FIG. 1A). At each image ray R ', there corresponds an object ray R, in the object space, which is the conjugate of the image ray R', by the input optical system 110.

The image rays R 'associated with the same optical element 121 correspond to object rays R, which intersect in the object plane τι. The set of object rays R ,, associated with all the optical elements 121 of the matrix 120, define a multitude of intersection points Pj, located in the object space.

These points of intersection Pj form the sampling points of the object space.

Figure 3 illustrates the plenoptic camera of Figure 2.

Figure 3 does not respect the rules of perspective. The object space, upstream of the input optical system 110, is represented in a perspective view, the object plane π being shown for three quarters. The image space, downstream of the input optical system 110, is represented in a perspective view, the image plane π 'being shown for three quarters. The object space and the image space are represented according to different angles of view.

Figure 3 illustrates: the image rays R ',, each passing through a pixel of a macro-pixel 131 and the optical center of the corresponding optical element 121; the object rays Ri, each being the conjugate of an image ray R ', by the input optical system 110; and the points of intersection Pj, each at the intersection between at least two object rays Ri.

As detailed above, each optical element 121 corresponds to a plurality of object rays R ,, which intersect in the object plane π.

Each optical element 121 therefore defines a point in the object plane π.

These different points of the object plane π are distributed according to a rectangular grid 140, where each of these points corresponds to the intersection between a vertical line and a horizontal line of the grid 140.

In other words, the points of intersection of the rectangular grid 140 are the conjugates by the optical input system of the respective optical centers of the optical elements 121.

FIG. 4 illustrates an example of matrix 120 of optical elements 121 according to the prior art. The optical elements 121 are divided into rows and columns, according to a rectangular distribution grid (shown in dashed lines). At each point of intersection between a horizontal line and a vertical line of the distribution grid, is the optical center of an optical element 121.

For example, two optical elements 121a, 121b (see FIG. 3), belonging to the same column of optical elements, are considered in the matrix 120.

The optical centers of the optical elements 121a, 121b are the conjugates of the points of intersection 141a and 141b of the gate 140, by the optical input system.

Each optical element 121a, respectively 121b, is also associated with a macro-pixel 131a, respectively 131b.

Vertical planes each receiving the optical centers of the optical elements 121a and 121b, and a column of pixels of the set formed by the macro-pixels 131a and 131b are considered.

Each of these vertical planes receives a set of coplanar image rays R ', each image ray R' being associated with the optical element 121a, or 121b.

This set of coplanar R 'image rays corresponds to a set of object rays R ,, each object ray R being associated with the optical element 121a, or 121b.

In the absence of distortion on the input optical system, these object rays R are also coplanar.

They extend in the same vertical plane passing through the points of intersection 141a and 141b.

Since these object rays R, are coplanar, there are many points of intersection Pj between an object ray associated with the optical element 121a, and an object ray associated with the optical element 121b.

Each intersection point Pj forms a sampling point of the object space.

We are now interested in the case in which the input optical system (referenced then 610 in the figures) has a geometric aberration, in particular distortion.

Distortion is a geometric aberration by which the image of a line is not necessarily a straight line.

The distortion is most visibly manifested on the straight lines: a square or rectangular grid imaged by a high-distortion optical system will thus for example have an outwardly curved planar grid shape (barrel distortion), or a plane shape of gate deformed inward (pincushion distortion), or a combination of the two distortions, or any other geometric distortion.

FIG. 5 corresponds to the representation of FIG. 3, in which the effect of this distortion has been highlighted.

In practice, this distortion is still present, but to varying degrees.

As detailed above, each optical center of an optical element 121 is the conjugate of a point of intersection in the object plane π.

These different points of intersection define a deformed grid 540, where each of these points of intersection corresponds to the intersection between a first line of the deformed grid 540, and a second line of the deformed grid 540.

The deformed gate 540 is the conjugate, by the input optical system 610, of the rectangular distribution grid in which the optical elements 121 are distributed (see FIG. 4).

The deformed grid 540 consists of a first series of lines, substantially vertical, and a second series of substantially horizontal lines.

All the lines of the deformed grid 540, except the two lines defining the center of the grid 540, are curved toward the center of the grid. The curvature increases as one moves away from the center of the deformed grid 540.

As before, two optical elements 121a, 121b, belonging to the same column of optical elements, are considered in the matrix 120, and a column of pixels of the set formed by the two corresponding macro-pixels 131a and 131b.

These two optical elements 121a, 121b are associated with a point of intersection 541a, respectively 541b of the grid 540.

The points 541a and 541b of the gate 540 are located on a curved line of the gate 540. Therefore, the axis connecting the points 541a and 541b is not a vertical axis.

Thus, the points 541a and 541b are not located in the same vertical plane.

An image ray R'a associated with the optical element 121a, and propagating in the vertical plane receiving the optical elements 121a and 121b and said column of pixels, is the conjugate of an object ray Ra situated in a first vertical plane, receiving point 541a.

An image ray R'b associated with the optical element 121b, and propagating in the vertical plane receiving the optical elements 121a and 121b and said column of pixels, is the conjugate of an object ray Rb situated in a second vertical plane, receiving point 541b, and distinct from the first vertical plane.

Therefore, the object rays Ra and Rb are not coplanar. They do not intersect, and do not together define a sampling point of the object space.

Thus, the inventors have been able to show that the distortion aberrations of the input optical system have the effect of reducing a number of sampling points of the object space.

They had the idea to modify the distribution in the plane of the optical elements of the matrix of optical elements, to find the lost sampling points and thus cancel totally or partially the effect of this distortion.

FIG. 6 illustrates a plenoptic camera 600 according to the invention.

The plenoptic camera 600 will only be described for its differences with respect to the plenoptic camera 100 according to the prior art.

As in Figures 3 and 5, Figure 6 does not respect the laws of perspective, the object space and the image space being represented at different angles of view.

We find the input optical system 610, the matrix 620 of optical elements 621, and the matrix optical sensor 630.

As detailed above, each optical center of an optical element 621 is the conjugate of a point situated in the object plane τι.

These different points of the object plane π, are distributed according to a rectangular grid 640, where each of these points corresponds to the intersection between a vertical line and a horizontal line of the grid 640. This rectangular grid 640 is called "object grid".

The object grid 640 extends in a plane orthogonal to the optical axis of the input optical system, in particular in the object plane π. It is not a material element of the plenoptic camera 600, but a virtual object simply used to describe a distribution in the object space.

According to the invention, the optical elements 621 are distributed according to a distribution grid 650, which is the image of this object grid, by the input optical system 610.

The distribution grid 650 comprises a first series of lines, substantially vertical, and a second series of substantially horizontal lines. At each intersection between a line of the first series and a line of the second series is the optical center of an optical element 621 of the matrix 620.

The distribution grid extends in the image plane π ', conjugated by the optical input system of the object plane π.

The distribution grid 650 does not designate a hardware element of the plenoptic camera 600, but a virtual object simply used to describe a distribution in the image space.

The distribution grid 650 is the image of a rectangular grid, by the input optical system 610 which here has a barrel distortion aberration.

Consequently, the distribution grid 650 is not a rectangular grid, but a grid deformed by the aberrations of the input optical system 610.

All the lines of the distribution grid 650, except the two lines defining the center of the grid 650, are curved towards the outside of the grid. The curvature increases as one moves away from the center of the distribution grid 650.

As before, two optical elements 621a, 621b are considered situated on the same substantially vertical line of the distribution grid 650.

These two optical elements 621a, 621b are associated with a point of intersection 641a, respectively 641b of the object grid 640.

Since the object grid 640 is a rectangular grid, the points 641a, and 641b are located on a line of this rectangular grid 640, in the same vertical plane.

An image ray R'a associated with the optical element 621a corresponds to an object ray Ra situated in this vertical plane.

An image ray R'b associated with the optical element 621b corresponds to an object ray Rb situated in this vertical plane.

The object rays Ra and Rb are therefore coplanar, despite the distortion aberration of the input optical system 610, and thanks to the distribution according to the invention of the optical elements 621.

These object rays Ra and Rb intersect, and together define a sampling point of the object space, as is the case in the absence of aberration on the input optical system (see Figure 3).

We thus find the sampling points of the object space that were lost in the prior art, in the presence of distortion aberrations on the input optical system.

Thus, the loss in precision in the three-dimensional reconstruction of an object due to these geometric aberrations is limited, and even canceled.

Figure 7A illustrates, in a front view, the object grid 640.

This is a rectangular grid consisting of a series of vertical straight lines 642, called the first series of lines of the object grid, and a series of horizontal straight lines 643, called the second series of lines of the grid object.

The lines of the first series of lines 642 are evenly distributed. In other words, a distance DI between two vertical lines 642 directly adjacent is fixed in space, regardless of the two vertical lines considered.

In the same way, the lines of the second series of lines 643 are equi-distributed. A distance D2 between two directly adjacent horizontal lines 643 is fixed in space, regardless of the two vertical lines considered.

The distances DI and D2 can be equal.

Figure 7B illustrates the distribution grid 650, solid line. The distribution grid 650 is the image of the object grid 640 shown in FIG. 7A, by the input optical system having barrel distortion.

The distribution grid 650 consists of a series of substantially vertical lines 652, called the first series of lines of the distribution grid, and a series of substantially horizontal lines 653, called the second series of lines of the distribution grid.

The lines of the first series of lines 652, respectively the second series of lines 653, consist of lines curved outwardly of the grid 650, and of a straight line passing through the center of the grid 650.

FIG. 7B also shows, for comparison, the distribution according to the prior art of the optical elements of the matrix of optical elements.

FIG. 7C illustrates the distribution according to the invention of the optical elements 621 of the matrix 620.

These optical elements 621 are distributed according to the distribution grid 650 illustrated in FIG. 7B. At each point of intersection between a line of the first series of lines 652 and a line of the second series of lines 653 is the optical center of an optical element 621.

This distribution of optical elements 621 along curved lines goes against the technical prejudices of those skilled in the art, which generally seeks to dispose these optical elements regularly.

It will be noted that, in a logical manner, the modification according to the invention of the distribution of the optical elements modifies the distribution of the macro-pixels on the matrix optical sensor.

As in the prior art, the plenoptic camera is advantageously sized so that the macro-pixels do not overlap.

FIG. 8 shows another distribution of the optical elements 621 of the matrix 620, when the input optical system has a pincushion distortion.

The lines of the first series of lines 852, respectively the second series of lines 853, consist of lines curved towards the inside of the distribution grid 850, and of a straight line passing through the center of the grid 850. The invention is not limited to these examples of distribution grids, and the optical elements can be distributed in a multitude of distribution grids, as a function, for example, of the geometric aberrations of the input optical system.

For example, when the input optical system exhibits a third-order barrel distortion in the Seidel polynomials, compensated by a fifth-order pincushion distortion in the Seidel polynomials, the distribution grid consists of two lines. straight through the center of the grid, and wave-shaped curved lines. Each curved line generally has two points of inflection, on either side of the middle of this line.

According to another variant, the distribution grid is the image of an object grid, in which the distance between the vertical lines is different from the distance between the horizontal lines.

According to one possible variant but less preferred, the distribution grid is the image of an object grid, in which the first series of parallel straight lines between them and the second series of straight lines parallel to each other, are intersecting but not perpendicular.

Figures 9A and 9B illustrate an advantageous embodiment of the invention.

According to this embodiment, the vertical lines 942 of the object grid 940 are not evenly distributed, or in other words they are irregularly distributed.

In other words, a distance between two vertical lines 942 directly adjacent is variable in space, according to the two vertical lines considered. The irregularity is configured to minimize the greatest distance between any point in the object space and the nearest sampling point.

The distance values between two directly adjacent vertical lines 942 are distributed, preferably randomly, around a first average value, and within a first predetermined interval.

The first average value is the average of said distances between two vertical lines 942 directly adjacent.

Random distribution refers to a distribution whose exact positions are randomly assigned.

For example, a lower bound of this first interval is 95% of said first average value, and an upper bound of this first interval is 105% of said first average value.

Alternatively, a lower bound of this first interval is 90% of said first average value, and an upper bound of this first interval is 110% of said first average value.

In the same way, the horizontal lines 943 of the object grid 940 are not equi-distributed, the distance values between two directly adjacent horizontal lines 943 being distributed, preferably randomly, around a second average value, and at the same time. within a second predetermined interval.

The second average value is the average of said distances between two directly adjacent horizontal lines 943.

The lower and upper bounds of the second interval are defined in the same way as those of the first interval.

The first and second average values may be equal.

FIG. 9A illustrates this object grid 940.

The distribution grid 950 according to the invention is illustrated in FIG. 9B, the input optical system here having a barrel distortion.

The optical elements 621 of the matrix 620 are arranged at intersections of the lines of this distribution grid 950.

This embodiment makes it possible to further increase a number of sampling points of the object space, and to obtain a more homogeneous distribution of these points in the object space.

It thus makes it possible to further improve a precision of the reconstruction in three dimensions.

This embodiment corresponds to a clever combination of the present invention, with the teaching of the patent application FR-1558338.

According to a variant not shown, one of the two series of lines of the object grid is equi-distributed.

FIGS. 10A and 10B illustrate another embodiment according to the invention, in which the plenoptic camera comprises two matrices of interleaved optical elements.

Figure 10A illustrates the two corresponding object grids 1040i (in solid lines) and 10402 (in dashed lines).

Each object grid 10401 and 10402 is a rectangular grid of the type of the grid object of the embodiment of FIG. 6.

The two object grids 10401 and 10402 are coplanar, and shifted relative to each other.

According to the invention, the plenoptic camera comprises a first matrix 620i of optical elements 621i (in solid lines), and a second matrix 6202 of optical elements 6212 (in dashed lines).

The optical elements 621i are distributed according to a first distribution grid, image by the optical input system of the object grid 1040i.

The optical elements 6212 are distributed according to a second distribution grid, image by the optical input system of the object grid 10402.

All optical elements 621i and 62l2 are coplanar.

Note that in the absence of distortion on the input optical system, these two optical element arrays together form a staggered arrangement of the optical elements.

This embodiment may be combined with the embodiment of Figs. 9A and 9B.

In order to further improve the density of the sampling points in the object space, and therefore the accuracy of the three-dimensional reconstruction of an object placed in this space, a dynamic sampling of this object space can be performed.

In other words, temporal or spatial multiplexing of the plenoptic camera in different ways, associated with different series of sample points of the object space, is performed so that the total number of sample points of the object space is increased.

For example, each optical element is mounted on a movable support connected to an actuator, to control a displacement of the optical element in the plane π ', independently of the position of the other optical elements.

It is thus possible to move the optical elements slightly, to arrange them successively according to different distribution grids. Each distribution grid corresponds to a certain sampling of the object space. Thus, in fine, a denser sampling of the object space is realized. The displacement is of the order of a fraction of the width of an optical element.

In a variant, the matrix of optical elements remains fixed, and the matrix optical sensor is mounted on a mobile support, connected to an actuator for controlling a displacement in the plane of the matrix optical sensor.

The displacement of the matrix optical sensor comprises a component parallel to the axis of the width of the matrix optical sensor and / or a component parallel to the axis of the length of the matrix optical sensor, in particular a horizontal component and / or a vertical component. Each component is less than the pixel pitch along the axis considered.

In a variant, the macro-pixels associated with the different optical elements overlap partially. The same pixel can then correspond to several optical elements.

A network of controllable shutters such as a liquid crystal screen is placed upstream of the matrix of optical elements, to block or let the light propagating towards the optical elements, so that at each moment a pixel does not corresponds to a single optical element.

A plurality of states of the array of shutters are defined. Each state has a different set of sampling points in the object space. By varying the state of the shutter array over time, a denser sampling of the object space is ultimately performed.

According to another variant, the plenoptic camera comprises, juxtaposed or nested in the same plane, at least two arrays of optical elements each dedicated to a different wavelength.

Upstream of each optical element is a bandpass filter, for associating each optical element with the detection of a particular wavelength, and defining said plurality of optical element arrays. Each matrix of optical elements corresponds to a different sampling of the object space. In operation, the object is illuminated successively at several wavelengths.

According to another variant, the plenoptic camera comprises several matrices of non-coplanar optical elements, each associated with a clean matrix optical sensor. One or more splitter blades (s) allows (tent) to separate a luminous flux having passed through the optical input system in several contributions. Each contribution is brought to one of the optical element matrices.

Preferably, the plenoptic camera comprises two matrices of optical elements perpendicular to each other, inclined at 45% relative to a separating plate. A first array of optical elements is opposite the optical input system, orthogonal to its optical axis.

Each array of optical elements may be dedicated to a different wavelength and / or sampling points different from the object space. The invention also relates to a method for positioning optical elements 621.

For this, we define an object grid formed by a first series of parallel lines intersecting with a second series of parallel lines. The object grid is placed in an object plane π, conjugated with a plane configured to receive the matrix 620 of optical elements (and more particularly the optical centers of these optical elements).

By simulation, the image of the object grid is calculated by the input optical system to obtain the distribution grid according to the invention, the input optical system having a distortion aberration. This step requires having previously characterized the optical response of the input optical system.

According to a less preferred variant, the image of the object grid is determined experimentally by the input optical system.

Next, the optical centers of the optical elements 621 are positioned at the intersection points of the lines of this distribution grid. The invention makes it possible at least partially to correct the effect of a distortion aberration of the input optical system.

In particular, it allows the use of an input optical system having a strong distortion aberration, without this affecting the accuracy of the three-dimensional reconstruction using the plenoptic camera. The invention is particularly advantageous when it is necessary to place the plenoptic camera close to the object to be imaged.

In this case, in order to keep a wide field, the input optical system has a short focal length (wide angle lens). However, such an optical system generally has strong distortion aberrations. The invention also makes it possible to reduce the manufacturing costs of a plenoptic camera, since it is no longer necessary to use an input optical system corrected for distortion aberrations, in order to overcome the disadvantages of distortion. The invention also makes it possible to use the entire field of view of the plenoptic camera, without having to reject the field edges, which are more sensitive to distortion. This improves the sampling of the object space, allowing a better accuracy of the three-dimensional reconstruction of an object.

Although the text develops more particularly the application of a plenoptic camera to the three-dimensional reconstruction of an imaged object, the invention is not limited to a use of the plenoptic camera as a three-dimensional vision system. One can for example use a plenoptic camera to reconstruct two-dimensional images of the same scene, with a focus in different planes of the object space.

Claims (15)

A plenoptic camera (600) comprising: an input optical system (610) configured to receive light rays from an object to be imaged; a matrix optical sensor (630) comprising a plurality of pixels; and a matrix (620; 620i; 62Ο2) of optical elements (621; 621i; 62I2) disposed between the input optical system and the matrix optical sensor, each optical element (621; 621i; 62I2) being associated with a a plurality of pixels of the matrix optical sensor, and the optical centers of the optical elements (621; 621; 62I2) being distributed according to a distribution grid at the points of intersection between a first series of lines and a second series of lines of this grid distribution; characterized in that the lines of the first and second series of lines (652, 653, 852, 853) of the distribution grid (650; 850; 950) comprise a plurality of curved lines, and that object grid (640; 940; 1040i; 10O2), whose distribution grid (650; 850; 950) is the image by the input optical system, consists of a first series of straight lines (642; 942 ), parallel to each other, and a second series of straight lines (643; 943), parallel to each other and intersecting with the lines of the first series of lines. 2. plenoptic camera (600) according to claim 1, characterized in that the input optical system (610) has an optical aberration type barrel distortion, and in that the curved lines (652, 653) of the grid of distribution (650; 950) are curved outwardly of said grid. Plenoptic camera (600) according to claim 1, characterized in that the input optical system (610) has a pincushion-type optical aberration, and that the curved lines (852; 853) of the grating of distribution (850) are curved towards the interior of said grid. 4. plenoptic camera (600) according to any one of claims 1 to 3, characterized in that the lines (642) of the first series of lines of the object grid (640) are distributed at regular intervals in a first step of grid, and / or the lines (643) of the second series of lines of the object grid (640) are distributed at regular intervals according to a second grid step that may be equal to the first grid step. 5. plenoptic camera (600) according to any one of claims 1 to 3, characterized in that the lines (942) of the first series of lines of the object grid (940) are distributed at irregular intervals, and / or the lines (943) of the second set of lines of the object grid (940) are distributed at irregular intervals. 6. plenoptic camera (600) according to claim 5, characterized in that the distances between two lines (942) directly adjacent to the first series of lines of the object grid (940) are between 95% and 105% of a first average value, and / or the distances between two lines (943) directly adjacent to the second series of lines of the object grid (940) are between 95% and 105% of a second average value which may be equal to the first average value. Plenoptic camera (600) according to claim 5 or 6, characterized by a random distribution of the distances between two lines (942) directly adjacent to the first series of lines of the object grid (940), and / or by a random distribution distances between two lines (943) directly adjacent to the second series of lines of the object grid (940). 8. Plenoptic camera (600) according to any one of claims 1 to 7, characterized in that the matrix (620; 620i; 62Ο2) of optical elements is a matrix of microlenses or a pinhole matrix. 9. plenoptic camera (600) according to any one of claims 1 to 8, characterized in that it comprises a first matrix (620i) of optical elements, and a second matrix (62Ο2) of optical elements, coplanar with the first matrix (620i), the optical elements (621i) of the first matrix being distributed in a first distribution grid, image by the optical input system of a first object grid (1040i), and the optical elements (62I2 ) of the second matrix being distributed according to a second distribution grid, image by the optical input system of a second object grid (IO4O2). 10. Plenoptic camera (600) according to any one of claims 1 to 9, characterized in that each optical element (621; 621i; 62I2) is mounted on a movable support connected to an actuator, the movable support and the actuator. being configured together to move the optical element independently of the other optical elements. 11. plenoptic camera (600) according to any one of claims 1 to 9, characterized in that the matrix optical sensor (630) is mounted in a plane. 12. Plenoptic camera (600) according to any one of claims 1 to 9, characterized in that it comprises a network of shutters controllable individually, upstream of the matrix (620; 620i; 62Ο2) of optical elements, each shutter being aligned with an optical element of said matrix for blocking or passing light rays having passed through the optical input system (610). 13. Plenoptic camera (600) according to any one of claims 1 to 9, characterized in that it comprises several sets of a matrix (620; 620i; 62Ο2) of optical elements and a matrix optical sensor (630). and at least one splitter plate for separating a light beam having passed through the input optical system (610) into plural contributions each directed to one of said sets. 14. plenoptic camera (600) according to any one of claims 1 to 9, characterized in that it comprises at least two matrices (620; 620i; 62Ο2) optical elements juxtaposed or nested in the same plane, each dedicated at a distinct wavelength, an optical filter being arranged upstream of each optical element. 15. A method for positioning the optical elements (621; 621i; 62I2) of a plenoptic camera (600) according to any one of claims 1 to 14, characterized in that it comprises the following steps: definition of the object grid (640; 940), located in an object plane (π), conjugated by the optical input system (610) of a plane (π ') for receiving the optical centers of said optical elements (621; 621i; 6212) ; calculating the image of the object grid by the input optical system (610) to obtain the distribution grid (650; 850; 950); and positioning the centers of the optical elements (621; 621i; 6212) at the points of intersection (641a, 641b) of the distribution grid (650; 850; 950).