WO2015067764A1

WO2015067764A1 - Optical assembly for plenoptic camera

Info

Publication number: WO2015067764A1
Application number: PCT/EP2014/074059
Authority: WO
Inventors: Valter Drazic; Neus SABATER
Original assignee: Thomson Licensing SAS
Current assignee: Thomson Licensing SAS
Priority date: 2013-11-08
Filing date: 2014-11-07
Publication date: 2015-05-14
Anticipated expiration: 2016-05-08
Also published as: TW201524208A; EP2871830A1

Abstract

A plenoptic camera comprising a primary lens (20), a lenslet array (21), a photosensor array (13), wherein a first color filter array (22) is arranged on said lenslet array and a second color filter is arranged on said photosensor array (13).

Description

OPTICAL ASSEMBLY FOR PLENOPTIC CAMERA

1. Domain of the invention.

The invention relates to the domain of optical assembly for plenoptic camera.

2. Prior art

According to the prior art, it is known to acquire different views of a same scene with a plenoptic camera, also called light-field camera. Figure 1 illustrates such a plenoptic camera 1 according to prior art. The plenoptic camera 1 is composed of a lens arrangement associated with a photosensor array 13, also called image sensor array 13. The photosensor array 13 comprises a large number m of photosensors 131 , 1 32, 133 to 1 m arranged in the form of a grid of X columns and Y lines, m corresponding to X time Y. A color filter array (CFA) 12 is arranged on the photosensor array 13. The CFA 12 typically arranges RGB (Red, Green and Blue) color filters on the photosensor array, the RGB arrangement taking for the example the form of a Bayer filter mosaic. Typically, one color filter (red, green or blue filter) is associated with one photosensor according to a predetermined pattern, which comprises 50 % green, 25 % red and 25 % blue in the example of a Bayer filter, such a pattern being also called RGBG, GRGB or RGGB pattern. The lens arrangement comprises a primary lens 10, also called main lens, and a lenslet array 1 1 , which comprises a plurality of n microlenses 1 1 1 , 1 12, 1 n, n being a positive integer greater than or equal to 2. The microlenses 1 1 1 , 1 12, 1 n are arranged in such a way as to be optically each associated with a plurality of photosensors. The number of photosensors optically associated with one microlens corresponds to the number of views of the scene acquired with the plenoptic camera 1 . To obtain the different views, the raw image (i.e. the color sensor data acquired with the photosensor array 13) is demosaiced then de-multiplexed. After the demosaicing step, RGB image data values are obtained at each pixel location for each view.

The demosaicing and de-multiplexing steps may lead to view- crosstalk artifacts and to missing color information artifacts (especially when the number of pixels optically associated with a microlens is an even number). Missing color information in a view means that at least one of the color component RGB (i.e. Red, Green or Blue) is missing in a view. Hence, whatever the number of pixel associated with a microlens is, even or odd, the views suffer from cross-talk and/or from missing color information.

3. Summary of the invention

The purpose of the invention is to overcome at least one of these disadvantages of the prior art.

More specifically, the purpose of the invention is to enhance the quality of images of a scene acquired when using a plenoptic optical assembly.

The invention relates to a plenoptic optical assembly comprising a primary lens and a lenslet array, the lenslet array comprising a plurality of microlenses, a color filter array being arranged on the lenslet array.

According to a particular characteristic, the color filter array comprises a plurality of color filters, the plurality of color filters being optically associated with the plurality of microlenses according to a predetermined pattern in such a way that each single color filter is optically associated with a different microlens.

Advantageously, the predetermined pattern associates two green color filters, one blue color filter and one red color filter with respectively each microlens of a set of four microlenses arranged in two lines and two columns.

According to a specific characteristic, the geometry of a microlens of the plurality of microlenses is adjusted according to the color filter optically associated with the microlens.

Advantageously, each microlens of the plurality of microlenses has a plane side and a convex side, the lenslet array having a first side, called plane side, formed with the plane sides of the plurality of microlenses and a second side, called convex side, formed with the convex sides of the plurality of microlenses, the color filter array being arranged on the first side.

According to another characteristic, the second side of the lenslet array faces the primary lens.

Advantageously, both first and second sides of each microlens of the plurality of microlenses are convex, the lenslet array having a first side formed with the first convex sides of the plurality of microlenses and a second side formed with the second convex sides of the plurality of microlenses, the second side of the lenslet array facing the primary lens, the color filter array being arranged on the first side. According to a particular characteristic, each microlens of the plurality of microlenses has a circular form.

According to another characteristic, each microlens of the plurality of microlenses has a rectangular form or a square form.

Advantageously, each microlens of the plurality of microlenses has an hexagonal form.

The invention also relates to a plenoptic camera comprising the plenoptic optical assembly.

According to an advantageous characteristic, the plenoptic camera comprises a photosensor array, a second color filter array being arranged on the photosensor array.

Advantageously, the plenoptic camera comprises a photosensor array comprising a plurality of photosensors, at least two photosensors of the plurality of photosensors being optically associated with each microlens of the plurality of microlenses. A group of at least two photosensors of the plurality of photosensors is optically associated with each microlens of the plurality of microlenses; the second color filter array comprises a plurality of second color filters, the plurality of second color filters being optically associated with the groups of photosensors according to the determined pattern in such a way that each single second color filter is optically associated with a different group of photosensors.

According to a specific characteristic, a gap is formed between the lenslet array and the photosensor array.

According to a further characteristic, a gap is formed between the first color filter array and the second color filter array.

According to another characteristic, the color filter array faces the photosensor array.

The invention also relates to a telecommunication device comprising the plenoptic camera.

4. List of figures

The invention will be better understood, and other specific features and advantages will emerge upon reading the following description, the description making reference to the annexed drawings wherein :

- figure 1 shows a plenoptic camera, according to the prior art;

- figure 2 shows a plenoptic optical assembly, according to a particular embodiment of the invention; - figure 3 shows light rays bundles passing through the plenoptic optical assembly of figure 2 and generating different views of a scene, according to a particular embodiment of the invention;

- figure 4 shows a color filter array associated with a lenslet array of the plenoptic optical assembly of figure 2, according to a particular embodiment of the invention;

- figures 5A and 5B show optical phase space representations of the association of points of a scene with microlens and color filter of the plenoptic optical assembly of figure 2, according to particular embodiments of the invention;

- figure 6 shows a representation of the different views of figure 3 once demultiplexed, according to a particular embodiment of the invention ;

- figures 7A and 7B show exemplar microlens arrangements for the plenoptic optical assembly of figure 2, according to two particular embodiments of the invention;

- figure 8 shows a plenoptic camera, according to a particular embodiment of the present disclosure;

- figure 9 shows details of the plenoptic camera of figure 8, according to a particular embodiment.

5. Detailed description of embodiments of the invention.

The invention will be described in reference to a particular embodiment of a plenoptic optical assembly comprising a primary lens (also called main lens) and a lenslet array. The main lens comprises one or several lenses and the lenslet array comprises a plurality of microlenses, i.e. at least two microlenses. According to this particular embodiment, a first color filter array is arranged on the lenslet array.

According to an advantageous embodiment of the present principles, a second color filter array is arranged on the photosensor array of a plenoptic camera comprising the abovementioned plenoptic optical assembly.

It is understood by "a color filter array is arranged on the lenslet array" a juxtaposition of the color filter array and the lenslet array. The color filter array is either placed on the lenslet array in a way as to be in direct contact with a face of the lenslet array (i.e. there is a direct contact between the color filter array and the lenslet array without any additional layer between the color filter array and the lenslet array) or placed on the lenslet array in a way as to be without direct contact on a face of the lenslet array (i.e. there is no direct contact between the color filter array and the lenslet array, e.g. one or several transparent layers having a depth of one or several micrometers, associated with either the color filter array and/or the lenslet array, separating the color filter array from the lenslet array).

It is understood by "a color filter array is arranged on the photosensor array" a juxtaposition of the color filter array and the photosensor array. The color filter array is either placed on the photosensor array in a way as to be in direct contact with a face of the photosensor array (i.e. there is a direct contact between the color filter array and the photosensor array without any additional layer between the color filter array and the photosensor array) or placed on the lenslet array in a way as to be without direct contact on a face of the photosensor array (i.e. there is no direct contact between the color filter array and the photosensor array, e.g. one or several transparent layers having a depth of one or several micrometers, associated with either the color filter array and/or the photosensor array, separating the color filter array from the photosensor array).

Arranging the color filter array on the lenslet array instead of on the photosensor array (as done in the prior art) enables to reduce inter-view crosstalk and/or of missing color information and enhances the quality of images resulting from the acquisition of a scene using the plenoptic optical assembly. Arranging an additional color filter array on the photosensor array enables to reduce even more inter-view crosstalk artefacts.

Figure 2 shows a plenoptic optical assembly 201 (also called plenoptic lens unit), according to a particular and non-limitative embodiment of the invention. The plenoptic optical assembly 201 comprises a primary lens 20, similar to the primary lens 1 0 of figure 1 . The primary lens 20 is advantageously formed of one or more lens elements. The plenoptic optical assembly 201 also comprises a lenslet array 21 comprising at least two microlenses 21 1 , 212, 21 n. For purposes of illustration, the lenslet array 21 is shown with a relative small number of microlenses, but the number of microlenses may extend up to several thousand or even one or several million of microlenses. A color filter array 22 is arranged on the lenslet array 21 . The color filter array 22 advantageously comprises a plurality of color filters 221 , 22, 22n, for example 3 color filters, i.e. for example one red filter, one green filter and one blue filter. The color filters are advantageously arranged according to a predetermined pattern in such a way that each single color filter (i.e. red, green and blue) is optically associated with a different microlens. For example, the color filter 221 (red, green or blue) is optically associated with the microlens 21 1 , the color filter 222 (red, green or blue) is optically associated with the microlens 21 2 and the color filter 22n (red, green or blue) is optically associated with the microlens 21 n. An optical association of a microlens with a color filter means that all light rays passing through a given microlens also pass through the color filter optically associated with this given microlens. An example of an arrangement of the color filters with regard to the microlenses will be described hereinbelow with more details with regard to figure 4.

The plenoptic optical assembly 201 advantageously forms a lens unit adapted to be associated with a camera body 202. The camera body comprises a photosensor array 13, which comprises a plurality m of photosensors 131 , 132, 133 to 13m. Each photosensor corresponds to a pixel of the raw image of the scene acquired with the photosensor array, each pixel covering a part (also called a point) of the scene. For purposes of illustration, the photosensor array 1 3 is shown with a relative small number of photosensors 131 to 13m. Naturally, the number of photosensors is not limited to the illustration of figure 2 but extends to any number of photosensors, for example several thousand or several millions of photosensors, for example 1 2.4 megapixels, a pixel corresponding to a photosensor (e.g., corresponding to an array of 4088x3040 pixels/photosensors). For associating the plenoptic optical assembly 201 with the camera body 202, the plenoptic optical assembly 201 comprises a first attaching part and the camera body comprises a second attaching part, the first and second attaching parts being compatible. Thanks to the first and second attaching parts, the plenoptic optical assembly 201 may be clipped into the camera body 202 or the plenoptic optical assembly 201 may be screwed with the camera body 202. An example of such first and second attaching parts of a lens unit configured to be associated with a camera body may be found in the Japanese patent application JP2013-1051 51 A, which was published on May 30, 201 3. The first and second attaching parts are configured in such a way that, once the plenoptic optical assembly 201 and the camera body 202 have been put together, the plenoptic optical assembly 201 and the camera body 202 form a plenoptic camera configured for acquiring multiple views of a scene at each acquisition of the scene. To that aim, a plurality of photosensors of the photosensor array 13 are optically associated with each microlens 221 to 22n of the lenslet array 22. For example, each microlens 221 to 22n of the lenslet array 22 is sized to correspond to an array of 2x1 , 4x4 or 10x10 photosensors. Each photosensor of the plurality of photosensors optically associated with one single microlens enables to acquire raw data representative of a pixel of the scene according to one view angle, as it will be described with more details with regard to figures 3 and 4. According to a variant, the plenoptic optical assembly 201 and the camera body 202 form one single body and are assembled without being detachable.

In an advantageous way, each microlens of the lenslet array 21 has a plane side and a convex side. The convex side is advantageously oriented toward the primary lens while the opposite side, i.e. the plane side, is oriented toward the photosensors array 1 3. The first side of the lenslet array 21 , called plane side, is formed with the plane sides of the microlenses 21 1 to 21 n. The second side of the lenslet array 21 , called convex side, is formed with the convex sides of the microlenses 21 1 to 21 n. The color filter array 22 is advantageously arranged on the first side of the lenslet array 21 , the first side of the lenslet array 21 being oriented toward (or facing) the photosensors array 13. According to a variant, the color filter array 22 is arranged on the second side, i.e. the convex side, of the lenslet array 21 . According to this variant, the color filter array 22 is arranged on the side of the lenslet array 21 being oriented toward (or facing) the primary lens 20. According to another variant, each microlens of the lenslet array 21 is formed with two opposite convex sides. The lenslet array 21 then comprises two convex sides, the color filter array 22 being either arranged on the convex side being oriented toward (or facing) the primary lens 20 or on the convex side being oriented toward (or facing) the photosensors array 13.

Advantageously, the geometry of each microlens 21 1 to 21 n of the lenslet array 21 is adjusted according to the color filter that is optically associated with it. The geometry of a microlens comprises the radius of curvature of the convex side of the microlens. Then, the microlenses of the lenslet array 21 optically associated with a red filter all have a same first radius of curvature; the microlenses of the lenslet array 21 optically associated with a green filter all have a same second radius of curvature, which is different from the first radius of curvature; and the microlenses of the lenslet array 21 optically associated with a blue filter all have a same third radius of curvature, which is different from the first and second radius of curvature. Indeed, the focal length of a microlens depends from both the radius of curvature of the lens and the refractive index of the light, the refractive index being dependent from the wave length of the light, the wave lengths of each of the components of the light (for example red, green and blue components) being different from each other. As one microlens is optically associated with one specific color filter (for example either red, green or blue filter), a given microlens will let the wave lengths of the light components centered around the color of the color filter go through the given microlens (e.g. a microlens associated with a blue color filter will let the wave lengths centered around blue), and the radius curvature of the given microlens may be advantageously adapted to the color component of the color filter associated with it. The following equation may be used to optimize the radius of curvature of a microlens for each color component:

1 1 (n - l )t

P = I ₌ (_n _ i)

R₁ R₂ nR₁R₂

where

P is the power of the microlens,

f is the focal length of the microlens,

n is the refractive index of the lens material,

Ri is the radius of curvature (with sign) of the lens surface closest to the light source, i.e. closest to the primary lens,

R₂ is the radius of curvature of the lens farthest from the light source, i.e. closest to the photosensor array,

t is the thickness of the microlens (corresponding to the distance along the microlens axis between the two surface vertices, i.e. corresponding to the biggest thickness of the microlens along the microlens axis) According to a variant, the microlenses optically associated with two different color filters (for example the red and blue filters or the blue and green filters) have all the same radius of curvature while the microlenses optically associated with the other color filter (for example the green filter or the red filter respectively) have a radius of curvature different than the one used for the microlenses of the two different color filters. According to another variant, all the microlenses of the lenslet array 21 have the same geometry (i.e. the same radius of curvature), whatever the color filter with which they are optically associated. This variant has the advantage of easing the manufacturing process of the microlenses, the microlenses being all identical.

The lenslet array 21 is manufactured according to any method known by the skilled person in the art. The lenslet array 21 is for example manufactured using a polymer-on-glass approach, including a stamping or replication process wherein the lenslet array 21 is fabricated as a polymer attached to a transparent glass surface. The microlenses of the lenslet array are for example shaped by using photolithographic techniques as to give them the appropriate curvatures, especially for the convex side(s). One side of the lenslet array 21 is advantageously coated with the color filters according to a predetermined pattern, as explained with regard to figure 4. The color filter array is for exampled manufactured directly on the lenslet array, for example with gelatine dyeing, dyed polyimide, pigmented polyimide and pigmented acrylates, applied by photolithographic printing and electro- deposition or according to any method known by the skilled person in the art. According to a variant, a passivation layer is formed on the color filter array 22 to protect it. According to a variant, the color filter array is manufactured independently from the lenslet array. According to this variant, the color filter array 22 is assembled with the lenslet array 21 , for example by bonding technique. The color filter array 22 may be placed directly on the lenslet array, i.e. with a direct contact between the color filter array 22 and the lenslet array may. According to another example, the color filter array is placed on the lenslet array without direct contact, i.e. for example with a transparent layer between the color filter array 22 and the lenslet array 21 .

A gap is advantageously formed between the lenslet array 21 and the photosensor array 13. When the color filter array 22 is arranged on the side of the lenslet array 21 facing the photosensor array 13, it is understood that the gap formed between the lenslet array 21 and the photosensor array 13 is formed between the color filter array 22 and the photosensor array 13. The gap may be composed of air, of an optical material having an index n (for example a glass layer) or of multiple layers comprising at least one layer air layer and at least one optical material layer. Using a glass layer to form the gap has the advantages of keeping the lenslet array at a same distance from the photosensor array in each and every location and of reducing this distance when needed. If d is the distance between the output of the lenslet array and the photosensor array according to a longitudinal axis, having a layer composed of an optical material with an index n (n > 1 , for example n = 1 .5) between the lenslet array and the photosensor array enables to set the distance to d/n without modifying d. By adapting/modifying the index of the optical material of the layer forming the gap, it is possible to adapt/modify a parameter representative of the distance between the lenslet array and the photosensor array without modifying the distance d. According to an embodiment, the lenslet array 21 is placed and adjusted to be approximately at one focal length f (i.e. the focal ± 5 %) from the photosensor array 13, where f is the focal length of the microlenses. The lenslet array 21 is focused at the image plane of the primary lens 20 of the plenoptic optical assembly 201 and the microlenses of the lenslet array 21 are focused at infinity. According to another embodiment, in order to increase or maximize spatial resolution, i.e. to achieve sharper, higher spatial resolution of microlens images, the microlenses are focused on the image created by the primary lens inside the plenoptic optical assembly 201 and in front of the lenslet array 21 (i.e. between the lenslet array 21 and the primary lens 20), instead of being focused on the primary lens itself. According to this embodiment, the lenslet array 21 may be located at distances greater than f or less than f from the photosensor array 13. For example, the lenslet array 21 may be placed at distance 4/3 f from the photosensor array 13, or at other distances that are multiple of f, e.g. 0.5 f , 1 .5 f or ¾ f.

Figure 3 illustrates light rays bundles passing through the plenoptic optical assembly of figure 2, according to a particular and non- limitative embodiment of the invention. Figure 3 shows three light rays bundles passing through three microlenses 31 , 32 and 33 respectively, the microlenses 31 to 33 belonging to the lenslet array 21 . Before passing through the microlenses 30, 31 and 32, the light rays have passed through the primary lens 20. Points P 341 , P₂ 342, P₃ 343, P₄ 344, P₅ 345, P₆ 346, P₇ 347 and P₈ 348 correspond to the intersections between the light rays coming from the primary lens (not illustrated on figure 3) and the image plane 3. Light rays 321 , 322, 323, 324, 325 and 326 pass through the microlens 33 and represents the light coming from the scene, images of which being acquired with the plenoptic camera comprising the plenoptic optical assembly 201 , and passing through the primary lens 20 before passing through the microlens 33. The plane 30 represents the plane of the photosensor array 13 illustrated on figure 2. Pixels 351 , 352, 353, 354, 355 and 356 (which each correspond to a photosensor of the photosensor array) correspond to the projection points of respectively the points P₃ 343, P₄ 344, P₅ 345, P₆ 346, P₇ 347 and P₈ 348 along respectively the light rays 321 , 322, 323, 324, 325, 326 passing through the microlens 33. The light ray 321 forms an angle θι with an horizontal line passing through the centre of the microlens 33; the light ray 322 forms an angle θ₂ with an horizontal line passing through the centre of the microlens 33; the light ray 323 forms an angle θ₃ with an horizontal line passing through the centre of the microlens 33; the light ray 324 forms an angle θ₄ with an horizontal line passing through the centre of the microlens 33; the light ray 325 forms an angle θ₅ with an horizontal line passing through the centre of the microlens 33; and the light ray 326 forms an angle θ₆ with an horizontal line passing through the centre of the microlens 33. Each photosensor 351 to 356 acquires image data of respectively the points 343 to 348 in association with location data of the points 343 to 348, the location data corresponding to the angle θι to θ₆ associated with the light ray 321 to 326 hitting the photosensors 351 to 356. A same point of the image plane is viewed according to several angles via different microlenses, i.e. 6 angles θ-ι to θ₆ according to the non-limitative embodiment of figure 3. The number of views of a same point of the image plane corresponds to the number of photosensors optically associated with a microlens. For example, the point P₄ 344 is viewed through the microlens 33 along the light ray 322 according to a viewing angle θ₂, but also through the microlens 32 along the light ray 327 according to a viewing angle θ₃, through the microlens 31 along the light ray 328 according to a viewing angle θ₄, but also according to the viewing angles θ _; θ₅ and θ₂, even if not represented on figure 3.

Naturally, the number of photosensors optically associated with a microlens is not limited to six, figure 3 being an exemplar representation in 2D, i.e. a cross-section in the plane. The number of photosensors optically associated with a microlens may be comprised for example between 2 photosensors and up to several dozens or even one or several hundred of photosensors, several photosensors acquiring image data of points of the plane image 3 according to a same viewing angle, i.e. the photosensors belonging to a same line (respectively column according to the chosen convention for defining a viewing angle, i.e. with respect to horizontal or vertical plane respectively) of the array of photosensors optically associated with a same microlens acquire image data of several points according to a same viewing angle. The number of photosensors optically associated with a microlens is not limited to an integer either but also extends to non-integer numbers, for example X + ½ or X + 113, where X is an integer number. Figure 4 illustrates a part 41 of a color filter array associated with a part 40 of the lenslet array 21 of the plenoptic optical assembly 201 of figure 2, according to a particular and non-limitative embodiment of the invention. The part 40 of the lenslet array comprises 14 microlenses 401 to 414 arranged according to two columns and seven lines. The part 41 of the color filter array (CFA) illustrated on figure 4 corresponds to the part 40 of the lenslet array. The CFA part 41 comprises 14 areas 41 1 to 424, each of the 14 areas 41 1 to 424 corresponding and being associated with one different microlens 401 to 414. According to the example of figure 4, each microlens 401 to 414 has a square form and each area 41 1 to 424 has the same square form, meaning that the surface of one area 41 1 to 424 is identical to the surface of one microlens 401 to 414. According to this example, the whole surface of a microlens is coated with a color filter. The CFA part of figure 4 comprises three different color filters, for example a red filter, a green filter and a blue filter arranged according to a predetermined pattern in such a way that each single color filter is optically associated with a different microlens. The optical association between a color filter and a microlens means that all light rays passing through a given microlens also pass through the color filter optically associated with the given microlens. The predetermined pattern illustrated on figure 4 is called RGGB pattern for 'Red', 'Green', 'Green', 'Blue' pattern. Indeed, for a block of four color filters (or in an equivalent way a block of four microlenses, according to two columns and two lines), for example the color filters 41 1 , 41 8, 41 2 and 41 9, the upper left color filter 41 1 is 'Blue', the upper right color filter 418 is 'Green', the lower left color filter 412 is 'Green' and the lower right color filter 41 9 is 'Red'. In other words, the predetermined pattern associates two green color filters, one blue color filter and one red color filter with respectively each microlens of a set of four microlenses arranged in two lines and two columns. This predetermined pattern is then repeated for each block of four color filters (for example the block including the color filters 413, 420, 414 and 421 ; the block including the color filters 41 5, 422, 41 6 and 423) on the whole color filter array. If the number of microlenses of the lenslet array is a multiple of four, the surface covered by the 'Blue' color filter is 25 %, the surface covered by the 'Green' color filter is 50 % and the surface covered by the 'Red' color filter is 25 %. On each color filter 41 1 to 424 is represented an array of pixels which correspond, for each color filter, to the projection of the photosensors optically associated with the color filter onto the color filter. Each array of pixels has advantageously a same number of pixels and comprises 6 lines L-i to l_6 and 6 columns Ci to C₆. Pixels of a same line are advantageously seen/acquired according to a same viewing angle, the pixels of the line L, being seen according to the viewing angle θι with respect to the horizontal plane (similarly d with respect to the vertical plane), the pixels of the line L₂ being seen according to the viewing angle θ₂ with respect to the horizontal plane (similarly C₂ with respect to the vertical plane), the pixels of the line l_₃ being seen according to the viewing angle θ₃ with respect to the horizontal plane (similarly C₃ with respect to the vertical plane), the pixels of the line L₄ being seen according to the viewing angle θ₄ with respect to the horizontal plane (similarly C₄ with respect to the vertical plane), the pixels of the line l_₅ being seen according to the viewing angle θ₅ with respect to the horizontal plane (similarly C₅ with respect to the vertical plane) and the pixels of the line L₆ being seen according to the viewing angle θ₆ with respect to the horizontal plane (similarly C₆ with respect to the vertical plane).

Naturally, the number of columns and lines is not limited to six by may be comprised between two and for example 1 0, 20 or 50. The predetermined pattern is not limited to a RGGB pattern either. According to variants, the predetermined pattern may be a RGBE pattern with one of the green filters modified to 'Emerald' (for a block of four color filters); a CYYM pattern with one 'Cyan' filter, two 'Yellow' filters and one 'Magenta' filter (for a block of four color filters) ; a CYGM pattern with one 'Cyan' filter, one 'Yellow' filter, one 'Green' filter and one 'Magenta' filter; a RGBW pattern with one 'Red' filter, one 'Green' filter, one 'Blue' filter and one 'White' filter, several arrangement being possible (for example arranged on a block of four color filters with 'White' for the upper left filter, 'Red' for the upper right filter, 'Blue' for the lower left filter and 'Green' for the lower right filter; or arranged on a block of 4x4 color filters with 'White', 'Blue', 'White', 'Green' for the first line, 'Blue', 'White', 'Green', 'White' for the second line below the first line, 'White', 'Green', 'White', 'Red' for the third line below the second line and 'Green', 'White', 'Red', 'White' for the fourth line below the third line).

The form of the microlenses is not limited to a square either. Microlenses may also take the form of a rectangle or any other form. Figures 7A and 7B provides two different examples of the form that may be taken by the microlenses.

Figure 7A illustrates a first non-limitative exemplar microlenses arrangement with microlenses having a form different from a square, according to a particular embodiment. On figure 7A, each microlens 701 to 712 has a circular form, all microlenses 701 to 712 having the same radius. As to optimize the coverage of the photosensors array 70 by the lenslet array (a part of which being illustrated on figure 7B), the microlenses 701 to 712 are advantageously arranged in a quincunx arrangement, thus reducing the number of photosensors not optically associated with a microlens. As in the embodiment of figure 4, the microlenses 701 to 712 are coated with a color filter array according to a predetermined pattern. The pattern illustrated on figure 7A corresponds to a pattern of 2x2 microlenses corresponding to a RGGB pattern, i.e. for a block of four microlenses (or in other words for a block of four color filters) arranged with two lines and two columns, the upper left color filter associated with the upper left microlens 701 is 'Blue', the upper right color filter associated with the upper right microlens 702 is 'Green', the lower left color filter associated with the lower left microlens 704 is 'Green' and the lower right color filter associated with the lower right microlens 705 is 'Red'. This pattern is repeated for each and every block of four microlenses, for example for the block composed of the microlenses 707, 708, 71 0 and 71 1 in a way as to cover the whole lenslet array 21 . Of course, the predetermined pattern may be any pattern different from the RGGB pattern, for example a RGBE pattern, a CYYM pattern, a CYGM pattern or a RGBW pattern in the form of a block of 2x2 microlenses or in the form of a block of 4x4 microlenses.

Figure 7B illustrates a second non-limitative exemplar microlenses arrangement with microlenses having a form different from a square, according to a particular embodiment. On figure 7B, each microlens 71 1 to 714 has an identical hexagonal form. As to optimize the coverage of the photosensors array 71 by the lenslet array (a part of which being illustrated on figure 7B), the microlenses 701 to 712 are advantageously arranged in a quincunx arrangement, thus reducing the number of photosensors not optically associated with a microlens. Compared to the arrangement of figure 7A, the arrangement of figure 7B has the advantage that there is no photosensor not covered by a microlens in the space located between several hexagonal microlenses. As in the embodiment of figure 4, the microlenses 71 1 to 714 are coated with a color filter array according to a predetermined pattern. The pattern illustrated on figure 7B corresponds to a pattern of 2x2 microlenses corresponding to a RGGB pattern, i.e. for a block of four microlenses (or in other words for a block of four color filters) arranged with two lines and two columns, the upper left color filter associated with the upper left microlens 71 1 is 'Blue', the upper right color filter associated with the upper right microlens 712 is 'Green', the lower left color filter associated with the lower left microlens 713 is 'Green' and the lower right color filter associated with the lower right microlens 714 is 'Red'. This pattern is repeated for each and every block of four microlenses of the lenslet array in a way as to cover the whole lenslet array 21 . Of course, the predetermined pattern may be any pattern different from the RGGB pattern, for example a RGBE pattern, a CYYM pattern, a CYGM pattern or a RGBW pattern in the form of a block of 2x2 microlenses or in the form of a block of 4x4 microlenses.

Figures 5A and 5B illustrate optical phase space representations of the association of points of a scene with microlens and color filter of the plenoptic optical assembly of figure 2, according to particular and non- limitative embodiments of the invention.

Figure 5A shows in an optical phase space 50 (position and angle) which points 501 x₀,yo through x₀,y-i₃ (the coordinates of which being expressed in the space of the photosensor array) are seen under which angle 500 and which color RGB (either Red, Green or Blue). Figure 5A illustrates the association between the points 501 of the scene with microlenses L₀i to L₀₇ (corresponding respectively to the microlenses Lens₀i 401 to Lens₀₇ 407 of figure 4) and the color filter 502 and 503. The color filter 502 is illustrated with average grey corresponding to the color filter 'Blue' associated with microlenses L₀i , L₀₃, L₀₅ and L₀₇ (respectively referenced 401 , 403, 405 and 407 and the color filter 'Blue' being referenced 41 1 , 413, 415 and 417 on figure 4). The color filter 503 is illustrated with light grey corresponding to the color filter 'Green' associated with microlenses l_₀₂, L₀₄ and L-06 (respectively referenced 402, 404 and 406 and the color filter 'Green' being referenced 412, 414 and 41 6 on figure 4). For instance, the point x₀,yo is seen by a green pixel under the angle θι through the lens L₀₂ and the same point x₀,y₀ is seen by a blue pixel under the angle θ₃ through the lens l_oi ; the point x₀,ye is seen by a blue pixel under the angle θ^ through the lens s, by a green pixel under the angle θ₃ through the lens L₀₄ and by a blue pixel under the angle θ₅ through the lens L₀₃.

Figure 5B shows in an optical phase space 51 (position and angle) which points 505 x^yo through x-i,y-|₃ (the coordinates of which being expressed in the space of the photosensor array) are seen under which angle 504 and which color RGB (either Red, Green or Blue). Figure 5B illustrates the association between the points 505 of the scene with microlenses L-n to L ₇ (corresponding respectively to the microlenses Lensn 408 to Lens ₇ 414 of figure 4) and the color filter 506 and 507. The color filter 506 is illustrated with light grey corresponding to the color filter 'Green' associated with microlenses L_{11 ;} L₁₃, L₁₅ and L₁₇ (respectively referenced 408, 410, 41 2 and 414 and the color filter 'Green' being referenced 41 8, 420, 422 and 424 on figure 4). The color filter 507 is illustrated with dark grey corresponding to the color filter 'Red' associated with microlenses L₁₂, L₁₄ and L₁₆ (respectively referenced 409, 41 1 and 41 3 and the color filter 'Red' being referenced 419, 421 and 423 on figure 4). For instance, the point x^yo is seen by a red pixel under the angle θ-ι through the lens L ₂ and the same point xi ,y₀ is seen by a green pixel under the angle θ₃ through the lens l_n ; the point Xi ,y₆ is seen by a green pixel under the angle θι through the lens Us, by a red pixel under the angle θ₃ through the lens L₁₄ and by a green pixel under the angle θ₅ through the lens L₁₃.

Figure 6 illustrates representations 61 to 66 of the different views according to the different viewing angles θι to θ₆ once de-multiplexed, according to a particular and non-limitative embodiment. De-multiplexing the different views 61 to 66 boils down to collect the pixels which belongs to the same viewing direction, i.e. to the same viewing angle θι to θ₆, which is straightforward when departing from figures 5A and 5B. The view 61 comprises all pixels as seen according to the viewing angle the view 62 comprises all pixels as seen according to the viewing angle θ₂; the view 63 comprises all pixels as seen according to the viewing angle θ₃; the view 64 comprises all pixels as seen according to the viewing angle θ₄; the view 65 comprises all pixels as seen according to the viewing angle θ₅ and the view 66 comprises all pixels as seen according to the viewing angle θ₆. As it appears clearly on figure 6, each view 61 to 66 comprises all needed color for a Bayer-like representation without missing color information, i.e. all pixels are associated with either a Red, Green or Blue color component, with no exception, and all views comprises RGB components arranged according to a BCFA (Bayer Color Filter Array) configuration : the light grey corresponds to the 'Green' color component, the average grey corresponding to the 'Blue' color component and the dark grey corresponding to the 'Red' color component as for figures 4, 5A, 5B, 7A and 7B. Be it for even or odd pixel/photosensors number associated with a microlens, all views 61 to 66 are obtained with all the colors arranged in a BCFA configuration and for the even number of pixels/photosensors, there is no view-crosstalk. Figure 8 illustrates a plenoptic camera 8, according to an exemplar embodiment of the present principles. The elements of the plenoptic camera 8, which are identical to the elements of the plenoptic camera of figure 2 have the same reference numbers. The plenoptic optical assembly of the plenoptic camera 8 is advantageously identical to the plenoptic optical assembly of figure 2. The plenoptic camera 8 thus comprises a primary lens 20, a lenslet array 21 comprising at least two microlenses 21 1 , 212, 21 n, a first color filter array 22 arranged on the lenslet array 21 , a photosensor array 13 comprising a plurality of photosensors 131 , 132, 133 to 13m. A second color filter array 81 is arranged on the photosensor array 13. The second color filter array 81 is advantageously arranged on the side of the photosensor array 1 3 facing the lenslet array 21 or facing the first color filter array 22 when the first color filter array is arranged on the side of the lenslet array 21 facing the photosensor array 13. A gap is advantageously provided between the second color filter array 81 and the lenslet array 21 or between the second color filter array 81 and the first color filter array 22 when the latter is arranged on the side of the lenslet array 21 facing the photosensor array 13. The gap may be composed of air, of an optical material having an index n (for example a glass layer) or of multiple layers comprising at least one layer air layer and at least one optical material layer. The second color filter array 81 advantageously comprises a plurality of second color filters 81 1 , 812 to 81 n, just as the first color filter array 22, for example 3 color filters, i.e. for example one red filter, one green filter and one blue filter. The second color filters are advantageously arranged according to the determined pattern used for forming the first color filter array 22 from the first color filters (i.e. red, green and blue). This means that all rays of light (or almost all of them, i.e. for example more than 95% or more than 98% of the rays of light) passing through a first color filter (red, green or blue according to the example) of the first color filter array 22 pass through a second color filter of the second color filter array 81 having the same color than the first color filter of the first color filter array 22 (i.e. respectively red, green or blue according to the example). Such a consistency between the color pattern of the first color filter array 22 and the color pattern of the second color filter array 81 is illustrated with more details on figure 9. A group of at least two photosensors of the plurality of photosensors is optically associated with each microlens of the plurality of microlenses. The second color filter array 81 comprises a plurality of second color filters, the plurality of second color filters being optically associated with the groups of photosensors according to the determined pattern in such a way that each single second color filter is optically associated with a different group of photosensors.

Figure 9 schematically illustrates the relationship between the first color filter array 22 and the second color filter array 81 , according to an exemplar embodiment of the present principles. To point out the group of photosensors 91 1 to 915 (respectively 921 to 925) of the photosensor array 13 optically associated with a given microlens 93 (respectively 94) of the microlens array 21 , rays 900 (shown in dashed line) (respectively 901 in solid line) departing from the periphery of the primary lens 20 and passing through the optical centre of the microlens 93 (respectively 94) are drawn. The area formed by these rays 900 (respectively 901 ) on the photosensor array corresponds to the photosensors optically associated with the microlens 93 (respectively 94). This group of photosensors 91 1 to 915 (respectively 921 to 925) is optically associated with a second color filter 91 (respectively 92) of the second color filter array 81 , the color of this second color filter 91 (respectively 92) being the same as the color of the first color filter associated with the microlens 93 (respectively 94). The photosensors of the group of photosensors optically associated with the given microlens 93 (respectively 94) are thus covered with a second color filter having the same color as the first color filter of the first color filter array 22 optically associated with this given microlens 93 (respectively 94). To determine the color of each color filter of the second color filter array 81 , the same reasoning is repeated for each second color filter. This means that the color pattern of the second color filter array 81 is the same as the color pattern of the first color filter array 22. Such an arrangement enables to filter the rays of light, which would pass through a microlens (and thus through the associated color filter of the first color filter array 22) and erroneously hit a photosensor of a group of photosensors not optically associated with this microlens. Indeed, if the color of the color filter associated with the exemplar microlens is red for example, only the photosensors associated with a color filter of the red color would receive a ray light being passed through a red color filter of the first color filter array 22. This enables to reduce view-crosstalk artefacts. Only two second color filters 91 , 92 of the second color filter array 81 have been illustrated on figure 9, for clarity purpose. Naturally, the number of second color filters is not limited to 2 but is advantageously equal to the number of first color filters of the first color filter array 22 (which is advantageously equal to the number of microlenses of the lenslet array 21 ) to cover the surface of the photosensor array 13.

Returning to the non-limitative example of figure 4, the determined color pattern of the first color filter array 22 is an RGGB pattern which is repeated for each block of four color filters. This same pattern is also used to form the second color filter array 81 , the photosensors covered by a given second color filter of this pattern corresponding to the photosensors which project onto the corresponding first color filter of the first color filter array 22. By taking the first color filter 41 1 ('Blue') of the first color filter array 22 as an example, the corresponding second color filter of the second color filter array would have the same color (i.e. 'Blue' according to the example of figure 4) and the group of photosensors covered by this 'Blue' filter of the second color filter would correspond to the photosensors which project onto the color filter 41 1 (illustrated by the pixels of the grid of pixels having U to L₆ rows and Ci to C₆ columns).

The form of the second color filters of the second color filter array 81 is advantageously the same as the form of the first color filters of the first color filter array (for example a square, a rectangle or a circle). According to a variant, the form of the second color filters of the second color filter array 81 may be different from the form of the first color filters of the first color filter array 22 (the first color filters of the first color filter array 22 may have a circular form (for example corresponding to the form of the microlenses) and the second color filters of the second color filter array 81 may have a square form).

Naturally, the association between the pixels of the views and the color components is not limited to a BCFA configuration as it depends on the arrangement of the color filters on the lenslet array. Color components different from RGB may be obtained when using color filters including color component different from RGB, for example Emerald, Yellow or Magenta. Naturally, the invention is not limited to the embodiments previously described.

In particular, the invention is not limited to a plenoptic optical assembly but also extends to any device integrating such a plenoptic optical assembly, for example a plenoptic camera comprising a photosensors array or a telecommunication device comprising a photosensors array.

Telecommunication devices includes, for example, smartphones, smartwatches, tablets, computers, mobile phones, portable/personal digital assistants ("PDAs"), and other devices that facilitate communication of information between end-users but also set-top-boxes.

Claims

1 . A plenoptic camera (8) comprising a primary lens (20), a lenslet array (21 ) and a photosensor array (13), the lenslet array (21 ) comprising a plurality of microlenses (21 1 , 212, 21 n), characterized in that a first color filter array (22) is arranged on said lenslet array (21 ) and a second color filter array (81 ) is arranged on said photosensor array (13).

2. The plenoptic camera (8) according to claim 1 , wherein the first color filter array (22) comprises a plurality of first color filters (221 , 222, 22n), the plurality of first color filters (221 , 222, 22n) being optically associated with the plurality of microlenses (21 1 , 21 2, 21 n) according to a determined pattern in such a way that each single first color filter (221 ; 222; 22n) is optically associated with a different microlens (21 1 ; 212; 21 n).

3. The plenoptic camera (8) according to claim 2, wherein the photosensor array (13) comprises a plurality of photosensors, a group (93, 94) of at least two photosensors (91 1 , 912, 913, 914, 915) of the plurality of photosensors being optically associated with each microlens (93) of the plurality of microlenses, and wherein the second color filter array (81 ) comprises a plurality of second color filters (91 , 92), the plurality of second color filters (91 , 92) being optically associated with the groups (93, 94) of photosensors according to said determined pattern in such a way that each single second color filter (91 ) is optically associated with a different group of photosensors (91 1 to 91 5).

4. The plenoptic camera (8) according to claim 2 or 3, wherein the determined pattern associates two green color filters (412, 418), one blue color filter (41 1 ) and one red color filter (419) with respectively each microlens of a set of four microlenses (401 , 402, 408, 409) arranged in two lines and two columns.

5. The plenoptic camera (8) according to one of claims 2 to 4, wherein the geometry of a microlens of said plurality of microlenses (21 1 , 212, 21 n) is adjusted according to the first color filter optically associated with said microlens.

6. The plenoptic camera (8) according to any one of claims 1 to 5, wherein each microlens of the plurality of microlenses (21 1 , 212, 21 n) has a plane side and a convex side, the lenslet array (21 ) having a first side, called plane side, formed with the plane sides of the plurality of microlenses and a second side, called convex side, formed with the convex sides of the plurality of microlenses, the first color filter array (22) being arranged on said first side.

7. The plenoptic camera (8) according to claim 6, wherein said second side of the lenslet array (21 ) faces the primary lens (20).

8. The plenoptic camera (8) according to any one of claims 1 to 5, wherein both first and second sides of each microlens of the plurality of microlenses (21 1 , 212, 21 n) are convex, the lenslet array (21 ) having a first side formed with the first convex sides of the plurality of microlenses and a second side formed with the second convex sides of the plurality of microlenses, the second side of the lenslet array (21 ) facing the primary lens (20), the color filter array (22) being arranged on said first side.

9. The plenoptic camera (8) according to any one of claims 1 to 8, wherein each microlens of the plurality of microlenses has a circular form.

10. The plenoptic camera (8) according to any one of claims 1 to 8, wherein each microlens of the plurality of microlenses has a rectangular form or a square form.

1 1 . The plenoptic camera (8) according to any one of claims 1 to 8, wherein each microlens of the plurality of microlenses has an hexagonal form.

12. The plenoptic camera according to any one of claims 1 to 1 1 , wherein a gap is formed between the lenslet array (21 ) and the photosensor array (13).

13. The plenoptic camera according to any one of claims 1 1 to 13, wherein the first color filter array (22) faces the second color filter array (81 ).

14. Telecommunication device comprising a plenoptic camera according to any one of claims 1 to 13.