WO2022063407A1

WO2022063407A1 - Compressed image sensor module obtained using pixel rearrangement

Info

Publication number: WO2022063407A1
Application number: PCT/EP2020/076827
Authority: WO
Inventors: Mikko Muukki
Original assignee: Huawei Technologies Co., Ltd.
Priority date: 2020-09-25
Filing date: 2020-09-25
Publication date: 2022-03-31
Also published as: CN116325778A

Abstract

In some examples, a two-dimensional filter array arrangement for use with an imaging device is configured to generate multi-band image data corresponding to an object or scene that is to be imaged. At least a portion of the filter array arrangement comprises an M x N array of spectral filter elements configured to define a set of unit cells, each unit cell comprising a P x Q sub-array of spectral filter elements where P < M and Q < N, the spectral filter elements of a unit cell restricting imaging to multiple selected regions of the electromagnetic spectrum, wherein a first unit cell comprises a set, A, of spectral filter elements of the filter array arrangement and a second unit cell that is contiguous with the first unit cell comprises a set, B, of spectral filter elements, wherein the first and second unit cells are arranged relative to each other so as to comprise a first set, X, of spectral filter elements, where X = { x | x ∈ (A ∩ B) }, and a second set, Y, of spectral filter elements, where Y = { y | y ∈ (A △ B) } and, wherein the first set, X, of spectral filter elements forms a repeating pattern for the filter array arrangement.

Description

COMPRESSED IMAGE SENSOR MODULE OBTAINED USING PIXEL REARRANGEMENT

TECHNICAL FIELD

Aspects relate, in general, to sensor modules, and more particularly, although not exclusively, to filter arrays for use in sensors of sensor modules.

BACKGROUND

Image sensors used in imaging devices, such as those implemented in user equipment for example, typically utilise multiple colour channels forming a colour filter array (CFA). A CFA can comprise a set of colour filters over a grid of photodetectors, thereby forming an image sensor for the device. A common pattern for the colour filters of a CFA for visible light imaging is a Bayer mosaic pattern, which comprises a 2:1 :1 ratio of green, red and blue filter elements.

Spectral sensors can use many other channels aside from those mentioned above in order to image wavelength bands of the electromagnetic spectrum which may not be visible to the naked eye. In the limit, a spectral sensor can use as many filter channels, covering respective different wavelength bands, as there are sensor pixels, but typically there will be some sensor pixels that use the same ‘colour’ filter as other pixels, as there are in the Bayer pattern for example.

In higher resolution spectral sensors, groups of filter arrangements can be repeated over the area of a spectral image sensor. For example, a 4x4 arrangement of filters forming a mosaic covering 16 different wavelength bands can be repeated in order to provide a high-resolution spectral sensor. Such image sensors can be used in various applications. For example, since spectral sensors can produce more information than normal image sensors (e.g. those with a Bayer CFA pattern) by being able to measure multiple smaller wavelength ranges, the extra information may be used to, for example, improve colours of images/videos, improve image segmentation, detect different materials or material properties/conditions and so on.

Although the use of such higher resolution spectral image sensors is attractive, the increase in the number of wavelength bands from which the sensor is able to capture information maps to an increase in the size of the image sensor.

SUMMARY

According to a first aspect, there is provided a two-dimensional filter array arrangement for use with an imaging device, the filter array arrangement configured to generate multi-band image data corresponding to an object or scene that is to be imaged, at least a portion of the filter array arrangement comprising an M x N array of spectral filter elements configured to define a set of unit cells, each unit cell comprising a P x Q sub-array of spectral filter elements where P < M and Q < N, the spectral filter elements of a unit cell restricting imaging to multiple selected regions of the electromagnetic spectrum, wherein a first unit cell comprises a set, A, of spectral filter elements of the filter array arrangement and a second unit cell that is contiguous with the first unit cell comprises a set, B, of spectral filter elements, wherein the first and second unit cells are arranged relative to each other so as to comprise a first set, X, of spectral filter elements, where X = { x | x e (A A B) }, and a second set, Y, of spectral filter elements, where Y = { y | y e (A A B) } and, wherein the first set, X, of spectral filter elements forms a repeating pattern for the filter array arrangement.

The filter array arrangement is particular useful for spectral imaging purposes since it enables multiple bands of the electromagnetic spectrum to be imaged using a smaller image sensor arrangement than would otherwise typically be the case. The relatively smaller arrangement, leading to commensurate advantageous reductions in sensor module size for example, is enabled by the manner in which filter elements can be arranged in order to provide unit cells that effectively share certain filter elements, as opposed to more traditional arrangements in which a mosaic of filter elements is repeated over the an image sensor area. Such arrangements generally result in certain channels being overrepresented, or at least being more prevalent in the array. In the present arrangement, such overrepresentation is leveraged in order to reduce the size of a unit cell comprising a number of different filter elements, but which includes a subset of filter elements that may be shared with a proximate unit cell.

In the two-dimensional filter array arrangement, the first set of spectral filter elements can form a first repeating pattern in a first direction of the filter array arrangement. The first set of the spectral filter elements can form a second repeating pattern in a second direction of the filter array arrangement that is orthogonal to the first direction. That is, unit cells may overlap in one or more dimensions of the arrangement.

In an example, the second set of spectral filter elements can form a third repeating pattern in first direction of the filter array arrangement. The second set of spectral filter elements can form a fourth repeating pattern in first direction of the filter array arrangement. Thus, there may be some unit cells in which there is a mirroring of certain spectral filter elements. For example, certain shared or unshared filter elements may be mirrored in, e.g., adjacent, proximate or contiguous unit cells. In an implementation of the first aspect, the relative complement of A in B, B \ A = 0. That is, for a pair of proximate or contiguous unit cells, there may be no spectral filter elements in the set A that are not also in the set B.

In an implementation of the first aspect, n(A \ B) - n(B \ A) = n(0). That is, the number of spectral filter elements, n(.), resulting from the difference between the relative complements may be null.

In some implementations of the first aspect, the relative complement of A in B, B \ A may not be equal to the relative complement of B in A, A \ B.

The first and second unit cells can share a vertical boundary. The first and second unit cells can share a horizontal boundary. So, for example, a pair of unit cells may have a vertical boundary that overlaps inasmuch as there are number of shared spectral filter elements that define a vertical boundary for the unit cells. This is similarly the case for a horizontal boundary.

A vertical and/or horizontal boundary between the unit cells can comprise at least one spectral filter element of first set of spectral filter elements.

In an implementation of the first aspect, the arrangement of spectral filter elements of the second set of spectral filter elements within the first and second unit cells can be symmetric with respect to a line of symmetry defined by a shared boundary between the first and second unit cells. The shared boundary can be, e.g., a horizontal or vertical boundary.

The arrangement of spectral filter elements of the second set of spectral filter elements within the first and second unit cells can be the same. In another example, the arrangement of spectral filter elements of the second set of spectral filter elements within the first and second unit cells can be different. For example, there may or may not be a mirroring of one or more spectral filter elements, or some other positioning that results in a change in the position of certain spectral filter elements in contiguous unit cells.

One or more spectral filter elements of the second set of spectral filter elements can be substituted with an alternate spectral filter element configured to restrict imaging to a region of the electromagnetic spectrum not covered by other spectral filter elements of the second set of spectral filter elements. The alternate spectral filter element can be provided in the first and/or the second unit cell. That is, a spectral filter element relating to a given channel can be substituted with another element that is sensitive to a different channel. In one example, a certain number of filter elements for a channel which may comprise a relatively higher number of filter elements in an arrangement compared to others (c.f. the green channel in the typical Bayer mosaic pattern), can therefore be replaced, thereby enabling a larger number of channels to be ‘imaged’. The replacement elements may be, broadly speaking, underrepresented compared to the number of elements for other channels in the arrangement (particularly those in which no replacements have occurred), but this may be perfectly acceptable in some circumstances.

According to a second aspect, there is provided an imaging device configured to generate multi-band image data representing a spectral image cube for an object or scene, the imaging device comprising, an imaging sensor comprising a two-dimensional sensor array, and a two- dimensional filter array arrangement as claimed in any preceding claim, wherein the two- dimensional filter array arrangement is positioned before, with respect to an optical path of the imaging device, or embedded in the sensor array. The two-dimensional sensor array can comprise a grid of photodetectors defining an array of pixels for the imaging device, wherein each of the spectral filter elements of the two-dimensional filter array arrangement is so positioned as to coincide with a pixel. The imaging device can further comprise a processor configured to generate multiple spectral band images of the object or scene, each spectral band image corresponding to image data generated from pixels restricted to imaging regions of the electromagnetic spectrum defined according to the spectral filter elements.

According to a third aspect, there is provided a method for providing a filter array arrangement, the method comprising placing a first unit cell relative to a second unit cell, the said first and second unit cells as provided according to a first aspect.

According to a fourth aspect, there is provided user equipment comprising an imaging device as provided according to the second aspect. In an implementation of the fourth aspect, the user equipment can be in the form of a smart device, such as a smart mobile telephone for example, that is configured to use data generated by the imaging device.

BRIEF DESCRIPTION OF THE DRAWINGS

For a more complete understanding of the present disclosure, reference is now made, by way of example only, to the following descriptions taken in conjunction with the accompanying drawings, in which:

Figure 1 is a schematic representation of a portion of a filter array arrangement; Figure 2 is a schematic representation of a portion of a filter array arrangement according to an example;

Figures 3a-d are schematic representations of a portion of a filter array arrangement according to an example;

Figure 4 is a schematic representation of a portion of a filter array arrangement according to an example;

Figure 5 is a schematic representation of a portion of a filter array arrangement according to an example;

Figure 6 is a schematic representation of a portion of a filter array arrangement according to an example;

Figure 7 is a schematic representation of a portion of a filter array arrangement according to an example;

Figures 8a-e are schematic representations of portions of a filter array arrangement according to examples; and

Figure 9 is a schematic representation of an imaging device according to an example.

DESCRIPTION

Example embodiments are described below in sufficient detail to enable those of ordinary skill in the art to embody and implement the systems and processes herein described. It is important to understand that embodiments can be provided in many alternate forms and should not be construed as limited to the examples set forth herein. Accordingly, while embodiments can be modified in various ways and take on various alternative forms, specific embodiments thereof are shown in the drawings and described in detail below as examples. There is no intent to limit to the particular forms disclosed. On the contrary, all modifications, equivalents, and alternatives falling within the scope of the appended claims should be included. Elements of the example embodiments are consistently denoted by the same reference numerals throughout the drawings and detailed description where appropriate. The terminology used herein to describe embodiments is not intended to limit the scope. The articles “a,” “an,” and “the” are singular in that they have a single referent, however the use of the singular form in the present document should not preclude the presence of more than one referent. In other words, elements referred to in the singular can number one or more, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises,” “comprising,” “includes,” and/or “including,” when used herein, specify the presence of stated features, items, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, items, steps, operations, elements, components, and/or groups thereof.

Unless otherwise defined, all terms (including technical and scientific terms) used herein are to be interpreted as is customary in the art. It will be further understood that terms in common usage should also be interpreted as is customary in the relevant art and not in an idealized or overly formal sense unless expressly so defined herein.

A spectral sensor can be configured to generate data covering multiple wavelength bands of the electromagnetic spectrum. Each band (or channel) typically covers a narrow band, which may have a full width half maximum (FWHM) which is, e.g., 10nm, 20nm or 50nm and channels may have separate widths. A spectral sensor may cover a wavelength range from 400nm to 1000nm, or even up to several urn.

Spectral sensors with multiple channels may be used to perform multi spectral imaging or hyper spectral imaging, the main difference for multi/hyper spectral imaging being the number of channels used/supported (with hyper spectral imaging typically using a number of channels that can be an order of magnitude larger than the number used for multi spectral imaging). A sensor with multiple channels may include channels with peak wavelengths (e.g. 400nm, 420nm, ... , 600nm, 620nm, 630nm, ... , 850nm, 940nm and so on), which can be chosen during a design phase. For example, the channels may be chosen such that they are equally spread over a given wavelength range or chosen to support a specific use case. For example, a spectral sensor to support colour processing may have many channels that are more densely packed in the visible wavelength area of the electromagnetic spectrum compared with, e.g., the near infra-red area.

In the context of, for example, user equipment such as smart phones, the benefits that high resolution spectral image sensors can provide may be offset by a commensurate and undesirable increase in size and weight stemming from their implementation. That is, in order to accommodate a larger sensor that can generate image data over a relatively larger number of channels compared with, e.g., a Bayer mosaic sensor, user equipment may need to be increased in size and have an increased weight.

According to an example, a multichannel filter array arrangement is provided that enables a small sensor/module size to be used, while at the same time substantially maintaining sensing quality. The arrangement comprises multiple spectral filter elements that are configured to restrict imaging to selected regions of the electromagnetic spectrum. Each spectral filter element can form a pixel. That is, each spectral filter element can be positioned over a detector of a grid of detectors. In some examples, identical pixels proximate to each other can be removed, which enables a smaller sensor size in one or more dimensions. This has a knock- on effect of enabling a smaller optics module height (since optics module height is proportional to the diagonal length of the active area of a sensor).

Figure 1 is a schematic representation of a portion of a filter array arrangement. The arrangement shown in figure 1 comprises a 2x3 CFA pattern, in which eight unit cells 101 (or ‘super pixels’, three of which are circled) are provided, each of which itself comprises six sensor pixels 103 (numbered 1 to 6 as shown). The term pixel as used herein refers to a filter element of a filter array arrangement for use with an imaging device, and the two are used interchangeably.

Accordingly, each sensor pixel represents a different spectral filter element. As such, the arrangement of figure 1 is capable of imaging up to six different channels, each of which may be selected to generate data from a specific band of the electromagnetic spectrum. Each unit cell 101 is repeated as shown in order to form the filter array arrangement in which a total of 48 sensor pixels are provided.

In order to arrive at a filter array arrangement according to an example, the pattern of sensor pixels of figure 1 can be rearranged, as will be described below.

Figure 2 is a schematic representation of a portion of a filter array arrangement according to an example. The arrangement of figure 2 shows a rearrangement of the pixel order of the sensor pixels 103 of figure 1 . In the rearrangement, the channel order of unit cells is changed such that channels with same channel response are proximate each other. Thus, in the first row 201 of the arrangement, the pixel order is 1 ,2, 3, 3, 2,1 (instead of 1 ,2, 3, 1 ,2, 3 as in figure 1). As with figure 1 , three unit cells are circled. Figure 3a is a schematic representation of a portion of a filter array arrangement according to an example. In the arrangement of figure 3a duplicated pixels are removed. That is, instead of just changing the channel order (as depicted in figure 2), overlapping unit cells are generated by removing duplicated pixels I compressing the pattern. This can be performed in one or more directions. With reference to figures 2 for example, unit cells 203 and 205 are compressed by removing pixels 207, 209 (or pixels 211 , 213) to form a pair of unit cells 301 , 303 that share pixels (i.e. , pixels 3 and 6 in figure 3). This has the advantageous effect of reducing the size of a sensor that uses the arrangement. As with figure 2, three unit cells are circled. The right-hand side of figure 3a depicts the same arrangement as the left-hand side, but without the circles present, in order to enable the numbers to be seen more clearly. This is similarly the case with figures 4 and 5.

Figure 3b is a schematic representation of a portion of a filter array arrangement according to an example. The example of figure 3b builds on that depicted in figure 3a in order to show how some channels may be overrepresented compared to others. In the example of figure 3b, channels 2 and 5 are more frequent than other channels. This will be expanded upon further below.

Figure 4 is a schematic representation of a portion of a filter array arrangement according to an example. In the arrangement of figure 4 further duplicated pixels are removed. That is, the pattern is compressed in another direction (orthogonal to the direction described with reference to figure 3). In the example of figure 4 and referring also to figure 3, it can be seen that duplicated pixels in the rows 305 are removed to compress those two rows down to one (401). Again, three unit cells are circled, showing the overlapping/shared pixels.

Figure 5 is a schematic representation of a portion of a filter array arrangement according to an example. In the arrangement of figure 5, which is an implementation derived from the arrangement shown in figure 3, compression is performed in one direction and extra pixels 501 for one or more other channels are added (i.e. two additional channels are catered for by addition of a pixel for channel 7 and one for channel 8). Note that the number of pixels for each of these channels is less than the number for the channels 1-6 as, in some circumstances, it may be sufficient to render those channels with a relatively lower resolution.

In some examples as described above some channels will occur more frequently than others. For example, with reference to figures 2, 3a and 3b, channels 2 and 5 are more frequent than other channels. If additional channels are to be included it is possible to substitute some of the filter elements that would be channel 2 or channel 5 with the alternate filter elements. Furthermore, when duplicated lines or columns have been removed only from one direction, the sensor may support binning of adjacent channels which have the same colour.

By replacing some filter elements of channels that are comparatively more frequent than others with additional channels (implemented by way of filter elements for the extra channels of interest), the number of channels that a sensor can be used to generate information for can be increased. For example, if some filter elements of channels 2 and 5 as depicted in figure 2 or 3b are replaced with channels ‘a’ and/or ‘b’, seven/eight channels can be used, although it is to be noted that since only some of the channels are substituted the ‘a’ and/or ‘b’ channels will provide lower resolution that rest of the channels. This may be acceptable in some application/use cases. Furthermore, if the ‘a’ and/or ‘b’ channels are located in different lines and columns of the array arrangement, for example as shown in figure 3c and figure 3d, they will provide sampling of those channels over a larger area of the array, which may be beneficial since it will provide a more accurate measure of for those channels compared to the case in which the corresponding filter elements are clumped together in a relatively small region of the array. In alternative form, ‘a’ and/or ‘b’ could be located in the same lines, for example line 2 instead of 2 and 4, and for example line 6 instead of 6 and 8.

Consider another example, represented in terms of a 4x4 CFA pattern, in which each unit cell consists of 16 sensor pixels. That is, a unit cell comprises 16 sensor pixels covering up to 16 channels.

Figure 6 is a schematic representation of a portion of a filter array arrangement according to an example. Figure 6 shows sensor pixels numbered from 1 to 16 forming a unit cell 601 , which is repeated to form the filter array arrangement 600 (this comprising a total of 256 sensor pixels). Four unit cells are circled by way of example.

Similarly to the example described above with reference to figures 1 to 5, the pixel order can be rearranged. According to an example, the channel order of filter elements in neighbouring unit cells is changed. After rearrangement, channels with same channel response are proximate each other. So, with reference to figure 6 for example, channel orders are rearranged such that in the first row 603 the order is 1 ,2, 3, 4, 4, 3, 2,1 (instead of 1 ,2, 3, 4, 1 ,2, 3, 4 as in figure 5). Note that it is shown in figure 6 that all lines are similarly mirrored, but it is also shown that pixels 6 and 7 are mirrored. In an alternative, 6 and 7 are not mirrored (i.e. all except channels 6,7,10,11 are mirrored). Following rearrangement, some duplicated pixels are removed. This is shown in figure 7, which is a schematic representation of a portion of a filter array arrangement according to an example.

In figure 7 the number of highlighted pixels is reduced to up to % of its original value in columns and lines. In this way overlapping unit cells can be provided in which some channels are shared between unit cells, as depicted by the overlapping circles. Sharing in this way reduces the required height and width of a sensor. In the example of figure 7, overlapping occurs happens in all pixels except 6, 7, 10 and 11 . Those pixels may be mirrored, as shown, or not. Mirroring them ensures that surrounding pixels are more similar than without mirroring, which is preferred.

Accordingly, a two-dimensional filter array arrangement for use with an imaging device is provided, in which the filter array arrangement is configured to generate multi-band image data corresponding to an object or scene that is to be imaged. The imaging device can therefore generate data representing an image cube of the object or scene. However, as a result of a rearrangement and sharing, compared to a typical spectral sensor array, the sensor size can be made significantly smaller, whilst maintaining the ability to generate high resolution data over the desired channels, thus enabling use in smaller form factor user equipment such as smart phones.

According to an example, the filter array arrangement comprises an M x N array of spectral filter elements. The filter elements are selected such that their spectral response enables selected wavelengths of the electromagnetic spectrum to pass to detectors aligned with respect to the filter elements in order to generate data representing multiple channels. These channels may include red, green and blue, for example, but may also comprise other channels, which may be outside of the scope of human vision, such as channels selected within the infra-red and ultraviolet regions of the electromagnetic spectrum.

The filter elements are configured to define a set of unit cells. For example, as noted above with reference to figures 2 to 7, some unit cells are circled. That is, a unit cell comprises a mosaic of filter elements, which mosaic can be repeated in order to form the filter array arrangement. Thus, it follows that, for a filter array arrangement comprising an M x N array of spectral filter elements, each unit cell comprises a P x Q sub-array of spectral filter elements where P < M and Q < N. The spectral filter elements of a unit cell restrict imaging to multiple selected regions of the electromagnetic spectrum. According to an example, a first unit cell comprises a set, A, of spectral filter elements of the filter array arrangement and a second unit cell that is contiguous with or proximate the first unit cell comprises a set, B, of spectral filter elements. The first and second unit cells are arranged relative to each other so as to comprise a first set, X, of spectral filter elements, where X = { x | x e (A A B) }, and a second set, Y, of spectral filter elements, where Y = { y | y e (A A B) } and, wherein the first set, X, of spectral filter elements forms a repeating pattern for the filter array arrangement. Put another way, with reference to figure 7 by way of example, a first unit cell 701 and a second unit cell 703 are contiguous. The first set, X, of filter elements of the unit cells 701 , 703 comprises filter elements of A that are also in B (or equivalently, all elements of B that are also in A). Thus, for the unit cells depicted in figure 7, X = {1-5, 8, 9, 12-16}, whereas Y = {6, 7, 10, 11}.

For edge unit cells 705, 707, X = {1-4, 8,12-16}, and Y = {5-7, 9-11}. The disparity between the constituent elements of X for unit cells 701 , 703; 705, 707 stems merely from their position within the array. That is, for example, unit cells 705 and 707 are arranged on the outer edge of the array and so there is a reduced degree to which elements are shared as a result of fewer surrounding unit cells.

As can be seen from, for example figure 7, a repeating pattern of unit cells is provided in order to define the overall array arrangement. The unit cells may be repeated in a vertical and/or a horizontal direction.

According to an example, a filter array arrangement, comprising lines/rows and columns of filter elements is thus configured such that at least some image lines have a filter element that has, on its left and right hand sides, filter elements at least one other channel response (e.g., with reference to figure 3a, filter element 1 and filter element 2 (and 3 and 2s) in line 1 (and lines 4, 5 & 8); 6 and 5s (and 4 and 5s) in line 3 (and, e.g., line 7). That certain pixel is shared by more than one unit cell.

This may be generalized such that at least one column or line of the arrangement has a certain filter element which has at least corresponding left and right or top and bottom filter elements with another channel response; i.e. that certain filter element is shared by more than one unit cell. This may not hold if certain channels are substituted in order to increase the range over which the arrangement can be used.

Figures 8a-e are schematic representations of portions of a filter array arrangement according to examples. In the examples of figures 8a to e, which are representations built upon a 4x4 mosaic of channels (i.e., where the filter array pattern size is larger than 3x3 e.g. 3x4 or 4x4 or larger), it is possible to generate different variants that can be provided by reordering the remaining filter elements. For example, different ordering may be generated to prioritize different items: e.g., to allow more similar surrounding (which may be easier for filter manufacturing) or allow more uniform sampling i.e. similar distance between filter elements of the same channel. In an example, in reordering, the mirror is not performed in some groups of pixels.

With reference to the variants depicted in figures 8a and 8b, it can be seen that at least some image lines have a certain filter element, which has on its left and right hand sides a filter element with another (different) channel response (e.g. filter element 4 and filter element 3, e.g. filter element 1 and filter element 2 in line 1 (and lines 7 & 13); 16 and 15s, 13 and 14s in line 4 (and line 10). That certain filter element is shared by more than one unit cell.

At least some columns have a certain filter element, which has a filter element with another (different) channel response above and below it (e.g. filter element 13 and filter element 9, e.g. filter element 1 and filter element 5 in first column (and columns 7 & 13); 4 and 8s, 16 and 12s in fourth column (and column 10) in figure 8a-8d). That certain filter element is shared by more than one unit cell. This can, in an example, be generalized such that at least one column or line has a certain filter element which has at least another (different) channel response to its left and right or top and bottom; i.e. that certain filter element is shared by more than one unit cell. Accordingly, the “line rule” (as described with reference to figures 8a and 8b) is valid for 8a and 8b (and not that shown in figures 8c and 8d for example), whereas the “column rule” is valid for all variants depicted in figures 8a-d.

Put another way, an image array comprises sub-patterns forming unit cells that overlap with each other at least in one of a horizontal or vertical direction, in which every 2nd unit cell in a horizontal direction is mirrored, and every 2nd unit cell in a vertical direction (of the array arrangement) is mirrored. Each mirrored unit cell is mirrored so that at least 4 corner filter elements are mirrored (e.g., 1 ,4 13,16 in above examples of figures 8a-e and 1 , 3, 4, 6 in above examples of figures 2-5).

Figure 9 is a schematic representation of an imaging device according to an example. The imaging device 900 is configured to generate multi-band image data representing a spectral image cube for an object or scene to be imaged. The imaging device 900 comprises an imaging sensor 901 comprising a two-dimensional sensor array 903. The sensor array 903 comprises a grid of detector elements 905. Each detector element can generate an electrical signal in response to exposure to electromagnetic radiation, e.g. light, UV and so on. In an example, some elements 905 may be sensitive only to particular regions of the electromagnetic spectrum. A two-dimensional filter array arrangement 907 is positioned before, with respect to an optical path 909 of the imaging device, the sensor array. Alternatively, the filter array arrangement 907 may be embedded within the sensor array 903 for example by manufacturing the filter on top of the detector element i.e. pixel, to be part of the sensor array. In the example of figure 9, the two-dimensional filter array arrangement 907 is a two- dimensional filter array arrangement as described above with respect to any one of figure 2 to 8a-e. Each of the spectral filter elements 907 of the two-dimensional filter array arrangement is so positioned as to coincide with a pixel for the sensor array 903. The imaging device comprises a processor 911 configured to generate multiple spectral band images of an object or scene, each spectral band image corresponding to image data generated from pixels restricted to imaging regions of the electromagnetic spectrum defined according to the spectral filter elements 907. The imaging device 900 may form part of user equipment 913.

Claims

1. A two-dimensional filter array arrangement for use with an imaging device (900), the filter array arrangement (907) configured to generate multi-band image data corresponding to an object or scene that is to be imaged, at least a portion of the filter array arrangement comprising: an M x N array of spectral filter elements (600) configured to define a set of unit cells (301 ; 303), each unit cell comprising a P x Q sub-array of spectral filter elements where P < M and Q < N, the spectral filter elements of a unit cell restricting imaging to multiple selected regions of the electromagnetic spectrum, wherein a first unit cell (301) comprises a set, A, of spectral filter elements of the filter array arrangement and a second unit cell (303) that is contiguous with the first unit cell comprises a set, B, of spectral filter elements, wherein the first and second unit cells are arranged relative to each other so as to comprise a first set, X, of spectral filter elements, where X = { x | x e (A A B) }, and a second set, Y, of spectral filter elements, where Y = { y | y e (A A B) } and, wherein the first set, X, of spectral filter elements forms a repeating pattern for the filter array arrangement (907).

2. The two-dimensional filter array as claimed in claim 1 , wherein the first set of spectral filter elements forms a first repeating pattern in a first direction of the filter array arrangement.

3. The two-dimensional filter array as claimed in claim 2, wherein the first set of the spectral filter elements forms a second repeating pattern in a second direction of the filter array arrangement that is orthogonal to the first direction.

4. The two-dimensional filter array as claimed in claim 2 or 3, wherein the second set of spectral filter elements forms a third repeating pattern in first direction of the filter array arrangement.

5. The two-dimensional filter array as claimed in claim 2 or 3, wherein the second set of spectral filter elements forms a fourth repeating pattern in first direction of the filter array arrangement.

6. The two-dimensional filter array as claimed in any preceding claim, wherein the relative complement of A in B, B \ A = 0.

7. The two-dimensional filter array as claimed in any preceding claim, wherein n(A \ B) - n(B \A) = n(0).

8. The two-dimensional filter array as claimed in any of claims 1 to 5, wherein the relative complement of A in B, B \ A is not equal to the relative complement of B in A, A \ B.

9. The two-dimensional filter array as claimed in any preceding claim, wherein the first and second unit cells share a vertical boundary.

10. The two-dimensional filter array as claimed in any of claims 1 to 8, wherein the first and second unit cells share a horizontal boundary.

11 . The two-dimensional filter array as claimed in claim 9 or 10, wherein a vertical and/or horizontal boundary between the unit cells comprises at least one spectral filter element of first set of spectral filter elements.

12. The two-dimensional filter array as claimed in any preceding claim, wherein the arrangement of spectral filter elements of the second set of spectral filter elements within the first and second unit cells is symmetric with respect to a line of symmetry defined by a shared boundary between the first and second unit cells.

13. The two-dimensional filter array as claimed in any of claims 1 to 11 , wherein the arrangement of spectral filter elements of the second set of spectral filter elements within the first and second unit cells is the same.

14. The two-dimensional filter array as claimed in any of claims 1 to 11 , wherein the arrangement of spectral filter elements of the second set of spectral filter elements within the first and second unit cells is different.

15. The two-dimensional filter array as claimed in any preceding claim, wherein one or more spectral filter elements of the second set of spectral filter elements are substituted with an alternate spectral filter element configured to restrict imaging to a region of the electromagnetic spectrum not covered by other spectral filter elements of the second set of spectral filter elements.

16. The two-dimensional filter array as claimed in claim 15, wherein the alternate spectral filter element is provided in the first and/or the second unit cell. 16

17. An imaging device (900) configured to generate multi-band image data representing a spectral image cube for an object or scene, the imaging device comprising: an imaging sensor (901) comprising a two-dimensional sensor array (903); and a two-dimensional filter array arrangement (907) as claimed in any preceding claim, wherein the two-dimensional filter array arrangement is positioned before, with respect to an optical path of the imaging device or embedded in the sensor array (903).

18. The imaging device as claimed in claim 17, wherein the two-dimensional sensor array (903) comprises a grid of photodetectors (905) defining an array of pixels for the imaging device, wherein each of the spectral filter elements of the two-dimensional filter array arrangement (907) is so positioned as to coincide with a pixel.

19. The imaging device as claimed in claim 17 or 18, further comprising a processor (911) configured to: generate multiple spectral band images of the object or scene, each spectral band image corresponding to image data generated from pixels restricted to imaging regions of the electromagnetic spectrum defined according to the spectral filter elements.

20. A method for providing a filter array arrangement, the method comprising: placing a first unit cell relative to a second unit cell as provided in any of claims 1 to 16.

21. User equipment (913) comprising an imaging device (900) as claimed in any of claims

17 to 19.