EP2471267A1

EP2471267A1 - Method for binning of a subset of colour-pixels and system

Info

Publication number: EP2471267A1
Application number: EP09782261A
Authority: EP
Inventors: Jan Klijn; Sasa Cvetkovic
Original assignee: Robert Bosch GmbH
Current assignee: Robert Bosch GmbH
Priority date: 2009-08-27
Filing date: 2009-08-27
Publication date: 2012-07-04
Also published as: WO2011023229A1

Abstract

A method for binning of a subset 4 of colour-pixels 6 having the same colour from a digital image 1 with pixels 2 having different colours into a representative binned pixel 5 is proposed, the subset 4 of colour-pixels 6 being arranged within a rectangular field, whereby in at least one outer row 3a,b of the rectangular field at least one of the colour-pixels and whereby in at least one outer column of the rectangular field at least one of the colour-pixels is positioned, whereby the colour-pixels 6 of the rectangular field 4 are accumulated in a weighted manner, whereby at least two of said colour-pixels 6 have different weight factors.

Description

description title

Method for Binning of a subset of colour-pixels and system State of the art

The invention relates to a method for binning a subset of colour-pixels having the same colour from a digital image with pixels having different colours into a representative binned pixel. More specifically, the invention relates to a method whereby a subset of colour-pixels being arranged within a rectangular field, whereby in at least one outer row of the rectangular field at least one of the colour-pixels is positioned and whereby in at least one outer column of the rectangular field at least one of the colour-pixels is positioned. Furthermore, the invention relates to a system adapted to carry out the method.

Binning is inter alia known as a process of combining charges from adjacent pixels in a CCD during read out. In on-chip binning this process is performed prior to digitization in the on-chip circuitry of the CCD by specialized control of the serial and parallel registers. The benefits of binning are an improved signal-to- noise ratio (SNR) and the ability to increase the frame rate. On the other hand side, the spatial resolution is reduced. For example, it is known to use a traditional 2^*2 binning, whereby in monochromatic applications the 2^*2 binning summarises the values or charges of four pixels lying adjacently in a rectangular area. In the case that from a colour image only pixels of one colour shall be summarized, the rectangular area is chosen, so that in each outer row and in each outer column pixels of the selected colour are arranged. In an analogous manner 3^*3 binning may also be realised.

Disclosure of the invention According to the invention a method for binning with the features of claim 1 and a system with the features of claim 10 are disclosed. Advantageous or preferred embodiments of the invention are described by the dependent claims, the description and the figures as attached.

As the conventional binning methods often suffer in that the resulting binned pixels are not regularly or equidistantly spaced, the resulting binned image suffers from aliasing and a resolution, which is more decreased than it was expected. The advantage of the present invention is that a way is opened for a better positioning of the centres of gravity of subsets of colour-pixels, which shall be accumulated to a binned pixel.

According to the invention, a method for binning of a subset of colour-pixels having the same colour from a digital image with pixels having different colours into a representative binned pixel is proposed. The digital image comprises pixels with different colours, for example red pixels, green pixels and blue pixels. It is especially preferred, that the digital image provides more green pixels than red and blue pixels. Especially, the number of green numbers is a double of the number of the red or blue pixels.

The subset of colour-pixels having the same colour, especially red, green or blue, are arranged within a rectangular field, whereby the definition of the rectangular fields is that at least one of the colour-pixels is positioned in one outer row and at least one of the colour-pixels is positioned in one outer column. Alternatively formulated, the colour-pixels define the rectangular field. Especially only the colour-pixels contributing to the binned pixel with a weight factor unequal to 0 are used to define the rectangular field. Between the colour-pixels defining the rectangular fields pixels with other colours may be provided. It is proposed, that the colour-pixels of the rectangular field are accumulated in a weighted - or weighed - manner, whereby at least two of said colour-pixels contribute to the binned pixel with different weight factors. It is underlined and will be explained later that a colour-pixel of the rectangular field may - in some embodiments - have a weight factor of 0. With the different weight factors it is now possible to adapt the centres of gravity of the colour-pixels of one rectangular field to the position of the binned pixels thus having an beneficial effect on aliasing and resolution. In a preferred embodiment of the invention and as already indicated above, at least one of the colour-pixels of the rectangular field is defined as a suppressed colour-pixel having the weight factor of 0 as a first weight factor and though is not contributing to the value of the binned pixel. The other or a subset of the colour- pixels of the rectangular field may contribute with a weight factor uneven to 0 as a second weight factor and thus contributing to the value of the binned pixel.

Furthermore, the other colour-pixels contributing to the value of the binned pixel are still defining the rectangular field as explained above.

In a preferred embodiment of the invention the rectangular field is a 3^*3 pixel field with nine pixels, comprising four or five colour-pixels. In the 3^*3 pixel field a 2^*2 binning is performed. The four or five colour-pixels are preferably arranged in the edges and - in case of five colour-pixels - additionally in the centre of the rectangular 3^*3 pixel field. It is yet furthermore preferred, that only three of the four or five colour-pixels contribute as three selected colour-pixels to the binned pixel. The fourth and - optionally fifth colour-pixel have the weight factor of 0 and thus consequently do not contribute to the value of the binned pixel. The advantage of the exclusion of one or two of the four or five colour-pixels is that the resulting centres of gravity of the remaining three selected colour-pixels are quite near to the position of the resulting binned pixel. The resulting binned pixels have a two pixel distance to each other int he original image.

In a possible improvement the three selected colour-pixels are arranged in a triangle having one orthogonal edge. The selected colour-pixel positioned in the orthogonal edge of the triangle preferably has a higher weight factor than the other two selected colour-pixels, so that the centre of gravity is drawn towards to this colour-pixel. Already in the first step using one or more suppressed pixel, the suppressed colour-pixel have another weight factors than the three remaining selected colour-pixels. In a second step the three selected colour-pixels have different weight factors, whereby the selected colour-pixels positioned in the right-angle edge is more important and thus has a higher weight factor than the other two selected colour-pixels. An advantage of the second step is that the centres of gravity of the three selected colour-pixels are nearer to the position of the representative binned pixel. The weight factors for the four colour-pixels are preferably chosen in the following order (importance or weight factor in a decreasing order): 1 . orthogonal edge colour-pixel

2. a first selected colour-pixel

3. a second selected colour-pixel

4. a first suppressed colour-pixel (and a second suppressed colour-pixel) is for example 2:1 :1 :0(:0). The weighting factors given here are just an example, also they do not need to be integers. In some embodiments a possibility for changing from the new binning method to a standard binning method with weights of 1 :1 :1 :1 (:1 ) is provided, for example in case the ambient light is low. In this case the spatial resolution decreases, but the light sensitivity increases.

In another embodiment of the invention the rectangular field is a 5^*5 pixel field with 25 pixels, comprising nine colour-pixels. In this connection it is preferred that all colour-pixels are selected colour-pixels and contribute to the value of the representative binned pixel. But it is furthermore preferred, that the weight factors of the nine colour-pixels are decreasing in accordance with the distance to the resulting binned pixel. The highest weighting factor is dedicated to the central colour-pixel, which has the same position as the representative binned pixel. Selected colour-pixels, which are adjacent to the central colour-pixel and/or in direct contact, for example over a diagonal line, have a smaller weight factor. Selected colour-pixels being arranged in the outer row and/or outer line of the rectangular field have the smallest weight factor.

In a preferred realisation of the invention a plurality of binned pixels resulting from a plurality of the rectangular fields are arranged in an equidistant and/or regular distance. One positive consequence of the new binning method is, that it is now possible to arrange the binned pixels in equidistant distance and thus minimize effects due to aliasing etc. In order to allow the regular and/or equidistant distance of the binned pixels it is preferred, that adjacent rectangular fields are partly overlapping. Another subject-matter of the present invention is a system with the features of claim 10 being adapted to carry out the method as described before and/or and laid down in the claims before. The system is characterised in that it is embodied as an on-chip solution and/or integrated in the sensor chip, also carrying the CCD- and/or CMOS-parts. Especially the system is operable to carry out the method as described before prior digitization of the pixel values.

In an embodiment of the invention the system comprises a register with at least or exactly four horizontal and four vertical address line, whereby each weighting factor of the colour-pixel of the rectangular field are provided by the register. In fact it is possible to describe all weight factors for all positions of the digital image within the said register for the rectangular fields comprising nine or 25 pixels.

In a further improvement of the invention the weighting can be achieved by changing the effective sensitivity of the colour-pixels by using a shutter with timing per pixel. Shuttering means that the pixels are only sensitive during the shutter time and not light-sensitive outside the shutter time. It is also possible to perform multiple exposures per frame or image before saving it. The effective shutter time (and amplitude) is then the sum of the single exposure times. Further features, advantages and effects of the invention will become apparent by the detailed description of an embodiment of the invention and the figures as attached. They show:

Fig. 1 a schematic illustration of a digital image for explaining the binning method according to the state of the art;

Fig. 2 a schematic diagram of a digital image for explaining a first

embodiment of the new binning method; Fig. 3 a schematic diagram of a further digital image for explaining a second embodiment of the invention; Fig. 4 a further schematic diagram of a digital image for explaining the register operation in connection with a second embodiment of the invention;

Fig. 5 a schematic diagram of a digital image for explaining a third

embodiment of the invention;

Fig. 6 a scheme for illustrating the shutter operations in connection with the present invention.

Fig. 1 shows a schematic diagram of a digital image 1 , comprising a plurality of pixels 2, which are arranged in rows and columns. In a first row 3a green and blue pixels are alternately arranged. In a second row 3b red and green pixels 2 are also alternately arranged, so that the number of green pixels is the same as the number of the red and blue pixels together.

Binning of images is in general used to reduce the resolution of a or the digital image 1. Thereby, a plurality of pixels 2 is accumulated to a binned pixel being representative for the accumulated pixels. In Figure 1 a traditional 2^*2 binning method is shown. A rectangular field 4 comprises nine pixels 2, whereby the rectangular field 4 is concentrated in a binned pixel 5 according to the colour of the pixel in the centre of the rectangular field. For example, in the upper left corner the rectangular field 4 has a central green pixel so that the binned pixel 5 has the colour green.

For binning the rectangular field 4 (also called bin) the five green pixels as selected colour-pixels 6 are accumulated to generate the value of the binned pixel 5. The next rectangular field in the same row has a distance of three pixels, so that the binned pixels 5 of a new sampling grid is not equidistant. This results in aliasing. Furthermore the resolution of the new sampling grid is only a third of the original resolution instead of the expected half.

Fig. 2 shows a first embodiment of the invention expressed as a new binning method. Again a digital image 1 is shown comprising pixels 2 in the same order as the digital image 1 in Fig. 1. As a difference to Fig. 1 only three of the five colour-pixels are now selected colour-pixels 6 and contribute to the value of the binned pixel 5. The centre of gravity of the binned pixel 5 is near to an ideal position 7 of the binned pixel 5. This aim is achieved by using different weighting factors for the colour-pixels 6 in the rectangular field 4. The central colour-pixel is dedicated to the weight factor 0, also as the colour-pixel on the averted side of the central pixel from the binned pixel 5. Only using three selected pixels 6 allows a good position of the centre of gravity.

Fig. 3 shows an improvement of the method of binning as shown in Fig. 2 whereby the selected colour-pixels 6 are accumulated using different weighting factors. As shown the selected colour-pixel 6 in the orthogonal edge of the triangle 8 is weighted with a factor of 2, whereby the other two selected colour- pixels 6 are weighted with a factor of 1 . These weight factors allow that the centre of gravity is at the same place as the ideal position 7 of the binned pixel 5.

Furthermore, further binned pixels 5 are generated column-wise and row-wise in a distance of two pixels, so that the resulting sampling grid is now equidistant. Furthermore the resolution of the sampling grid is the half of the original resolution.

Fig. 4 is a further diagram of the embodiment of the invention as shown in Fig. 3, whereby additionally the rows and columns are tagged with register numerals between 0...3 for the columns and 4...7 for the rows. A register of 4^*4 weighting factors is enough to achieve the weighting as shown in Fig. 3. For example, at each position with the register numerals (1 , 4) the weighting factor 2 occurs, etc. So it is a further advantage of the invention that the weight factors can be presented in hardware in a very simple manner. Fig. 5 shows a third embodiment of the invention, whereby a 3^*3 binning is performed, whereby each rectangular field comprises 25 pixels. As the 3^*3 binning situation is different, the ideal centre of gravity 7 can already be achieved without weighting. Still weighting can be beneficial in order to change the resolution curve in such a way that minimal aliasing is occurring. In the example as shown in Fig. 5 the weighting factors decrease in dependence of the distance between the position of the binned pixel 5 and the selected colour-pixels 6. The weighting of the selected colour-pixels 6 can be achieved by changing the effective sensitivity of the pixels by using a shutter with timing per pixel.

Shuttering means that the pixels are only sensitive during the shutter time and not sensitive outside the shutter time. Because of the regular pattern controlling the exposure time can be achieved by a combination of four horizontal and four vertical address lines.

Figure 6 shows a scheme for illustrating the shutter operations in connection with the present invention. In the first line, frame times t are displayed, whereby the first frame time t belongs to a first frame A, the second frame time t to the next frame B etc. In the following lines, the shutter times for the pixels having specific register addresses are shown. Each pixel with the register data (6, 1 ) is exposed for a certain time indicated by the numeral 2 during the frames A, B, C. All pixels having the register data (6, 2) are exposed for a shutter time being the half of the shutter time of the pixel with the pixel data (6, 1 ). The register data (6, 0) displayed in the last line corresponds to the weighting factor 4 and thus to the double of the shutter time of pixels having the register data (6, 1 ).

In operation, the pixels 2 are shuttered according to their weight factors referenced by their position in the pixel field and defined by the entries in the 4^*4 register. Afterwards, the values of the pixels 2 of one rectangular field 4 are accumulated to one binned pixel 5.

Claims

Claims:

1. Method for binning of a subset (4) of colour-pixels (6) having the same colour from a digital image (1 ) with pixels (2) having different colours into a representative binned pixel (5), the subset (4) of colour-pixels (6) being arranged within a rectangular field, whereby in at least one outer row (3a, b) of the rectangular field at least one of the colour-pixels and whereby in at least one outer column of the rectangular field at least one of the colour-pixels is positioned, characterised in that the colour-pixels (6) of the rectangular field (4) are accumulated in a weighted manner, whereby at least two of said colour-pixels (6) have different weight factors.

2. Method according to claim 1 , characterised in that at least one of the colour- pixels (6) of the rectangular field is a suppressed colour-pixel having the weight factor of zero, not contributing to the value of the binned pixel (5).

3. Method according to claim 1 or 2, characterised in that the rectangular field is a 3 x 3 pixel field with nine pixels (2), comprising four or five colour-pixels (6).

4. Method according to claim 3, characterised in that only three selected colour- pixels (6) contribute to the binned pixel (5), the three selected colour-pixels (6) being arranged in a triangle.

5. Method according to claim 4, characterised in that the selected colour-pixel (6) being positioned in the rectangular edge of the triangle has a higher weight factor than the other two selected colour-pixels (6).

6. Method according to claim 1 or 2, characterised in that the rectangular field is a 5 x 5 pixel field (4) with 25 pixels (2), comprising nine colour-pixels (6).

7. Method according to claim 6, characterised in that the weighting factors of the nine colour-pixels are decreasing in accordance with the distance to the position of the resulting binned pixel (5).

8. Method according to one of the preceding claims, characterised in that a plurality of binned pixels (5) resulting from a plurality of the rectangular fields (4) are arranged in a regular sampling grid.

9. Method according to one of the preceding claims, characterised in that adjacent rectangular fields (4) are partly overlapping.

10. System adapted to carry out the method according to one of the preceding claims, characterised in that the system is embodied as an on-chip system.

1 1 . System according to claim 10, characterised by a register comprising four horizontal and four vertical address lines, whereby each weight factor of the colour-pixels (2) of the rectangular field (4) are provided by the register.

12. System according to claim 10 or 1 1 , characterised in that the weight factors are embodied as shutter times of the respective colour-pixels (6).