WO2009090217A1 - Camera device for image acquisition of flat or nearly flat objects - Google Patents

Camera device for image acquisition of flat or nearly flat objects Download PDF

Info

Publication number
WO2009090217A1
WO2009090217A1 PCT/EP2009/050430 EP2009050430W WO2009090217A1 WO 2009090217 A1 WO2009090217 A1 WO 2009090217A1 EP 2009050430 W EP2009050430 W EP 2009050430W WO 2009090217 A1 WO2009090217 A1 WO 2009090217A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
image sensor
camera device
transmitting element
mask
Prior art date
Application number
PCT/EP2009/050430
Other languages
French (fr)
Inventor
Lorenz Mueller
Roger Cattin
Marcel Jacomet
Original Assignee
Axsionics Ag
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Axsionics Ag filed Critical Axsionics Ag
Publication of WO2009090217A1 publication Critical patent/WO2009090217A1/en

Links

Classifications

    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/42Diffraction optics, i.e. systems including a diffractive element being designed for providing a diffractive effect
    • G02B27/46Systems using spatial filters
    • GPHYSICS
    • G03PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03BAPPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B19/00Cameras
    • G03B19/02Still-picture cameras
    • G03B19/16Pin-hole cameras
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/10Image acquisition
    • G06V10/12Details of acquisition arrangements; Constructional details thereof
    • G06V10/14Optical characteristics of the device performing the acquisition or on the illumination arrangements
    • G06V10/147Details of sensors, e.g. sensor lenses
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18Eye characteristics, e.g. of the iris
    • G06V40/19Sensors therefor
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/50Constructional details
    • H04N23/55Optical parts specially adapted for electronic image sensors; Mounting thereof
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00Circuitry of solid-state image sensors [SSIS]; Control thereof
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B2207/00Coding scheme for general features or characteristics of optical elements and systems of subclass G02B, but not including elements and systems which would be classified in G02B6/00 and subgroups
    • G02B2207/129Coded aperture imaging

Definitions

  • Camera device for image acquisition of flat or nearly flat objects
  • the invention relates to a camera device having a low height for image acquisition of flat or nearly flat objects, wherein the camera is provided for especially but not exclusively objects in the form of smart cards, e.g. credit cards.
  • Camera devices having a low height are well known from prior art. For example it is well known to provide a mobile phone with a camera. Since the size of a mobile phone is crucial it is important to use cameras with optical elements having a small scale. However the height of such cameras lies still in the range of several millimetres or even centimetres.
  • DE 199 17 890 discloses an image capture system.
  • Said image capture system comprises a plurality of micro lenses that are arranged side by side. Furthermore the system comprises an image detector behind said micro lenses. Since there is a plurality of micro lenses arranged, said lenses provide a plurality of images onto a corresponding image sensor, i.e. every optical channel corresponds to one cell of the detector.
  • EP 0 809 124 shows a short focal length image sensor having a lenslet array being positioned over a photosensitive imaging ar ⁇ ray having a number of photosensitive sites corresponding in number to at least the number of lenslets. Thereby each lenslet is allocated to a photosensitive site.
  • the term photosensitive sites is understood as to be one pixel of an image sensor. In an RGB-environment the photosensitive site are three pixels namely a red, green and blue pixel.
  • the lenslets have to be arranged with respect to the photosensitive sites such that a one lenslet is allocated to the respective site which is a single pixel.
  • An object of the present invention is to provide a device, which does not have the disadvantages according to devices of prior art.
  • a camera device shall have a small height with least possible optical elements as well as sufficient image quality.
  • a further object of the present invention is to provide a camera device to acquire an image of a human eye for identification purposes.
  • a camera device for image acquisition.
  • Said device comprises an image sensor that is arranged behind a transmitting element for transmitting image information to said image sensor.
  • the transmitting element comprises a first surface facing said image sensor and a second surface through which said image information enters into the transmitting element.
  • Said image sensor is able to provide electrical signals representing said image information.
  • a mask having a plurality of transparent regions and opaque regions is ar- ranged on the second surface .
  • the transparent regions are realized without any optical focusing device. This is due to the fact that the openings in the mask act as a lens.
  • Said transparent regions are preferably pinholes . Therefore the mask acts as a multiple pinhole imaging device.
  • the advantage to devices using lenses is that such a mask has an unlimited depth of sharpness. As the distance from the mask to the image sensor is rather small, the usual disadvantages of small holes, such as poor light intensity and effects of diffraction are not an issue any more .
  • Such a device according to the present is particularly advantageous for the image acquisition of the iris of a human eye for identification purposes.
  • Fig. 1 shows schematically a cross section of a camera device according to a first embodiment of the present invention
  • Fig. 2 shows schematically a cross section of a camera device according to a second embodiment of the present invention
  • Fig- 3 shows schematically a cross section of a camera device according to a third embodiment of the present invention
  • Fig. 4a, 4b show a pattern of a mask that is used by the cam- era devices according to the embodiments of figures 1 to 3 ;
  • Fig. 4c shows an original image and its representation on the image sensor extending over a determined field of pixels
  • Fig. 4d shows the representations on the image sensor
  • Fig. 4e shows on the left one representation of the image source, extending over a dedicated field of the image sensor and on the right a high resolution image reconstructed from a plurality of such representation;
  • Fig. 5a,b show viewing rays originating from an exemplary point passing through the camera device
  • Fig. 6a shows a source for image information captured by a camera device according to the present invention
  • Fig. 6b shows the representative of figure 6b as projected onto an image sensor
  • Fig. 6c shows a representative image of the source
  • Fig. 7 shows schematically a first measurement method
  • Fig. 8 shows schematically a second measurement method
  • Fig. 9 shows schematically a third measurement method
  • Fig. 10 shows the a possible situation where total reflection occurs
  • Fig. 11 shows the relation between openings in the mask and the resulting image.
  • FIG. 1 shows a first embodiment of a camera device according to the present invention.
  • the camera device comprises an image sensor 1, a transmitting element 2 and a mask 3.
  • Such a device may be used to acquire an image of a human eye, in particu- lar of the iris of a human eye.
  • image information can be used for identification purposes.
  • Due to the arrangement of the transmitting element 2 and the mask 3 a plurality of representa ⁇ tion images will be provided on the image sensor.
  • An algorithm can be used to provide or determine, respectively, a high reso ⁇ lution counterpart image.
  • Said counterpart image represents the image (e.g. human eye) to be acquired.
  • Said transparent regions of the mask are arranged in respect of the image sensor in a manner that each transparent region is assigned to an essentially different array or field of pixels on the image sensor.
  • the image sensor 1 is able to convert image information into a data set or electrical signals.
  • an image sensor 1 is a low-cost CCD (charge-coupled device) or a CMOS (Complementary metal-oxide-semiconductor) sensor.
  • CCD charge-coupled device
  • CMOS Complementary metal-oxide-semiconductor
  • any other image sensor may be used as image sensor 1.
  • Said image sensor 1 comprises a surface 11 with an array of pixels or a pixel matrix having several pixels 10.
  • such an array usually comprises more than one million pixel or more, i.e. a matrix of more than 1000x1000 pixels.
  • the transmitting element 2 is arranged on the surface 11 of the image sensor 1.
  • Said transmitting element 2 is a layer having a first surface 21 and a second surface 22, both of which are preferably more or less planar and which even surfaces 21 and 22 are essentially parallel to each other.
  • Said first surface 21 of the transmitting element 2 faces the surface 11 of the image sensor l.
  • the transmitting element 2 is a transparent layer. With other words said transmitting element 2 is able to transmit image information to the image sensor 1.
  • the transmit- ting element 2 can be made out of glass, transparent plastics or air.
  • a thickness T of the transmitting element 2 is defined as being the measure from the first surface 21 to the second surface 22 of the transmitting element 2.
  • Said measure or thickness T is preferably between 50 ⁇ m and 800 ⁇ m. Even more preferably be ⁇ tween 100 ⁇ m and 500 ⁇ m. in case of the transmitting element 2 being in glass or plastic, it works as a spacer. If the trans ⁇ mitting element 2 is made of air, then separate spacers are pro ⁇ vided at the edges of the image sensor 1 between the layer 1 and the mask 3. Additionally there may then be provided intermediate pillars as spacers.
  • the mask 3 is arranged on the second surface 22 of the transmitting element 2.
  • Said mask 3 comprises transparent regions 30 and opaque regions 31. This means that image information is being transferred to the transmitting element 2 via the transparent regions 30, while the opaque regions 31 hinder image information to pass the mask 3.
  • the mask 3 is generally an opaque layer having transparent regions or openings 30.
  • Said openings are preferably circular and may also be designated as pinholes 30 that are arranged in the mask 3.
  • said openings may have any other cross-section such as elliptical, rectangular or quadratic. They may also be filled by a transparent material or with parts of the transmitting element 2. Such a filling prevents dust or dirt from being entered into the pinholes 30. It is possible to provide the mask 3 as a coating or included in an upper part of the transmitting element 2.
  • an opening 30 is circular, the diameter of said opening is preferably between 5 ⁇ m and 50 ⁇ m.
  • the largest clearance is between 5 ⁇ m and 50 ⁇ m.
  • the opening as shown in figure 1 shows the surface 32 of the opening as being convex in its cross sectional view. But in fact a surface that extends perpendicular to the surface of the mask is preferable. This means that the opening comprises a cylindri ⁇ cal surface.
  • image information can pass through the pinholes 30 of the mask 3. After leaving the mask 3 said image information is being coupled into the transmitting element via its second surface 22. After passing the transmitting element 2 the image information is being coupled out of the transmitting element 2 via its first surface 21. Afterwards the image information is being provided for the image sensor 1.
  • One single pinhole 30 is assigned to a dedicated field of pixels of the image sensor 1. As it can be seen with regard to figure 4c, one pinhole 30 provides one image representation onto the respective field of pixels. Thereby the image sensor has more or less the same number of fields as pinholes 30 which means that also the same numbers of image representations will be detected by the image sensor 1.
  • a dedicated field or array of said image sensor 1 is arranged wherein said field or array comprises preferably more than 25 pixels, such that on each of said dedicated fields one representation of said image information is shown. In other embodiments more than 50, more than 100 or more than 200 pixels are also possible. If the resolution shall be higher it is also possible to arrange more than 300, 400, 500 or even 1000 pixels per transparent region or pinhole 30.
  • the geometrical form of the fields may be variable. Preferably fields having the shape of a rectangle, a square, a circle or an ellipse are used.
  • Said dedicated field comprises a determined number of pixels be ⁇ ing arranged in said dedicated field.
  • said field is defined as a circle having a diameter of preferably 50 to 150 pixels.
  • said field may also be defined as a rec ⁇ tangle or a square, whereby the edge(s) of said rectangle or square is (are) between 50 to 150 pixels. This means that on a sensor having a number of 2 Megapixels several thousand low resolution images can be provided. However sensors having a larger or smaller surface may also be used.
  • Such transmitting element 2 and the mask 3 is advantageous, because the manufacturing of both is easy and rather inexpensive compared to more sophisticated optics.
  • a particular advantage is that an alignment of the mask 3 relative to the image sensor 1 is not necessary as the field of pixels being allocated to the pinholes 30 in the mask are defined as being those pixels which are underneath the respective pinhole.
  • a camera device according to the present invention can be produced in very high numbers at very low costs . Since smart cards are issued at high number, it is interesting to provide a smart card with such a camera device according to the present invention.
  • the mask 3 is placed at a distance from the image sensor 1 in order to produce images on the surface of the image sensor 1. Said images can be provided in a non-overlapping or in an overlapping manner.
  • the parameters which determine whether the image shall be overlapping or non-overlapping are the thickness T of the transmitting element 2, the width of the opening 30, the dis- tance from one opening 30 to a neighboring opening 30 and the thickness of the mask 3, which also represents the length of the opening in the mask 3. Each of said four parameters can be adjusted.
  • non-overlapping images has to be understood that a first representation image on the image sensor 1 does not inter ⁇ fere with a neighboring second representation image. Hence the images are adjacent to each other. If non-overlapping images are being produced said distance or thickness T has to be decreased or the distance between the openings 30 has to be increased. However the thickness T is also dependent on the width or clearance of the openings 30 and on the size of the pixels of the image sensor 1, respectively, if non-overlapping images shall be the result . However the above described parameters can be chosen as such that there is a distance between the representation images . Areas on the image sensor which are not being provided with image information, i.e. areas between the representation images, will not be considered when the counterpart image is being determined by an algorithm.
  • Overlapping images may also be provided due to an adjustment of the above mentioned parameter, i.e. the thickness T and the width or clearance of the openings, respectively.
  • An overlapping image is present when a first representation image on the image sensor does interfere partially with a neighboring second representation image. This means that some parts of said first and of said second representation image are provided for a common pixel area on the image sensor 1. Said overlapping areas must not necessarily be considered when the counterpart image is being determined.
  • the size of the image of a particular pinhole is limited by total reflection of the light rays on the second sur ⁇ face 21 of the transmitting element 2 and on the coating of the sensor 1, as depicted in Figure 10.
  • the angle of total reflection depends on the refraction index nl of the transmitting element 2 and the refraction index n2 of the coating of the sensor 1. If the angle ⁇ c is smaller than sin(n2/nl), total reflection occurs and the incident light does not reach the sensor 1. This is shown by means of light ray 100' and 101' .
  • the thickness of the mask 3 is preferably such that rays with maximum opposite inclinations passing through adjacent openings 30 do not impinge on the same pixel or only impinge on pixels being edge pixels or border pixels for different representation images .
  • Determination methods and algorithms to determine a counterpart image based on the low-resolution images are known by the person skilled in the art. Furthermore a super-resolution method as described herein may also be used for that purpose.
  • FIG. 2 shows a second embodiment of a camera device according to the present invention, wherein same parts are designated with the same reference numerals as in the first embodiment.
  • the transmitting element 2 comprises optional diaphragms 20 that extend from the first sur ⁇ face 21 to the second surface 22.
  • said dia ⁇ phragms extend through the whole transmitting element 2.
  • the diaphragms 20 provide a plurality of channels through the transmitting element 2.
  • said diaphragms 20 are arranged displaced from the openings 30 in the mask 3. This means that a single diaphragm 20 is only arranged in an opaque region 31 of the mask 3.
  • the diaphragms 20 are placed between the openings 30 of the mask 3.
  • Said diaphragms 20 have preferably a circular or rectangular cross-section.
  • the diaphragms 20 are being placed between the openings 20 to prevent light from falling from one opening 30 into the image region of another opening 30. This means that the light is being absorbed by the diaphragms.
  • the arrangement of the diaphragms 20 prevents interference between an image information being provided by a first opening and image information being provided by a further opening.
  • Said channel that is being provided by the diaphragms 20 is assigned to one opening 30 and accordingly to a field of pixels as defined above by means of the first embodiment.
  • Figure 3 shows a third embodiment of a camera device according to the present invention, wherein same parts are designated with the same reference numerals as in the first and the second embodiment .
  • the first side 21 of the transmitting element 2 is provided with a lens structure 4.
  • Said lens structure 4, that can also be designated as micro lens 4 act as an aperture element.
  • Said micro lenses are being distributed over the first surface 21 of the transmitting element.
  • Said micro lenses are arranged in a con ⁇ cave manner and have a shape that is partly spheroidal. Alterna ⁇ tively a convex shape is also possible.
  • Each opening in the mask is assigned to one lens, whereby said lens is assigned to a dedicated field of pixels.
  • said dedicated field may be chosen smaller as defined above, since rays which would intersect on or above border pixels are now refracted, so that no intersections may occur.
  • one lens is assigned to a field of pixels having a diameter of 5 to 100, in particular 30 to 100 pixel or having an edge length of 30 to 100 pixels.
  • said field of pixels comprises more than 25 pixels. In other embodiments more than 50, more than 100 or more than 200 pixels are also possible. If the resolution shall be higher it is also possible to arrange more than 300, 400, 500 or even 1000 pixels.
  • Figures 4a and 4b show in a schematic manner two possible pattern of the openings or pinholes 30 and the opaque region 31 of the mask 3.
  • the openings are arranged in a regular rectangular pattern or grid.
  • the arrows that are designated with a letter X or Y, respectively, represent the direction of the pattern.
  • the directions X and Y are perpendicular to each other.
  • the distance between from one opening 30 to another opening 30 is preferably equal in both directions X and Y.
  • Figure 4b shows a further possible arrangement of the openings 30 in the mask 3.
  • said openings 30 are ar ⁇ ranged in a hexagonal grid.
  • the mask 3 has the same size as the image sensor 1 in order to cover the whole surface of the later.
  • the mask 3 is preferably arranged in a manner that each opening is assigned to a field of pixels of the image sensor. As it is outlined by means of figure 1 to 3. As a result of such an ar ⁇ rangement image information that enters the camera device through a dedicated opening is being provided to a dedicated number of pixels of the image sensor. Hence every group of pixels has a dedicated opening through which image information is receivable .
  • Figure 4c to 4e illustrate the representation of an original image onto the image sensor. Thereby one single pinhole 30 is assigned or allocated to a dedicated field of pixels 10 of the image sensor 1.
  • the original image or image source I has the shape of letter P.
  • This image is then projected via the pinholes 30 onto the image sensor which results in a plurality of representations of the image information I' being projected onto the surface of the image sensor 1.
  • Each representation extends over the field of pixels.
  • each pinhole 30 is allocated to a dedicated field of pixels.
  • one representation I' extends over a plurality of pixels 10.
  • Figure 4d shows a section of an image sensor with a plurality of image representations I' . In X-direction there are 22 columns, whereas in Y-direction there are 9 rows of representations. This means that the 198 representations as depicted in figure 4d are projected via the same number of pinholes onto a part of the image sensor as shown in figure 4d.
  • Figure 4e shows on the left hand side a zoomed image representation extending over a dedicated field of pixels.
  • Said dedicated field of pixels is encompassed by a symbolic rectangle R.
  • Preferably said field comprises 20x20 up to 100x100 pixels. Other ranges are also possible and described herein.
  • Providing a larger number of image representation such that they extend over said field of pixels has the advantage that a counterpart image can be computed by using algorithms such as a mul- tiframe super-resolution algorithm.
  • Figure 5a shows some viewing rays 51 to 54 that originate from an exemplary point 8, which represents original image information to be captured by the camera device in order to provide a data set that represents the original image information.
  • Said point 8 is located at a distance D from the surface of the mask 3.
  • said rays of the original image information pass through the openings or pinholes 30. After being transmitted by the transmitting element 2 said rays reach the image sensor 1. Due to the arrangement of the diaphragms 20 or preferably due to the width of the openings 30 in combination with the thickness of the mask 3 it is prevented that rays 51 to 54 reached a field of pixel that is not the dedicated field of the opening through which said ray enters the transmitting element .
  • a plurality of images will be pro ⁇ vided on the image sensor 1. This means that a plurality of low resolution images of the same object will be detected by the im ⁇ age sensor 1. If said images are provided in a non-overlapping manner, the images are being displaced from each other by a displacement S.
  • Said displacement S depends on the spacing between the openings 30 in the mask 3 and on the unknown distance D between the object or source and the mask. In order to combine all several images into an image of higher resolution which is the basis for the data set representing the original image information, this displacement S has to be known.
  • Each of these images is depicted at a slightly different position. Said position depends on the spacing from one pinhole 30 to another pinhole 30. Furthermore said position depends on the unknown distance D of the original image from the camera. The distance D is unknown since the user is able to place the point 8, i.e. the original image, at a variable distance D.
  • the displacements S of the same object in the different images is required to be known. These displacements S can be evaluated from the images with sub-pixel precision using structures of the item that provides the image information.
  • said item is an iris of a human eye said iris provides the characteristics.
  • the mask 3 is planar, and the human iris can also be assumed to be flat. Therefore, the distance D can be as ⁇ sumed as being constant or at least similar all over an iris of a human eye. Additionally it has to be noted that the human iris is rather flat . Furthermore the images captured by each of the openings 30 are captured at the same time. The similarity of the distance D and the recording at the same time allow planar meas ⁇ urement methods . The aim of all these methods is to determine the distance between two images of the plurality of images as shown in Figure 6b .
  • a first planar measurement method of the distance S is the detection of the center of the pupil in each image.
  • Figure 7 shows a first image Il and a second image 12 of the same eye E having a pupil P and an iris R.
  • Said images Il and 12 are representation images on the image sensor of an original image (i.e. of the eye E in this case).
  • the edges at the border between the pupil P and the iris R are detected and an ellipse is fitted to them.
  • the center of this ellipse is the center of the pupil.
  • the offset of this center compared to the center of the image i.e. the position of the opening 31 gives the distances Sx and Sy in the directions of a coordinate system.
  • a second planar measurement method is a motion estimation technique, for instance block matching.
  • Figure 8 shows on the left an image Il of one opening in the mask, where a block Bl of for instance 16x16 pixels has been selected. In the neighboring image 12, this block Bl is compared to a block B2 of the same size, whereas a dissimilarity function between these blocks is computed. In the simplest case, this is the sum of absolute difference between the pixels of each block Bl and B2.
  • the position of the block B2 in the second image is varied and the dissimilarity is minimized. The position with minimal dissimilarity delivers the searched distances Sx and Sy.
  • a third planar measurement method of the distances S is the measurement of the shift between the images using phase correlation techniques.
  • Figure 9 shows two neighbored images Il and 12 of the same eye.
  • a patch Jl for instance centered on the pupil
  • the shift of this patch J2 is determined by phase correlation technique.
  • Phase correlation is a well known method to measure translation between two images. Given two images f and g, where f is the patch of the left image and g is the shifted version of f in the right iamge. From these images, their two dimensional Fourier transforms F and G are computed. The normalized cross power spectrum N is then computed to as
  • D is the decimation or upsampling matrix, which defines the ratio between the size of the high resolution image and the size of the low resolution image.
  • the theoretical limit for this magnification is sqrt (N) , if N low resolution images are provided.
  • Fk is the transformation matrix which describes simply the displacement Sk of the image Yk measured in the previous step.
  • H is the blur matrix representing a local Gaussian point spread function. It can also be simplified to a local average function, ⁇ is a regularization factor determining the amount each iteration affects the current estimation. The summation occurs in the affected pixels of a high resolution image of the same size as Xn. This algorithm contains only very simple operations that can easily be implemented on smart cards .
  • the camera devices according to the present invention can be integrated into a smart card.
  • a smart card is used to identify a user while logging into a computer system.
  • a smart card comprises a low power processor in order to execute a simple shift-and-add-resolution computation.
  • FIG. 5b shows a situation that is analog to the one as shown in Figure 5a.
  • the transmitting element 2 is provided with the lens 4 according to the third embodiment.
  • the image information provided by the original image 8 is being supplied to a dedicated array of pixels 12, having a length of five pixels in this illustrative drawing.
  • Preferred embodiments of such a device can use a dedicated array of pixels 12 having a length between 20 and 100 pixels, especially between 30 bis 60 pixels .
  • the transmitting structure 2 may comprise diaphragms 20 as well as a lens structure 4 for providing definite light paths towards the sensor elements for light transmitted via adjacent pinholes 30.
  • the distance of adjacent pinholes 30 has to be at least F according to Fig. 11.
  • Figure 6a shows an original image I that is to be captured by the camera device according to present invention in order be represented by a data set that is being provided by the image sensor.
  • the original image is a letter W P" arranged in a rectangular field.
  • Figure 6b shows the original image as it is being projected onto the image sensor 1.
  • a mask 3 comprises as an example nine circular openings which result in nine non-overlapping representations I' of the original image on the image sensor 1.
  • the circle that surrounds the image information I' is a resultant of the edge of the opening in the mask 3.
  • the image information I' comprises a low-resolution, hence a plurality of low resolution images will be provided onto the image sensor 1. Due to the displacement of the openings 30 said images are slightly displaced.
  • the area as designated with the reference letter Z is preferably not included into the calculation.
  • the openings in the mask are arranged such that an image is displayed on a determined area on the image sensor 3.
  • the diameter of the openings in the mask 3, the distance from one opening to another and the thickness of the transmitting element 2 are designed such that the determined areas on the image sensor do not overlap. This may also be achieved by means of the lenses 4 .
  • Fig. 6c shows a counterpart image of the letter "P" with the rectangular field, whereby said counterpart image represents the same image information as the original image.
  • the counterpart image is being provided by an algorithm based on the data that is being provided by the image sensor or a super-resolution algorithm as described above. For the present embodiment this means that counterpart image bases on the nine images provided by the openings in the mask.
  • the information of the counterpart image can for example be used for authorizing purposes to gain access to a computer system.
  • the mask 3 is provided with a mirror 32.
  • the mirror 32 is arranged on the surface of the mask 3 that faces the image to be captured, e.g. a human eye.
  • the opaque mask 3 is covered with the mirror 32 on the surface that does not face the transmitting element 2.
  • the mirror surface 32 is used to align the camera device according to the present invention. The user is able to recognize its own eye in the mirror and is therefore able to align the camera device exactly. This is a particular advantage since the openings 31 in the mask are so small that they may not be visible by the user.
  • the mirror 32 is preferably a separate layer that coats the mask 3. An axis that extends perpendicular from the surface of the mirror 32 defines the optical axis of the camera device.
  • a total reflectance mirror may also be used to provide the mask 3 and the mirror 32.
  • the mirror is arranged directly on the second surface 22 of the transmitting element 2 and that the mirror comprises all the features of the mask 3 as disclosed above, e.g. the pinholes 30.
  • the mirror 32 may also comprise printed or engraved horizontal and/or vertical and/or angular lines. Alternatively or additionally a printed or engraved circle in the center of the mirror can also be arranged. Said printed or engraved struc ⁇ tures allow a better alignment of the mirror by the user.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Theoretical Computer Science (AREA)
  • Signal Processing (AREA)
  • Ophthalmology & Optometry (AREA)
  • Human Computer Interaction (AREA)
  • Optics & Photonics (AREA)
  • Vascular Medicine (AREA)
  • Image Input (AREA)

Abstract

A camera device for image acquisition comprises an image sensor 1 that is arranged behind a transmitting element 2 for transmitting image information to said image sensor 1. The transmitting element 2 comprises a first surface 21 facing said image sensor 1 and a second surface 22 through which said image information enters into the transmitting element 2. Said image sensor 1 is able to provide electrical signals representing said image information. A mask 3 having a plurality transparent regions 30 and opaque regions 31 is arranged on the second surface 22.

Description

Camera device for image acquisition of flat or nearly flat objects
Field of the invention
The invention relates to a camera device having a low height for image acquisition of flat or nearly flat objects, wherein the camera is provided for especially but not exclusively objects in the form of smart cards, e.g. credit cards.
Prior Art
Camera devices having a low height are well known from prior art. For example it is well known to provide a mobile phone with a camera. Since the size of a mobile phone is crucial it is important to use cameras with optical elements having a small scale. However the height of such cameras lies still in the range of several millimetres or even centimetres.
DE 199 17 890 discloses an image capture system. Said image capture system comprises a plurality of micro lenses that are arranged side by side. Furthermore the system comprises an image detector behind said micro lenses. Since there is a plurality of micro lenses arranged, said lenses provide a plurality of images onto a corresponding image sensor, i.e. every optical channel corresponds to one cell of the detector.
Providing the lenses in the arrangement according to DE 199 17 890 results in a costly device. Furthermore the precise alignment of the lenses with respect to the detector cells is very delicate. Therefore a displacement between the detector cells and the lenses is very likely. Therefore the resolution and contrast of the image decreases very rapidly. EP 0 809 124 shows a short focal length image sensor having a lenslet array being positioned over a photosensitive imaging ar¬ ray having a number of photosensitive sites corresponding in number to at least the number of lenslets. Thereby each lenslet is allocated to a photosensitive site. The term photosensitive sites is understood as to be one pixel of an image sensor. In an RGB-environment the photosensitive site are three pixels namely a red, green and blue pixel. However, it is a drawback that the lenslets have to be arranged with respect to the photosensitive sites such that a one lenslet is allocated to the respective site which is a single pixel.
Summary of the invention
An object of the present invention is to provide a device, which does not have the disadvantages according to devices of prior art. In particular such a camera device shall have a small height with least possible optical elements as well as sufficient image quality. Furthermore it is an object of the camera device to provide a plurality of image representations onto an image sensor in order to determine a single image representation based on the signals as detected by the image sensor. A further object of the present invention is to provide a camera device to acquire an image of a human eye for identification purposes.
According to the invention there is provided a camera device for image acquisition. Said device comprises an image sensor that is arranged behind a transmitting element for transmitting image information to said image sensor. The transmitting element comprises a first surface facing said image sensor and a second surface through which said image information enters into the transmitting element. Said image sensor is able to provide electrical signals representing said image information. A mask having a plurality of transparent regions and opaque regions is ar- ranged on the second surface .
It is an advantage of the present invention that the transparent regions are realized without any optical focusing device. This is due to the fact that the openings in the mask act as a lens.
Said transparent regions are preferably pinholes . Therefore the mask acts as a multiple pinhole imaging device. The advantage to devices using lenses is that such a mask has an unlimited depth of sharpness. As the distance from the mask to the image sensor is rather small, the usual disadvantages of small holes, such as poor light intensity and effects of diffraction are not an issue any more .
Such a device according to the present is particularly advantageous for the image acquisition of the iris of a human eye for identification purposes.
Brief description of the drawings
The drawings will be explained in greater detail by means of a description of an exemplary embodiment, with reference to the following figures:
Fig. 1 shows schematically a cross section of a camera device according to a first embodiment of the present invention;
Fig. 2 shows schematically a cross section of a camera device according to a second embodiment of the present invention;
Fig- 3 shows schematically a cross section of a camera device according to a third embodiment of the present invention;
Fig. 4a, 4b show a pattern of a mask that is used by the cam- era devices according to the embodiments of figures 1 to 3 ;
Fig. 4c shows an original image and its representation on the image sensor extending over a determined field of pixels;
Fig. 4d shows the representations on the image sensor;
Fig. 4e shows on the left one representation of the image source, extending over a dedicated field of the image sensor and on the right a high resolution image reconstructed from a plurality of such representation;
Fig. 5a,b show viewing rays originating from an exemplary point passing through the camera device;
Fig. 6a shows a source for image information captured by a camera device according to the present invention;
Fig. 6b shows the representative of figure 6b as projected onto an image sensor
Fig. 6c shows a representative image of the source;
Fig. 7 shows schematically a first measurement method;
Fig. 8 shows schematically a second measurement method;
Fig. 9 shows schematically a third measurement method;®
Fig. 10 shows the a possible situation where total reflection occurs; and
Fig. 11 shows the relation between openings in the mask and the resulting image.
Detailed description of the preferred embodiments Figure 1 shows a first embodiment of a camera device according to the present invention. The camera device comprises an image sensor 1, a transmitting element 2 and a mask 3.
Such a device according to the embodiments of the present invention may be used to acquire an image of a human eye, in particu- lar of the iris of a human eye. Such image information can be used for identification purposes. Due to the arrangement of the transmitting element 2 and the mask 3 a plurality of representa¬ tion images will be provided on the image sensor. As the plural¬ ity of representation images are projected onto a corresponding field of pixels being a part of the whole image sensor the de¬ tected representation images have a low resolution. An algorithm can be used to provide or determine, respectively, a high reso¬ lution counterpart image. Said counterpart image represents the image (e.g. human eye) to be acquired. Said transparent regions of the mask are arranged in respect of the image sensor in a manner that each transparent region is assigned to an essentially different array or field of pixels on the image sensor.
The image sensor 1 is able to convert image information into a data set or electrical signals. Preferably such an image sensor 1 is a low-cost CCD (charge-coupled device) or a CMOS (Complementary metal-oxide-semiconductor) sensor. However one has to note that any other image sensor may be used as image sensor 1. Said image sensor 1 comprises a surface 11 with an array of pixels or a pixel matrix having several pixels 10. Nowadays such an array usually comprises more than one million pixel or more, i.e. a matrix of more than 1000x1000 pixels.
The transmitting element 2 is arranged on the surface 11 of the image sensor 1. Said transmitting element 2 is a layer having a first surface 21 and a second surface 22, both of which are preferably more or less planar and which even surfaces 21 and 22 are essentially parallel to each other. Said first surface 21 of the transmitting element 2 faces the surface 11 of the image sensor l. Preferably the transmitting element 2 is a transparent layer. With other words said transmitting element 2 is able to transmit image information to the image sensor 1. The transmit- ting element 2 can be made out of glass, transparent plastics or air.
A thickness T of the transmitting element 2 is defined as being the measure from the first surface 21 to the second surface 22 of the transmitting element 2. Said measure or thickness T is preferably between 50 μm and 800 μm. Even more preferably be¬ tween 100 μm and 500 μm. in case of the transmitting element 2 being in glass or plastic, it works as a spacer. If the trans¬ mitting element 2 is made of air, then separate spacers are pro¬ vided at the edges of the image sensor 1 between the layer 1 and the mask 3. Additionally there may then be provided intermediate pillars as spacers.
The mask 3 is arranged on the second surface 22 of the transmitting element 2. Said mask 3 comprises transparent regions 30 and opaque regions 31. This means that image information is being transferred to the transmitting element 2 via the transparent regions 30, while the opaque regions 31 hinder image information to pass the mask 3. With other words the mask 3 is generally an opaque layer having transparent regions or openings 30. Said openings are preferably circular and may also be designated as pinholes 30 that are arranged in the mask 3. Alternatively said openings may have any other cross-section such as elliptical, rectangular or quadratic. They may also be filled by a transparent material or with parts of the transmitting element 2. Such a filling prevents dust or dirt from being entered into the pinholes 30. It is possible to provide the mask 3 as a coating or included in an upper part of the transmitting element 2.
If an opening 30 is circular, the diameter of said opening is preferably between 5 μm and 50 μm. For other shapes of the opening 30, it is possible to define the largest clearance as being between 5 μm and 50 μm.
The opening as shown in figure 1 shows the surface 32 of the opening as being convex in its cross sectional view. But in fact a surface that extends perpendicular to the surface of the mask is preferable. This means that the opening comprises a cylindri¬ cal surface.
As it can be seen from figure 1, image information can pass through the pinholes 30 of the mask 3. After leaving the mask 3 said image information is being coupled into the transmitting element via its second surface 22. After passing the transmitting element 2 the image information is being coupled out of the transmitting element 2 via its first surface 21. Afterwards the image information is being provided for the image sensor 1.
One single pinhole 30 is assigned to a dedicated field of pixels of the image sensor 1. As it can be seen with regard to figure 4c, one pinhole 30 provides one image representation onto the respective field of pixels. Thereby the image sensor has more or less the same number of fields as pinholes 30 which means that also the same numbers of image representations will be detected by the image sensor 1.
With other words: per transparent region or pinhole 30 a dedicated field or array of said image sensor 1 is arranged wherein said field or array comprises preferably more than 25 pixels, such that on each of said dedicated fields one representation of said image information is shown. In other embodiments more than 50, more than 100 or more than 200 pixels are also possible. If the resolution shall be higher it is also possible to arrange more than 300, 400, 500 or even 1000 pixels per transparent region or pinhole 30. The geometrical form of the fields may be variable. Preferably fields having the shape of a rectangle, a square, a circle or an ellipse are used.
Said dedicated field comprises a determined number of pixels be¬ ing arranged in said dedicated field. Preferably said field is defined as a circle having a diameter of preferably 50 to 150 pixels. Alternatively said field may also be defined as a rec¬ tangle or a square, whereby the edge(s) of said rectangle or square is (are) between 50 to 150 pixels. This means that on a sensor having a number of 2 Megapixels several thousand low resolution images can be provided. However sensors having a larger or smaller surface may also be used.
The use of such transmitting element 2 and the mask 3 is advantageous, because the manufacturing of both is easy and rather inexpensive compared to more sophisticated optics. A particular advantage is that an alignment of the mask 3 relative to the image sensor 1 is not necessary as the field of pixels being allocated to the pinholes 30 in the mask are defined as being those pixels which are underneath the respective pinhole. This means that a camera device according to the present invention can be produced in very high numbers at very low costs . Since smart cards are issued at high number, it is interesting to provide a smart card with such a camera device according to the present invention.
Due to the arrangement of the thin transmitting element 2, the mask 3 is placed at a distance from the image sensor 1 in order to produce images on the surface of the image sensor 1. Said images can be provided in a non-overlapping or in an overlapping manner. The parameters which determine whether the image shall be overlapping or non-overlapping are the thickness T of the transmitting element 2, the width of the opening 30, the dis- tance from one opening 30 to a neighboring opening 30 and the thickness of the mask 3, which also represents the length of the opening in the mask 3. Each of said four parameters can be adjusted.
The term non-overlapping images has to be understood that a first representation image on the image sensor 1 does not inter¬ fere with a neighboring second representation image. Hence the images are adjacent to each other. If non-overlapping images are being produced said distance or thickness T has to be decreased or the distance between the openings 30 has to be increased. However the thickness T is also dependent on the width or clearance of the openings 30 and on the size of the pixels of the image sensor 1, respectively, if non-overlapping images shall be the result . However the above described parameters can be chosen as such that there is a distance between the representation images . Areas on the image sensor which are not being provided with image information, i.e. areas between the representation images, will not be considered when the counterpart image is being determined by an algorithm.
Overlapping images may also be provided due to an adjustment of the above mentioned parameter, i.e. the thickness T and the width or clearance of the openings, respectively. An overlapping image is present when a first representation image on the image sensor does interfere partially with a neighboring second representation image. This means that some parts of said first and of said second representation image are provided for a common pixel area on the image sensor 1. Said overlapping areas must not necessarily be considered when the counterpart image is being determined.
Additionally, the size of the image of a particular pinhole is limited by total reflection of the light rays on the second sur¬ face 21 of the transmitting element 2 and on the coating of the sensor 1, as depicted in Figure 10. The angle of total reflection depends on the refraction index nl of the transmitting element 2 and the refraction index n2 of the coating of the sensor 1. If the angle Θc is smaller than sin(n2/nl), total reflection occurs and the incident light does not reach the sensor 1. This is shown by means of light ray 100' and 101' .
Figure 11 depicts the relation between the mentioned parameters and the field width of the image on the sensor 1. Given the diameter P of the opening 31, the thickness D of the mask 3 and the thickness T of the transmitting element 2, the field width F of the image becomes F=(P/D)*T , in case of a cylindrical surface of the opening 31.
Due to the thickness D of the mask diffraction effects may occur.
The thickness of the mask 3 is preferably such that rays with maximum opposite inclinations passing through adjacent openings 30 do not impinge on the same pixel or only impinge on pixels being edge pixels or border pixels for different representation images .
Determination methods and algorithms to determine a counterpart image based on the low-resolution images are known by the person skilled in the art. Furthermore a super-resolution method as described herein may also be used for that purpose.
Figure 2 shows a second embodiment of a camera device according to the present invention, wherein same parts are designated with the same reference numerals as in the first embodiment. According to said second embodiment the transmitting element 2 comprises optional diaphragms 20 that extend from the first sur¬ face 21 to the second surface 22. This means that said dia¬ phragms extend through the whole transmitting element 2. With other words it can be said that the diaphragms 20 provide a plurality of channels through the transmitting element 2. As it can be seen from figure 2 said diaphragms 20 are arranged displaced from the openings 30 in the mask 3. This means that a single diaphragm 20 is only arranged in an opaque region 31 of the mask 3. Hence the diaphragms 20 are placed between the openings 30 of the mask 3. Said diaphragms 20 have preferably a circular or rectangular cross-section. The diaphragms 20 are being placed between the openings 20 to prevent light from falling from one opening 30 into the image region of another opening 30. This means that the light is being absorbed by the diaphragms. Hence the arrangement of the diaphragms 20 prevents interference between an image information being provided by a first opening and image information being provided by a further opening.
Said channel that is being provided by the diaphragms 20 is assigned to one opening 30 and accordingly to a field of pixels as defined above by means of the first embodiment.
Figure 3 shows a third embodiment of a camera device according to the present invention, wherein same parts are designated with the same reference numerals as in the first and the second embodiment .
The first side 21 of the transmitting element 2 is provided with a lens structure 4. Said lens structure 4, that can also be designated as micro lens 4, act as an aperture element. Said micro lenses are being distributed over the first surface 21 of the transmitting element. Said micro lenses are arranged in a con¬ cave manner and have a shape that is partly spheroidal. Alterna¬ tively a convex shape is also possible.
Each opening in the mask is assigned to one lens, whereby said lens is assigned to a dedicated field of pixels. Due to the ef¬ fect of the lens said dedicated field may be chosen smaller as defined above, since rays which would intersect on or above border pixels are now refracted, so that no intersections may occur. Preferably one lens is assigned to a field of pixels having a diameter of 5 to 100, in particular 30 to 100 pixel or having an edge length of 30 to 100 pixels. Most preferably said field of pixels comprises more than 25 pixels. In other embodiments more than 50, more than 100 or more than 200 pixels are also possible. If the resolution shall be higher it is also possible to arrange more than 300, 400, 500 or even 1000 pixels. With such a configuration it is possible to provide more low resolution images while using the same image sensor as with the first and the second embodiment This means that on a sensor having a number of 2 Megapixels several thousand low resolution images can be provided. However sensors having a larger or smaller surface may also be used.
Figures 4a and 4b show in a schematic manner two possible pattern of the openings or pinholes 30 and the opaque region 31 of the mask 3.
As it can be seen from the embodiment of a mask in figure 4a the openings are arranged in a regular rectangular pattern or grid. The arrows that are designated with a letter X or Y, respectively, represent the direction of the pattern. Preferably the directions X and Y are perpendicular to each other. The distance between from one opening 30 to another opening 30 is preferably equal in both directions X and Y.
Figure 4b shows a further possible arrangement of the openings 30 in the mask 3. In this arrangement said openings 30 are ar¬ ranged in a hexagonal grid.
Preferably the mask 3 has the same size as the image sensor 1 in order to cover the whole surface of the later.
The mask 3 is preferably arranged in a manner that each opening is assigned to a field of pixels of the image sensor. As it is outlined by means of figure 1 to 3. As a result of such an ar¬ rangement image information that enters the camera device through a dedicated opening is being provided to a dedicated number of pixels of the image sensor. Hence every group of pixels has a dedicated opening through which image information is receivable .
Figure 4c to 4e illustrate the representation of an original image onto the image sensor. Thereby one single pinhole 30 is assigned or allocated to a dedicated field of pixels 10 of the image sensor 1.
The original image or image source I has the shape of letter P. This image is then projected via the pinholes 30 onto the image sensor which results in a plurality of representations of the image information I' being projected onto the surface of the image sensor 1. Each representation extends over the field of pixels. Thereby each pinhole 30 is allocated to a dedicated field of pixels. Hence one representation I' extends over a plurality of pixels 10. This means with other words that the original image I is represented in a plurality of fields of pixels as a plurality of image representations I' . Figure 4d shows a section of an image sensor with a plurality of image representations I' . In X-direction there are 22 columns, whereas in Y-direction there are 9 rows of representations. This means that the 198 representations as depicted in figure 4d are projected via the same number of pinholes onto a part of the image sensor as shown in figure 4d.
Figure 4e shows on the left hand side a zoomed image representation extending over a dedicated field of pixels. Said dedicated field of pixels is encompassed by a symbolic rectangle R. Preferably said field comprises 20x20 up to 100x100 pixels. Other ranges are also possible and described herein.
Providing a larger number of image representation such that they extend over said field of pixels has the advantage that a counterpart image can be computed by using algorithms such as a mul- tiframe super-resolution algorithm.
On the right hand side a counterpart image I'' is shown that has been computed by using multiframe super-resolution algorithms as described below.
Figure 5a shows some viewing rays 51 to 54 that originate from an exemplary point 8, which represents original image information to be captured by the camera device in order to provide a data set that represents the original image information. Said point 8 is located at a distance D from the surface of the mask 3. As it can be seen in the Figure, said rays of the original image information pass through the openings or pinholes 30. After being transmitted by the transmitting element 2 said rays reach the image sensor 1. Due to the arrangement of the diaphragms 20 or preferably due to the width of the openings 30 in combination with the thickness of the mask 3 it is prevented that rays 51 to 54 reached a field of pixel that is not the dedicated field of the opening through which said ray enters the transmitting element .
Due to the arrangement of a plurality of openings 30 in the opaque layer of the mask 3, a plurality of images will be pro¬ vided on the image sensor 1. This means that a plurality of low resolution images of the same object will be detected by the im¬ age sensor 1. If said images are provided in a non-overlapping manner, the images are being displaced from each other by a displacement S. Said displacement S depends on the spacing between the openings 30 in the mask 3 and on the unknown distance D between the object or source and the mask. In order to combine all several images into an image of higher resolution which is the basis for the data set representing the original image information, this displacement S has to be known.
Each of these images is depicted at a slightly different position. Said position depends on the spacing from one pinhole 30 to another pinhole 30. Furthermore said position depends on the unknown distance D of the original image from the camera. The distance D is unknown since the user is able to place the point 8, i.e. the original image, at a variable distance D.
However in order to combine all images into an image of higher resolution, the displacements S of the same object in the different images is required to be known. These displacements S can be evaluated from the images with sub-pixel precision using structures of the item that provides the image information.
If said item is an iris of a human eye said iris provides the characteristics. The mask 3 is planar, and the human iris can also be assumed to be flat. Therefore, the distance D can be as¬ sumed as being constant or at least similar all over an iris of a human eye. Additionally it has to be noted that the human iris is rather flat . Furthermore the images captured by each of the openings 30 are captured at the same time. The similarity of the distance D and the recording at the same time allow planar meas¬ urement methods . The aim of all these methods is to determine the distance between two images of the plurality of images as shown in Figure 6b .
A first planar measurement method of the distance S is the detection of the center of the pupil in each image. As an illustration, Figure 7 shows a first image Il and a second image 12 of the same eye E having a pupil P and an iris R. Said images Il and 12 are representation images on the image sensor of an original image (i.e. of the eye E in this case). Using image processing, the edges at the border between the pupil P and the iris R are detected and an ellipse is fitted to them. The center of this ellipse is the center of the pupil. The offset of this center compared to the center of the image (i.e. the position of the opening 31) gives the distances Sx and Sy in the directions of a coordinate system.
A second planar measurement method is a motion estimation technique, for instance block matching. Figure 8 shows on the left an image Il of one opening in the mask, where a block Bl of for instance 16x16 pixels has been selected. In the neighboring image 12, this block Bl is compared to a block B2 of the same size, whereas a dissimilarity function between these blocks is computed. In the simplest case, this is the sum of absolute difference between the pixels of each block Bl and B2. Using a search technique, the position of the block B2 in the second image is varied and the dissimilarity is minimized. The position with minimal dissimilarity delivers the searched distances Sx and Sy.
A third planar measurement method of the distances S is the measurement of the shift between the images using phase correlation techniques. Figure 9 shows two neighbored images Il and 12 of the same eye. In the left image, a patch Jl (for instance centered on the pupil) of the iris pattern including the pupil is selected. On the right image, the shift of this patch J2 is determined by phase correlation technique. Phase correlation is a well known method to measure translation between two images. Given two images f and g, where f is the patch of the left image and g is the shifted version of f in the right iamge. From these images, their two dimensional Fourier transforms F and G are computed. The normalized cross power spectrum N is then computed to as
FG where G* is the conjugate complex of G. The phase correlation PC=IFFT(N) is the inverse Fourier transform of N. The shift between the two images is then found by determination of the position of the peak in PC. This peak can also be determnined with sub-pixel precision.
As soon as the individual displacements S are determined for all low resolution images, a larger image with higher resolution can be computed using multiframe super-resolution algorithms. Because the computation power of smart card is small, a simplified version of the algorithm described in ,,Fast and Robust Multi- frame Super Resolution", IEEE Transactions on Image Processing, Vol. 13, No. 10, October 2004 by Sina Farsiu, M. Dirk Robinson, Michael Elad, and Peyman Milanfar is applied. An iterative algorithm may be formulated for the description of this algorithm. Given an image Xn of higher resolution than the individual low resolution images, an instance Yk of the kth low resolution im¬ age is added to the image Xn resulting in image Xn+1 using
Figure imgf000019_0001
where D is the decimation or upsampling matrix, which defines the ratio between the size of the high resolution image and the size of the low resolution image. The theoretical limit for this magnification is sqrt (N) , if N low resolution images are provided. Fk is the transformation matrix which describes simply the displacement Sk of the image Yk measured in the previous step. H is the blur matrix representing a local Gaussian point spread function. It can also be simplified to a local average function, β is a regularization factor determining the amount each iteration affects the current estimation. The summation occurs in the affected pixels of a high resolution image of the same size as Xn. This algorithm contains only very simple operations that can easily be implemented on smart cards .
The camera devices according to the present invention can be integrated into a smart card. Such a smart card is used to identify a user while logging into a computer system. Preferably a smart card comprises a low power processor in order to execute a simple shift-and-add-resolution computation.
It is an advantage of a camera device according to the present invention that its use is independent from any depth of focus issues. Cameras using microlenses do not have this advantage and have a fixed depth of focus at a given distance. This is very important for iris recognition on a smart card, because the distance between the card and the eye can only be adjusted manually and will not be very precise. Figure 5b shows a situation that is analog to the one as shown in Figure 5a. The transmitting element 2 is provided with the lens 4 according to the third embodiment. As it can be seen the image information provided by the original image 8 is being supplied to a dedicated array of pixels 12, having a length of five pixels in this illustrative drawing. Preferred embodiments of such a device can use a dedicated array of pixels 12 having a length between 20 and 100 pixels, especially between 30 bis 60 pixels .
Although the above description comprises embodiments showing a device having sufficient distance between adjacent pinholes 30 or embodiments having additional diaphragms 20 to block the path of light between adjacent pinholes 30 or embodiments having a lens structure 4 on the sensor 1, it is clear that these features can be combined, e.g. the transmitting structure 2 may comprise diaphragms 20 as well as a lens structure 4 for providing definite light paths towards the sensor elements for light transmitted via adjacent pinholes 30.
If no overlapping images are to be produced and no lenses as in the embodiment according to Fig. 5b or diaphragms 20 are used then the distance of adjacent pinholes 30 has to be at least F according to Fig. 11.
If lenses as in the embodiment according to Fig. 5b are used then no overlapping images should be created. This relates to possible distances between adjacent pinholes 30 being in the range that a sufficient number of pixels can be used for the recognition of the object, i.e. the human iris. This can usually be accomplished with 30 pixels and this currently relates to a distance between 200 and 300 micrometers. At the same time every device according to such an embodiment can use any method to determine the distance between the adja¬ cent images. As examples three methods are explained above. Ad¬ ditionally the construction of the improved image with a super- resolution technique is explained as an example which can be applied to any disclosed embodiment of the device and focussing technique .
Figure 6a shows an original image I that is to be captured by the camera device according to present invention in order be represented by a data set that is being provided by the image sensor. The original image is a letter WP" arranged in a rectangular field.
Figure 6b shows the original image as it is being projected onto the image sensor 1. In this exemplary embodiment a mask 3 comprises as an example nine circular openings which result in nine non-overlapping representations I' of the original image on the image sensor 1. The circle that surrounds the image information I' is a resultant of the edge of the opening in the mask 3. The image information I' comprises a low-resolution, hence a plurality of low resolution images will be provided onto the image sensor 1. Due to the displacement of the openings 30 said images are slightly displaced. The area as designated with the reference letter Z is preferably not included into the calculation.
The openings in the mask are arranged such that an image is displayed on a determined area on the image sensor 3. Preferably the diameter of the openings in the mask 3, the distance from one opening to another and the thickness of the transmitting element 2 are designed such that the determined areas on the image sensor do not overlap. This may also be achieved by means of the lenses 4 .
Fig. 6c shows a counterpart image of the letter "P" with the rectangular field, whereby said counterpart image represents the same image information as the original image. The counterpart image is being provided by an algorithm based on the data that is being provided by the image sensor or a super-resolution algorithm as described above. For the present embodiment this means that counterpart image bases on the nine images provided by the openings in the mask.
The information of the counterpart image can for example be used for authorizing purposes to gain access to a computer system.
In a further embodiment the mask 3 is provided with a mirror 32. The mirror 32 is arranged on the surface of the mask 3 that faces the image to be captured, e.g. a human eye. With other words, the opaque mask 3 is covered with the mirror 32 on the surface that does not face the transmitting element 2. The mirror surface 32 is used to align the camera device according to the present invention. The user is able to recognize its own eye in the mirror and is therefore able to align the camera device exactly. This is a particular advantage since the openings 31 in the mask are so small that they may not be visible by the user. The mirror 32 is preferably a separate layer that coats the mask 3. An axis that extends perpendicular from the surface of the mirror 32 defines the optical axis of the camera device.
Alternatively a total reflectance mirror may also be used to provide the mask 3 and the mirror 32. This means that the mirror is arranged directly on the second surface 22 of the transmitting element 2 and that the mirror comprises all the features of the mask 3 as disclosed above, e.g. the pinholes 30. Additionally the mirror 32 may also comprise printed or engraved horizontal and/or vertical and/or angular lines. Alternatively or additionally a printed or engraved circle in the center of the mirror can also be arranged. Said printed or engraved struc¬ tures allow a better alignment of the mirror by the user.
List of reference numerals
1 image sensor
2 transmitting element
3 mask
4 lens
5 viewing rays
10 pixel
20 channels
21 first surface
22 second surface
30 pinholes or openings
31 opaque region
32 mirror
51 first ray
52 second ray
53 third ray
54 fourth ray
T thickness
D distance
S displacement
I original image
I' representation of the image information
I' ' counterpart image
E eye
P pupil
R iris B block of pixels
Z region between representation images

Claims

Claims
1. Camera device for acqusition of an original image for identification purposes comprising an image sensor (1) that is arranged behind a transmitting element (2) for transmitting im¬ age information to said image sensor (1) , wherein the transmitting element (2) comprises a first surface (21) facing said image sensor and a second surface (22) through which said image information enters into the transmitting element, and wherein said image sensor (1) is able to provide electrical signals representing said original image and comprises a plurality of pixels, characterized in that a mask (3) is arranged on the second surface (22) and in that said mask comprises a plurality of transparent regions (30) and opaque regions (31) , wherein transparent regions (30) of the mask (3) are arranged in respect of the image sensor (1) in a manner that each transparent region is assigned to an essentially different array of pixels on the image sensor (1) .
2. Camera device according to claim 1, wherein said array comprises more than 25 pixels, and wherein on each of said dedicated array one representation of said image information is shown, wherein the image is transmitted through the assigned transparent region (30) .
3. Camera device according to claim 1 or 2, characterized in that the transparent regions (30) are arranged regularly spaced from each other and the transmitting element has a thickness (T) such that the images representation are provided in a non- overlapping manner.
4. Camera device according to any of the preceding claims, characterized in that the transmitting element (2) has a thickness (T) that is between 50 μm and 800 μm in particular between 100 μm and 500 μm.
5. Camera device according to any of the previous claims, characterized in that the image sensor (1) and the mask (3) is separated through spacers and in that the transmitting element (2) comprises an air film.
6. Camera device according to any of the previous claims, characterized in that the transparent regions (30) have a circular or elliptical or rectangular or quadratic shape and in that the largest clearance is between 5 μm and 50 μm.
7. Camera device according to any of the previous claims, characterized in that the thickness of the mask (3) is such that rays (51-53) with maximum opposite inclinations passing through adjacent openings (30) do not impinge on the same pixel or only impinge on pixels being edge pixels for different representation images .
8. Camera device according to any of the previous claims 1 to 6, characterized in that a microlens distribution (4) is provided on the image sensor (1) either between the image sensor
(1) and said transmitting element (2) or the microlens distribution (4) being part of the region of the transmitting element
(2) adjacent to the image sensor (1) and comprising said first surface (21) facing said image sensor.
9 • Camera device according to any of the previous claims 1 to 6, characterized in that diaphragms (20) are provided within said transmitting element (2) for assignment of a determined region of pixels of the image sensor (1) for each transparent re- gion ( 30 ) .
10. Camera device according to any of the previous claims 1 to 6, characterized in that a first representation image (I',
11, 12) on the image sensor (1) does not interfere with a neighboring second representation image (I', II, 12), so that non-overlapping images will be provided.
11. Camera device according to any of the previous claims 1 to 6, characterized in that a first representation image (I',
11, 12) on the image sensor (1) does interfere partially with a neighboring second representation image (I' , II, 12) , so that overlapping images will be provided.
12. Use of a device according to any of the previous claims, characterized in that the device is being used for image acquisition of an eye (E) , in particular of an iris of a human eye.
PCT/EP2009/050430 2008-01-18 2009-01-15 Camera device for image acquisition of flat or nearly flat objects WO2009090217A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
EP08150418.5 2008-01-18
EP08150418 2008-01-18

Publications (1)

Publication Number Publication Date
WO2009090217A1 true WO2009090217A1 (en) 2009-07-23

Family

ID=39321341

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2009/050430 WO2009090217A1 (en) 2008-01-18 2009-01-15 Camera device for image acquisition of flat or nearly flat objects

Country Status (1)

Country Link
WO (1) WO2009090217A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3159825A4 (en) * 2014-06-20 2018-02-14 Vkansee Technology Company Ltd. Image acquisition apparatus, terminal device, liquid crystal terminal device and image acquisition method
US20230138710A1 (en) * 2020-07-16 2023-05-04 Panasonic Intellectual Property Corporation Of America Multi-pinhole camera and image identification system

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1999026419A1 (en) * 1997-11-14 1999-05-27 Tangen Reidar E Optoelectronic camera and method for image formatting in the same
DE19917890A1 (en) * 1999-04-20 2000-11-30 Siemens Ag Low-profile image acquisition system
EP1239519A2 (en) * 2001-03-05 2002-09-11 Canon Kabushiki Kaisha Image pickup model and image pickup device
WO2005069607A1 (en) * 2004-01-20 2005-07-28 Frauhofer-Gesellschaft Zur Förderung Der Angewandten Forschung E.V. Image recording system, and use thereof

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1999026419A1 (en) * 1997-11-14 1999-05-27 Tangen Reidar E Optoelectronic camera and method for image formatting in the same
DE19917890A1 (en) * 1999-04-20 2000-11-30 Siemens Ag Low-profile image acquisition system
EP1239519A2 (en) * 2001-03-05 2002-09-11 Canon Kabushiki Kaisha Image pickup model and image pickup device
WO2005069607A1 (en) * 2004-01-20 2005-07-28 Frauhofer-Gesellschaft Zur Förderung Der Angewandten Forschung E.V. Image recording system, and use thereof

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
BROWN C: "Multiplex imaging with multiple-pinhole cameras", JOURNAL OF APPLIED PHYSICS, AMERICAN INSTITUTE OF PHYSICS. NEW YORK, US, vol. 45, no. 4, 1 April 1974 (1974-04-01), pages 1806 - 1811, XP002436598, ISSN: 0021-8979 *
LIEBE C C: "Solar Compass Chip", IEEE SENSORS JOURNAL, IEEE SERVICE CENTER, NEW YORK, NY, US, vol. 4, no. 6, 1 December 2004 (2004-12-01), pages 779 - 786, XP011122172, ISSN: 1530-437X *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3159825A4 (en) * 2014-06-20 2018-02-14 Vkansee Technology Company Ltd. Image acquisition apparatus, terminal device, liquid crystal terminal device and image acquisition method
US10083335B2 (en) 2014-06-20 2018-09-25 Vkansee Technology Company Ltd. Image acquisition apparatus, terminal device, liquid crystal terminal device and image acquisition method
US20230138710A1 (en) * 2020-07-16 2023-05-04 Panasonic Intellectual Property Corporation Of America Multi-pinhole camera and image identification system

Similar Documents

Publication Publication Date Title
US9772432B2 (en) Diffraction gratings producing curtains of minimum intensity separated by foci and extending to a focal plane
US20190191993A1 (en) Systems and Methods for Lensed and Lensless Optical Sensing of Binary Scenes
CN103472592B (en) A kind of fast high-throughout polarization imaging method of illuminated and polarization imager
EP3129813B1 (en) Low-power image change detector
US9876043B2 (en) Optical sensing with tessellated diffraction-pattern generators
US7009652B1 (en) Image input apparatus
US9204067B2 (en) Image sensor and image capturing apparatus
KR20190020730A (en) Optical Field Imaging Device and Depth Acquisition and 3D Imaging Method
JP5914055B2 (en) Imaging device
CN201043890Y (en) Optical imaging distance measuring device for single-aperture multiple imaging
US9746593B2 (en) Patchwork Fresnel zone plates for lensless imaging
US10176562B2 (en) Optical flow sensing and pattern recognition with anti-symmetric phase gratings
KR20080089604A (en) Image detection system and method for producing at least one image detection system
CN101960861A (en) Sensor with multi-perspective image capture
CN1702452B (en) Digital micromirror multi-target imaging spectrometer device
US10274652B2 (en) Systems and methods for improving resolution in lensless imaging
RU2580870C2 (en) High-resolution imaging system
WO2010119447A1 (en) Imaging system and method
JP2002191060A (en) Three-dimensional imaging unit
WO2009090217A1 (en) Camera device for image acquisition of flat or nearly flat objects
WO2020037650A1 (en) Imaging assembly, touch screen, camera module, smart terminal, cameras, and distance measuring method
CN109348149A (en) A kind of display methods of display and image with imaging function
CN210836126U (en) Photosensitive pixel, image collector, fingerprint collection equipment and display equipment
CN114609619B (en) Optical positioning system and method
CN112528708A (en) Photosensitive pixel, image collector, fingerprint collection equipment and display equipment

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 09702987

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 12.11.2010)

122 Ep: pct application non-entry in european phase

Ref document number: 09702987

Country of ref document: EP

Kind code of ref document: A1