WO2015193152A1

WO2015193152A1 - Method for detecting a viewing-angle-dependent feature of a document

Info

Publication number: WO2015193152A1
Application number: PCT/EP2015/062943
Authority: WO
Inventors: Andreas Hartl; Dieter Schmalstieg; Olaf Dressel
Original assignee: Bundesdruckerei Gmbh
Priority date: 2014-06-17
Filing date: 2015-06-10
Publication date: 2015-12-23
Also published as: EP3158543B1; DE102014108492A1; EP3158543A1

Abstract

The invention relates to a method (100) for detecting a viewing-angle-dependent feature of a document using a photographic camera, wherein the viewing-angle-dependent feature has viewing-angle-dependent representations, with capture (101) of a first image of the document by the photographic camera in a first spatial orientation of the document relative to the photographic camera, in order to obtain a first document image, capture (103) of a second image of the document by the photographic camera in a second spatial orientation of the document relative to the photographic camera, in order to obtain a second document image, and capture (105) of an image difference between the first document image and the second document image, in order to detect the viewing-angle-dependent feature of the document.

Description

METHOD FOR DETECTING AN INDICATOR-RELATED CHARACTERISTIC

A DOCUMENT

The present invention relates to the field of detection of viewing-angle-dependent features, in particular of holograms, on documents.

Viewing angle dependent features, such as holograms, are used for a variety of applications to enable verification of authenticity or authenticity of documents. For example, viewing-angle dependent features are applied to identification documents or banknotes to make it difficult to copy the documents. Viewpoint-dependent features can have viewing angle-dependent representations.

A verification of a viewing-angle-dependent feature for checking the authenticity or authenticity of a document can be performed manually by a person. The detection of the viewing angle-dependent feature of the document is done visually by the person. Subsequently, the viewing angle-dependent feature can be visually verified by the person, for example by a visual comparison of the representations of the viewing angle-dependent feature with previously known reference representations. Detection and verification of a viewing-angle-dependent feature by a person is usually very time-consuming.

Therefore, the use of electronic assistance systems is of particular interest for the verification of a viewing-angle-dependent feature of a document. For this purpose, it is desirable to perform a detection of the viewing angle-dependent feature of the document in an automated manner. However, there are no methods that allow detection of the viewing angle dependent feature of the document in an automated manner.

In A. Hartl, J. Grubert, D. Schmalstieg, and G. Reitmayr, "Mobile interactive hologram verification", in ISMAR, pages 75 - 82, 2013, an approach for the mobile verification of holograms is described.

In A. Hartl, and G. Reitmayr, "Rectangular target extraction for mobile augmented reality applications", in Proceedings of the International Conference on Pattern Recognition, pages 81-84, 2012, an approach for the extraction of rectangular objects is described. It is therefore the object of the present invention to provide an efficient concept for detecting a viewing angle dependent feature of a document.

This object is solved by the features of the independent claims. Advantageous forms of further development are the subject of the dependent claims, the description and the drawings.

The invention is based on the finding that the above object can be achieved by capturing images of the document in different spatial positions relative to the document and by determining an image difference between the captured images. By capturing the images of the document in different spatial positions, it is achieved that the viewing angle-dependent feature in the images of the document has different viewing angle-dependent representations. By determining the image difference between the captured images, image areas with strong optical changes can be efficiently assigned to the viewing-angle dependent feature.

According to a first aspect, the invention relates to a method for detecting a viewing-angle-dependent feature of a document using an image camera, wherein the viewing angle-dependent feature comprises viewing angle-dependent representations, with a capture of a first image of the document by the image camera in a first spatial position of the document relative to the image camera to obtain a first document image, capturing a second image of the document by the image camera in a second spatial position of the document relative to the image camera to obtain a second document image, and detecting an image difference between the first document image and the second document image, to detect the viewing angle dependent feature of the document. Thereby, the advantage is achieved that an efficient concept for detecting a viewing angle-dependent feature of a document can be realized.

The viewing angle-dependent feature may have viewing angle-dependent representations and / or lighting angle-dependent representations.

The document may be one of the following documents: an identity document such as an identity card, a passport, an access control card Permission card, business card, tax stamp, ticket, birth certificate, driver's license, car pass, or a form of payment, such as a bank card or credit card. The document may further comprise an electronically readable circuit, such as an RFID chip. The document may be single-layer or multi-layered as well as paper and / or plastic-based. The document may be constructed of plastic-based films which are joined together to form a card body by means of gluing and / or lamination, the films preferably having similar material properties.

The first spatial position of the document relative to the image camera may include an arrangement and / or inclination of the document relative to the image camera. The first spatial position may comprise a six degree of freedom pose, where three degrees of freedom may be associated with the assembly, and wherein three degrees of freedom may be associated with the inclination, including, for example, translation and rotation. The second spatial position of the document relative to the image camera may include an arrangement and / or inclination of the document, for example comprising a translation and a rotation, relative to the image camera. The second spatial position may include a six-degree of freedom pose, where three degrees of freedom may be associated with the assembly, and wherein three degrees of freedom may be associated with the inclination, including, for example, translation and rotation.

The first document image may be a color image or a grayscale image. The first document image may include a plurality of pixels. The second document image may be a color image or a grayscale image. The second document image may include a plurality of pixels. The first document image and the second document image can form a picture stack.

The image difference between the first document image and the second document image may be detected based on the plurality of pixels of the first document image and the plurality of pixels of the second document image.

According to one embodiment, the method comprises capturing a plurality of images of the document by the image camera in different spatial locations of the document relative to the image camera, wherein capturing the first image of the document comprises selecting the first image from the plurality of images in the first Spatial location, and wherein capturing the second image of the document comprises selecting the second image from the plurality of images in the second spatial location. As a result, the advantage is achieved that the first document image and the second document image can be detected in different spatial positions. The capturing of the plurality of images of the document may include determining a respective spatial location based on a respective image. The respective spatial locations may be compared to the first spatial location to select the first image from the plurality of images. The respective spatial locations may be compared to the second spatial location to select the second image from the plurality of images. The first spatial position and the second spatial position can be predetermined.

According to one embodiment, detecting the first image of the document further comprises perspective equalization of the first document image based on the first spatial position, and detecting the second image of the document further comprises perspective equalizing the second document image based on the second spatial position. This achieves the advantage that the image difference can be detected efficiently.

By the perspective equalization of the first document image, a rectangular-shaped first document image can be provided. By the perspective equalization of the second document image, a rectangular second document image can be provided.

The perspective equalization of the first document image may include scaling the first document image. The perspective equalization of the second document image may include scaling the second document image.

According to one embodiment, the method further comprises determining the first spatial position of the document relative to the image camera on the basis of the first document image and / or determining the second spatial position of the document relative to the image camera on the basis of the second document image. As a result, the advantage is achieved that the first spatial position and / or the second spatial position can be used to select the first image and / or the second image from the plurality of images. The first spatial position and / or the second spatial position can also for perspective equalization of the first document image and / or the second document image are used.

Determining the first spatial position and determining the second spatial position may include determining a respective homography.

According to one embodiment, the respective spatial position is determined by means of edge detection. As a result, the advantage is achieved that the respective spatial position relative to rectangular documents can be determined efficiently. The edge detection may comprise a detection of lines, rectangles, parallelograms or trapezoids in the first document image and / or in the second document image. The edge detection can be performed using a Hough transform. According to one embodiment, the respective document image is low-pass filtered for noise reduction. This achieves the advantage that the image difference can be detected efficiently.

The low-pass filtering can be performed by means of a windowed average filter or a windowed Gaussian filter. The low-pass filtering may further comprise determining a respective integral image of the respective document image, wherein the low-pass filtering may be performed using the respective integral image. According to one embodiment, the first document image is compared to the second document image to determine an orientation of the first document image relative to the second document image, wherein the first document image and the second document image are aligned with respect to each other based on the particular orientation. This provides the advantage that the image difference can be efficiently determined.

The comparing the first document image with the second document image may include extracting and comparing image features of the first document image and the second document image. The image features may be, for example, BRISK image features or SURF image features. According to one embodiment, determining the image difference between the first document image and the second document image comprises determining a difference image based on the first document image and the second document image, the difference image indicating an image difference between the first document image and the second document image. This provides the advantage that the image difference can be displayed efficiently based on the difference image.

The difference image can be a grayscale image. The difference image may include a plurality of pixels. The difference image can also be assigned to a picture stack.

According to one embodiment, an average is determined from a first pixel value of a pixel of the first document image and a second pixel value of a pixel of the second document image, wherein a first deviation of the first pixel value from the average value is determined, wherein a second deviation of the second pixel value from the average value and wherein the image difference is detected based on the first deviation and the second deviation. This achieves the advantage that the image difference can be detected efficiently.

The first pixel value and / or the second pixel value may be gray level values. The mean can be an arithmetic mean or a median. The deviation may be a quadratic deviation or an absolute deviation. According to an embodiment, a first document image mask is determined based on the first document image, wherein a second document image mask is determined based on the second document image, and wherein the image difference is detected based on the first document image mask and the second document image mask. This provides the advantage that the image difference can be efficiently determined.

The first document image mask may include pixels having binary-valued pixel values to indicate valid and invalid pixels of the first document image. The second document image mask may include pixels having binary-valued pixel values to indicate valid and invalid pixels of the second document image. According to one embodiment, the respective document image mask displays pixels of the respective document image which are usable for detecting the image difference. As a result, the advantage is achieved that only valid pixels of the respective document image are used to detect the image difference.

For example, a pixel of a respective document image may be invalid if the pixel is associated with a portion of the document that has been incompletely captured.

According to an embodiment, the image difference is segmented into a plurality of image segments, wherein the viewing angle dependent feature of the document is detected based on at least one image segment of the plurality of image segments. As a result, the advantage is achieved that image segments can be used to detect the viewing angle-dependent feature. The image difference may be displayed by a difference image, with the difference image being segmented into the plurality of image segments.

The segmentation may be performed by means of a pixel-oriented image segmentation method, an edge-oriented image segmentation method, a region-oriented image segmentation method, a model-oriented image segmentation method, or a texture-oriented image segmentation method. The image segmentation method may include, for example, a maximally stable extremal region (MSER) method or a mean-shift method. The image segments can be contiguous image segments. According to an embodiment, an image segment measure is determined for an image segment of the plurality of image segments, wherein the determined image segment measure is compared to a predetermined image segment measure to qualify the image segment for the detection of the viewing angle dependent feature. This achieves the advantage that an image segment which has the predetermined image segment size can be used for the detection of the viewing angle-dependent feature.

The image segment measure may be an area of the image segment, a aspect ratio of the image segment, a compactness of the image segment, a pixel value of a pixel of the image segment, or a homogeneity measure of the image segment. According to one embodiment, an image segment of the plurality of image segments is associated with a first document image segment of the first document image and a second document image segment of the second document image, wherein the first document image segment is compared with the second document image segment to qualify the image segment for the detection of the viewing angle dependent feature. As a result, the advantage is achieved that a suitable image segment can be used for the detection of the viewing angle-dependent feature.

The comparison of the first document image segment with the second document image segment can be carried out by means of a normalized cross-correlation. The image segment can be qualified, for example, for the detection of the viewing angle-dependent feature if the first document image segment and the second document image segment are different.

According to one embodiment, the viewing-angle-dependent feature comprises a hologram or a printing ink with viewing angle-dependent reflection properties or absorption properties. As a result, the advantage is achieved that the viewing angle-dependent feature can be easily realized with viewing angle-dependent representations. According to a second aspect, the invention relates to a mobile device for detecting a viewing-angle-dependent feature of a document, wherein the viewing-angle-dependent feature has viewing angle-dependent representations, with an image camera, which is designed to capture a first image of the document in a first spatial position of the document relative to the image camera to obtain a first document image, and to acquire a second image of the document in a second spatial position of the document relative to the image camera to obtain a second document image, and a processor which is adapted to make a difference in image between the first document image and the image second document image to detect the viewing angle-dependent feature of the document. Thereby, the advantage is achieved that an efficient concept for detecting a viewing angle-dependent feature of a document can be realized.

The mobile device may be a mobile phone or a smartphone. The image camera can be a digital image camera. The processor can execute a computer program. The mobile device may further comprise a lighting device for illuminating the document. The illumination device may be an LED illumination device.

The method can be carried out by means of the mobile device. Other features of the mobile device result directly from the functionality of the method.

According to a third aspect, the invention relates to a computer program with a program code for carrying out the method when the computer program is executed on a computer. This provides the advantage that the process can be automated and repeatable.

The computer program may be in machine readable form. The program code may comprise a sequence of instructions for a processor. The computer program can be executed by the processor of the mobile device.

The invention can be implemented in hardware and / or software.

Further embodiments will be explained with reference to the accompanying figures. Show it:

1 is a diagram of a method for detecting a viewing-angle-dependent feature of a document according to an embodiment;

FIG. 2 is a diagram of a mobile device for detecting a viewing-angle-dependent feature of a document according to an embodiment; FIG.

3 is a diagram of a method for detecting a viewing-angle-dependent feature of a document according to an embodiment; 4 is a diagram of a detection scenario for detecting a perspective-dependent feature of a document according to an embodiment;

5 is a diagram of a plurality of captured images of the document according to an embodiment; 6 is a surface diagram of a difference image according to an embodiment;

7 shows a diagram of a difference image and a contour diagram of a segmented difference image according to an embodiment;

8 shows contour diagrams with image segments for a plurality of captured images of a document according to an embodiment; and

9 is a diagram of a plurality of spatial locations for acquiring a plurality of images of the document according to one embodiment.

1 shows a diagram of a method 100 for detecting a viewing-angle-dependent feature of a document according to an embodiment. The method 100 is performed using an image camera. The viewing angle-dependent feature has viewing angle-dependent representations.

The method 100 comprises capturing 101 a first image of the document by the image camera in a first spatial position of the document relative to the image camera to obtain a first document image, capturing 103 a second image of the document by the image camera in a second spatial position of the document relative to the image camera to obtain a second document image, and detecting 105 an image difference between the first document image and the second document image to detect the viewing-angle dependent feature of the document. The viewing angle-dependent feature may have viewing angle-dependent representations and / or lighting angle-dependent representations.

The document may be one of the following: an identity document such as a passport, a passport, an access control card, a passport, a business card, a tax stamp, a ticket, a birth certificate, a driver's license, a vehicle pass, or a means of payment such as a Bank card or a credit card. The document may further comprise an electronically readable circuit, such as an RFID chip. The document may be single-layer or multi-layered as well as paper and / or plastic-based. The document can be constructed of plastic-based films, which can be used for a Card body are joined together by means of gluing and / or lamination, wherein the films preferably have similar material properties.

The first spatial position of the document relative to the image camera may include an arrangement and / or inclination of the document, for example comprising a translation and a rotation, relative to the image camera. The first spatial location may include a six degree of freedom pose, where three degrees of freedom may be associated with the array, and where three degrees of freedom may be associated with the slope.

The second spatial position of the document relative to the image camera may include an arrangement and / or inclination of the document relative to the image camera. The second spatial position may comprise a six-degree-of-freedom pose, where three degrees of freedom may be associated with the assembly, and where three degrees of freedom may be associated with the inclination. The first document image may be a color image or a grayscale image. The first document image may include a plurality of pixels. The second document image may be a color image or a grayscale image. The second document image may include a plurality of pixels. The first document image and the second document image can form a picture stack.

The image difference between the first document image and the second document image may be detected based on the plurality of pixels of the first document image and the plurality of pixels of the second document image. FIG. 2 shows a diagram of a mobile device 200 for detecting a viewing-angle-dependent feature of a document according to one embodiment. The viewing angle-dependent feature has viewing angle-dependent representations.

The mobile device 200 includes an image camera 201 that is configured to capture a first image of the document in a first spatial position of the document relative to the image camera to obtain a first document image, and a second image of the document in a second spatial location of the document relative to capture the image camera to obtain a second document image, and a processor 203, which is formed, an image difference between the first document image and the second document image to detect the viewing angle-dependent feature of the document.

The mobile device 200 may be a mobile phone or a smartphone. The image camera 201 may be a digital image camera. The processor 203 may execute a computer program. The image camera 201 may be connected to the processor 203.

The mobile device 200 may further include a lighting device for illuminating the document. The illumination device may be an LED illumination device.

The method 100 may be performed by the mobile device 200. Further features of the mobile device 200 result directly from the functionality of the method 100. FIG. 3 shows a diagram of a method 100 for detecting a viewing-angle-dependent feature of a document according to an embodiment. The method 100 includes a step sequence 301 and a step sequence 303. The step sequence 301 is performed for each captured image. The step sequence 303 is performed once per document.

The step sequence 301 includes a step 305 of image selection, a step 307 of registering an image, and a step 309 of spatially filtering the image. A plurality of acquired images and a plurality of specific spatial locations are processed by the step sequence 301 to provide an image stack.

The step sequence 303 comprises a step 31 1 of a difference image generation and a step 313 of a segmentation and filtering. The image stack is processed by the step sequence 303 to provide the location of the features. The diagram therefore shows step sequences 301, 303 which can be carried out for a detection of a viewing angle-dependent feature, for example a hologram, per image and per document, as well as an evaluation of the image stack.

According to one embodiment, an automatic detection of viewing-angle dependent features of a document in real time using Performed images which are detected by a mobile device, such as a standard smartphone. Using a robust algorithm for capturing the document in the images, such as by tracking the document, an image stack of images of the document can be constructed and evaluated to automatically determine the location and size of viewing-angle-dependent features of the document determine. Automatic detection of both the existence and location of viewing-angle dependent features on a document can be accomplished using a mobile augmented reality (AR) arrangement. This opens up a multiplicity of application fields, for example a detection of document layouts for a subsequent classification step or an automatic modeling with verification. Due to the efficiency of the approach, this task can be performed in real time on a mobile device such as a standard smartphone. Through the use of mobile devices and the robustness of the approach, tools can be provided which allow anyone to verify the authenticity or authenticity of documents, even without advanced training.

The following is an approach to capture and track the document.

Documents are usually made of paper or cardboard and have a rectangular shape. For reasons of robustness and efficiency, the focus is on flat areas of documents. Detecting such documents with a mobile device can be a challenging task due to varying personal data on the document, due to changes in the viewing angle, due to lighting, due to unexpected user behavior, and / or due to limitations of the image camera. Consequently, multiple captured images should be evaluated for robustness, which can be achieved using a mobile augmented reality (AR) device.

First, a suitable document template can be generated which can be used for a picture-to-picture tracking or for a dedicated registration step. This can be based on an algorithm for the detection of perspective distorted rectangles, and be executed in real time on a mobile device, and thus serve as a basic building block. The user may be asked to place an image camera of the mobile device in front of a document or object and to trigger the detection. Within a predetermined region of interest (ROI) in a captured image, an edge image may be calculated using, for example, a Canny edge detector with automatic threshold selection. Image areas with textual structures can be filtered to remove noise, followed by detection of lines, for example, using a Hough transform. The detected lines can be grouped according to their coarse direction. An initial hypothesis for a rectangular area may be formed by considering pairs of line bundles, which may comprise a total of four lines, for example.

By overlapping the lines and ensuring that the intersections are in the area of interest, the number of hypotheses can be significantly reduced. A final ordered list of rectangular hypotheses can be generated by computing a support function on an extended edge image. The top candidate of the list can be selected and a homography can be computed to produce an equalized representation. The dimensions of the rectified image may be determined by averaging the pixel width and / or height of the chosen hypothesis.

The equalized image may be used to generate a planar tracking template, which may be displayed as an image pyramid at run time, and which may be tracked using natural image features. A Harris corner detector and a normalized cross correlation (NCC) may be used to align image areas over subsequent images and to provide homography between the current image and the rectified image or tracking template. A motion model can be used to estimate and predict the motion of the image camera, thus saving computational resources. Thus, the algorithm can be executed in real time on mobile devices, such as standard smartphones, and can provide a full six-degree-of-freedom (6DOF) attitude or pose for each captured image. The arrangement has the advantage of allowing interaction with previously unknown documents with any personal data. In the context of the following computer vision (CV) algorithms, knowing a current viewing angle may be advantageous as it may allow to work with equalized images and control image capture.

The following is an approach for creating a picture stack explained. To judge the existence of viewing-angle dependent features on the document, the document should be captured from multiple angles. A minimum number of n = 2 images captured from appropriate viewing angles can be used to infer the existence of such features. For reasons of robustness, however, generally more viewing angles should be used. Based on the results of the document trace, the algorithm comprises three main parts to generate an image stack: the image selection step 305, the equalization and / or the registration step 307, and the spatial filtering step 309.

The image selection in step 305 may be performed as follows. Ideally, the image stack should comprise a plurality of images with spatial locations that best utilize the variability of the viewing-angle dependent feature. For inexperienced users, this task can be challenging. Therefore, the task of image selection, in favor of repeatability and reduced cognitive burden, should not be performed by the user. The particular poses can be used to automatically select images based on a 2D orientation map. In addition, the visibility and similarity to the template can be taken into account to select suitable images.

The equalization or registration in step 307 may be performed as follows. For each image passing the selection step, an estimated homography from the tracking space location may be used to produce an equalized image. A complete set of images can thus form a stack of images of equal size. Basically, the document tracking algorithm can be robust and can successfully track the document over a wide range of viewing angles. However, portions of the document may move out of the current camera image and the images may have perspective distortions. The rectified images may therefore be incomplete and / or not ideally aligned. In an additional step, alignment adjustment may be performed using image feature extraction, windowed matching and / or homography estimation. However, this can reduce the frame rate, which may not be desirable. Since images are continuously captured and provided by the image camera, inappropriate equalized or registered images can be discarded using NCC rating, which can be computationally more efficient. Due to real-time tracking, this can be an efficient way to automatically select images.

Spatial filtering in step 309 may be performed as follows. Any new layer that is dropped onto the stack of equalized images can be spatially filtered to better deal with noise and remaining registration inaccuracies. For this task, a windowed mean value filter can be used, which can be based on an integral image calculation. Incomplete image information, such as undefined and / or black areas on equalization, may be taken into account by detecting valid image areas used in filtering using a second mask. Spatial filtering in step 309 may be performed using a predetermined window size, for example, 3x3 pixels. In the following, an approach for the detection of the viewing angle-dependent feature is explained. Unlike other effects, such as highlights, visual changes caused by viewing-angle-dependent features remain spatially constant. The approach is based on the concept of tracking changes in representations over time, using the rectified image stack as a starting point.

The algorithm for processing the image batch comprises two main parts: the step 31 1 of generating a difference image by statistically based evaluation and the step 313 of segmenting and searching a mode for producing a final detection result. In addition, an optional verification step may be performed which may use NCC calculations on the estimated location of the viewing-angle dependent feature between equalized or registered images of the image stack to discard false-positive detections. The generation of the difference image in step 31 1 can be performed as follows. The image stack can be understood as a temporal sequence for each layer (x, y). The degree of change may be evaluated by calculating an appropriate amount of deviation of a model m at the location (x, y) over the entire image stack, with respect to document image masks that may be determined in the previous step. Thus, an intermediate representation may be provided for signs of viewing angle dependency, which may also be referred to as a difference image. According to one embodiment, the viewing angle dependent feature is a hologram and the difference image is a hologram map.

In order to detect a difference in the image which can be indicated by a difference image, an average value m ₀ or a median ΓΤΗ in connection with a mean square deviation in the image space according to FIG

e ₀ (x, y) ^ (^ (a ::, ^) ^ m.) ² (1)

1 = 1 or a mean absolute deviation according to

L (x, y)

^ i (·· y) = - \ ^v d ^x > y) - ^m ( ² )

with ^m £ i ^m o, ^m i} be used in various combinations. Here, L (x, y) may denote the number of image stack layers comprising valid pixel values for the layer (x, y) corresponding to the document image masks, where vi (x, y) may designate a pixel value in layer I. In the case of the pair m _o , eo, the model generation and deviation calculation can be done directly, and require only a small amount of computational resources.

The segmentation and filtering in step 313 may be performed as follows. Dominant spatial peaks within the image difference or the difference image and adjacent image areas with large changes of comparable value or amount are to be localized. Consequently, this provides one Image segmentation task, wherein the choice of the image segmentation method can affect both the quality and the runtime.

Since the content of the difference image may depend on the type of document, using a global threshold may not be sufficient in some cases. Then locally calculated thresholds can be used, which can be additionally adjusted using global information. To save runtime, integral images can be used for filtering.

The calculated ranges can then be filtered to reduce the number of false positive detections. Criteria regarding a minimum area, a aspect ratio, and a compactness together with a minimum pixel value and / or a homogeneity for the obtained area may be used. The process of detecting a perspective-dependent feature may include detecting the document and moving a mobile device with an image camera or moving the document, and capturing images of the document and associated spatial locations. These data can then be processed and analyzed by the algorithm. In this case, a lighting device of the mobile device can be switched on or off. The illumination device can be advantageous in order to capture all relevant representations of the viewing angle-dependent feature.

The generation and updating of the image stack can be performed per image. The generation and evaluation of the difference image with an optional validation step can then be carried out. Upon successful detection of the viewing angle dependent feature of the document, the viewing angle dependent feature in an image may be highlighted using a surrounding frame or box at the appropriate location.

Real-time tracking of the document can be used to obtain registered images from a plurality of viewing angles. In this case, only the difference image can be segmented in order to obtain possible image areas, which can then be validated. As a result, a method can be realized which can be easily integrated into existing document verification applications.

4 shows a diagram of a detection scenario for detecting a perspective-dependent feature 402 of a document 401 according to an embodiment.

The diagram shows the detection of a plurality of images of the document 401 from different spatial positions. In this case, a first document image 403 in a first spatial position, a second document image 405 in a second spatial position, a third document image 407 in a third spatial position, and an Nth document image 409 in an Nth spatial position are detected.

5 shows a diagram of a plurality of captured images of the document according to an embodiment.

A first document image 403 from a first spatial position, a second document image 405 from a second spatial position, a third document image 407 from a third spatial position, and an Nth document image 409 from an Nth spatial position are shown one above the other in the form of an image stack. The first document image 403, the second document image 405, the third document image 407, and the Nth document image 409 are shown together with respective document image masks.

The document can be tracked, whereby a plurality of images of the document can be captured from different spatial positions. Based on an estimated spatial position or homography, each document image can be equalized and placed on the image stack. The captured document images may be equalized and may have a predetermined resolution.

6 shows a surface diagram 600 of a difference image according to an embodiment. The surface diagram 600 shows pixel values of the difference image as a function of a position (x, y) for a document.

For a layer of x = 130 to x = 140 and of y = 10 to y = 20, the surface plot 600 has high pixel values. Using a Image segmentation method, an image segment may be determined in the difference image associated with that region.

FIG. 7 shows a diagram 701 of a difference image and a contour diagram 703 of a segmented difference image according to an embodiment.

The diagram 701 shows pixel values of the difference image as a function of a position (x, y) for a document. The diagram 701 corresponds to a scaled intensity image. The contour plot 703 shows a segmentation result based on the difference image using an adaptive threshold comparison. An image segment for a position of x = 130 to x = 140 and of y = 10 to y = 20 is highlighted. The viewing angle-dependent feature of the document can be detected on the basis of the image segment.

8 shows contour diagrams 801, 803, 805, 807 with image segments for a plurality of captured images of a document according to an embodiment. The plurality of captured images include a first document image 403, a second document image 405, a third document image 407, and an N-th document image 409.

The first document image 403 is assigned the contour diagrams 801. The second document image 405 is assigned the contour diagrams 803. The third document image 407 is assigned the contour diagrams 805. The Nth document image 409 is assigned the contour charts 807.

Various segmentation techniques can be used, such as a Maximally Stable Extremal Regions (MSER) method or a mean-shift method. Further, to reduce the influence of reflections on the document, a highlight detector can be used, and further, inpainting can be performed. In addition, the plurality of captured images or the image stacks can be further analyzed.

The contour diagrams 801, 803, 805, 807 show a segmentation of a picture stack, for example with a layer. The document images 403, 405, 407, 409 are shown in the upper row. The middle row shows image segments, which for example be determined by means of an MSER method. The lower row shows image segments which are determined, for example, by the MSER method, using modified images of the document, for example using highlight detection and / or inpainting. Thus, an approach for automatically detecting a perspective-dependent feature of a document using a mobile device can be realized. Previously unknown documents can be detected and tracked. In addition, the position of one or more viewing angle-dependent features, if present, can be determined automatically. The detection of viewpoint dependent features may be a first step in automated testing and verification of viewpoint dependent features. The perspective-dependent features may be embedded features.

9 shows a diagram 900 of a plurality of spatial positions for capturing a plurality of images of the document according to one embodiment.

The chart 900 includes a 2D orientation map for capturing the images of the document and / or monitoring the capture of the images of the document from various angles.

Predetermined spatial positions for capturing images of the document are highlighted by dots. The predetermined spatial positions may correspond to an azimuth and elevation of the document relative to an image camera. The predetermined spatial positions can be defined in quantized and / or discretized form.

The following summarizes the overall concept.

Viewpoint-dependent features, such as holograms, can change their representations depending on the viewing direction and direction of illumination of existing light sources in the environment. Viewpoint-dependent features can be delimited from the environment in the document and / or have a limited extent in the document. For the detection of viewing angle-dependent features, a local change in the appearance with respect to the viewing angle can be used. The document should be taken from different angles. Therefore For example, a Mobile Augmented Reality (AR) device can be used for image acquisition.

Since there is no reference information, for example the document type, the area of the document should first be detected. Thereafter, a document image or an equalized document image may be passed to a tracking algorithm. Thus, information about the spatial position can be available in every single document image. Disregarding rotation around a line of sight, the acquisition of the images may be controlled with an orientation map which may indicate an angle to the x-axis and the y-axis. This can be filled according to the current spatial position or pose and ensure that the document is viewed from different angles. The extraction of the document can then be carried out by an equalization by means of the determined spatial position of the tracker. Thus, an image stack with equalized and / or registered images can be formed.

For the detection and separation of inappropriate rectified and / or registered images, an additional check can be carried out by means of a normalized cross-correlation. After completion of the acquisition process, a model can be formed from the image stack (mo, ΓΤΗ). The deviations can be fused by using each layer of the image stack by means of a deviation measure (eo, ei) to form a difference image, for example in the form of a hologram map. This difference image characterizes the document with regard to the position and extent of viewing angle-dependent features. Subsequently, it can be segmented to obtain a set of image segments. The filtered and validated image segments can represent the result of the detection.

The verification and / or validation of the image segments may reduce the number of false-positive detected viewing-angle dependent features. In this case, a respective image segment can be extracted from each layer of the image stack. Each image segment or patch can then be compared to the remaining image segments or patches by a normalized cross correlation (NCC) and classified as a match or a deviation using a threshold th _nC c. If the relative proportion above a threshold value th _va iidation is, it can be assumed that the current image segment having sufficient visual changes with a change of the viewing angle. The illustrated approach can be extended as follows. A more detailed analysis of the registered image batch can be performed. First, highlights that are caused by, for example, a lighting device or an LED light can be detected and removed. In addition, each layer of the image stack can be segmented individually using, for example, the Maximally Stable Extremal Region (MSER) method. From the obtained image segments sequences of image segments can be extracted, which can be approximately locally constant. Each sequence can then be viewed, segmented, filtered, and validated as a single difference image, such as a hologram map.

A further development of the approach is described below. Segmentation of the difference image using local adaptive thresholding with automatic selection of a suitable window size can be used to improve the scaling invariance. For filtering image segments, the determined image segment may be used in the filtering instead of a respective bounding rectangle. A characterization of the peaks detected in the previous step in the difference image can be realized by a comparison with the immediate environment in the difference image. This eliminates the verification or validation step using normalized cross-correlation (NCC) depending on the application.

A detection of viewing-angle-dependent features, for example of holograms, on unknown documents without existing reference information by means of a mobile device can be carried out. It is thus achieved that a detection of a viewing angle-dependent feature can also be carried out without knowledge of the document type or the document layout.

LIST OF REFERENCE NUMBERS

Method for detecting a viewing-angle-dependent feature

101 Capture a first image of the document

103 Capture a second image of the document

105 Capturing a difference in image

200 mobile device for detecting a viewing angle dependent feature

201 image camera

203 processor

301 -303 steps

305-313 steps

401 document

402 Viewpoint dependent feature

403 First document image

405 Second document image

407 Third document image

409 N-th document image

600 surface diagram

701 diagram

703 Contour diagram

801 -807 Contour Diagrams

900 diagram

Claims

1 . A method (100) for detecting a perspective-dependent feature (402) of a document (401) using an image camera (201), the viewing-angle-dependent feature (402) having viewing angle-dependent representations, comprising:

Capturing (101) a first image of the document (401) by the image camera (201) in a first spatial position of the document (401) relative to the image camera (201) to obtain a first document image (403); Capturing (103) a second image of the document (401) by the image camera (201) in a second spatial position of the document (401) relative to the image camera (201) to obtain a second document image (405); and

Detecting (105) an image difference between the first document image (403) and the second document image (405) to detect the perspective-dependent feature (402) of the document (401).

2. The method of claim 1, wherein the method comprises acquiring a plurality of images of the document by the image camera in different spatial positions of the document relative to the image camera; wherein capturing (101) the first image of the document (401) comprises selecting the first image from the plurality of images in the first spatial location, and wherein capturing (103) the second image of the document (401) comprises selecting the second image from the plurality of images in the second spatial position.

The method (100) of claim 1 or 2, wherein detecting (101) the first image of the document (401) further comprises perspectively equalizing the first document image (403) based on the first spatial position, and wherein the capturing (103) the second image of the document (401) further comprises a perspective equalization of the second document image (405) on the basis of the second spatial position.

4. The method of claim 1, further comprising determining the first spatial position of the document relative to the image camera based on the first document image and / or determining the second spatial image The spatial position of the document (401) relative to the image camera (201) based on the second document image (405).

5. The method (100) according to claim 4, wherein the respective spatial position is determined by means of edge detection.

6. Method (100) according to one of the preceding claims, wherein the respective document image (403, 405) is low-pass filtered for noise reduction.

The method (100) of any of the preceding claims, wherein the first document image (403) is compared to the second document image (405) to form a first document image (403)

Aligning the first document image (403) with respect to the second document image (405), and wherein the first document image (403) and the second document image (405) are aligned with respect to each other based on the particular orientation.

The method (100) of any of the preceding claims, wherein an average is determined from a first pixel value of a pixel of the first document image (403) and a second pixel value of a pixel of the second document image (405), wherein a first deviation of the first pixel value from the average value is determined, wherein a second deviation of the second pixel value from the average value is determined, and wherein the image difference is detected on the basis of the first deviation and the second deviation.

The method (100) of any of the preceding claims, wherein a first document image mask is determined based on the first document image (403), wherein a second document image mask is determined based on the second document image (405), and wherein the image difference is based on the first document image Document image mask and the second document image mask is detected.

The method (100) of claim 9, wherein the respective document image mask displays pixels of the respective document image (403, 405) usable for detecting (105) the image difference.

1 1. The method (100) of any one of the preceding claims, wherein the image difference is segmented into a plurality of image segments, and wherein the View-angle-dependent feature (402) of the document (401) is detected on the basis of at least one image segment of the plurality of image segments.

12. The method of claim 1, wherein an image segment measure is determined for an image segment of the plurality of image segments, and wherein the determined image segment measure is compared to a predetermined image segment measure to qualify the image segment for the detection of the viewing angle dependent feature ,

The method (100) of any one of the preceding claims, wherein the viewing angle dependent feature (402) comprises a hologram or a printing ink having viewing angle dependent reflection or absorption properties.

14. A mobile device (200) for detecting a viewing angle dependent feature (402) of a document (401), wherein the viewing angle dependent feature (402) comprises viewing angle dependent representations comprising: an image camera (201) configured to image a first image of the document (401 ) in a first spatial position of the document (401) relative to the image camera (201) to obtain a first document image (403) and a second image of the document (401) in a second spatial position of the document (401) relative to capture the image camera (201) to obtain a second document image (405); and a processor configured to detect an image difference between the first document image and the second document image to detect the perspective-dependent feature of the document.

A computer program comprising program code for carrying out the method (100) of any one of claims 1 to 13 when the computer program is run on a computer.