WO2013144136A1

WO2013144136A1 - Method for detecting a perspectively distorted polygonal structure in an image of an identification document

Info

Publication number: WO2013144136A1
Application number: PCT/EP2013/056389
Authority: WO
Inventors: Andreas Hartl; Gerhard REITMEYR; Olaf Dressel; Frank Fritze
Original assignee: Bundesdruckerei Gmbh
Priority date: 2012-03-29
Filing date: 2013-03-26
Publication date: 2013-10-03
Also published as: DE102012205079A1; EP2831806A1

Abstract

The invention relates to a method for detecting a perspectively distorted polygonal structure in an image of an identification document, comprising detecting (301) edges in the image in order to obtain an edge image, detecting (303) a plurality of polygonal edge structures in the edge image, determining (305) metrics for each polygonal edge structure, and selecting (307) that polygonal edge structure which has the greatest metrics as the perspectively distorted polygonal structure.

Description

A method of detecting a perspective distorted polygon structure in an image of an identification document

The present invention relates to the detection of a perspective distorted

Polygon structure, for example, a perspective distorted rectangle, in an image of an identification document.

For checking people using identification documents, such as

For example, passports or ID cards or to verify the authenticity of an identification document, is usually a picture of the

Identification document by means of a camera of a document tester

added. In addition to stationary document testers can

Document verification and mobile document testing devices, such as smartphones are used. One problem with capturing an image of an identification document by means of a camera of a mobile document validator is the perspective distortion that occurs with a tilt of the mobile document verification device relative to the camera

Identification document may arise. An identification document points

in particular a plurality of polygonal structures which are represented, for example, by edges of identification documents or by boundaries of fields arranged therein, for example data fields, and in an optical recording

can be distorted in perspective. This complicates, for example, the automatic tracking of these polygonal structures, for example, to read data stored therein.

Due to a variety of possible perspective distortions of

Identification document in an image can not do this in one

Document checker or in a tracking system.

When using a mobile document validator, the quality of the captured image also depends on the quality of the camera as well as on the outside

Environmental influences, such as reflections and the like, from. This can be the Detectability of polygonal structures in an image further affect or even lead to a mobile document tester having a camera with it

insufficient imaging properties for detection is not suitable.

For localization of polygonal structures such as rectangles, for example, the Hough transformation can be used, as described in the document C. R. Jung and R.

Schramm, Rectangle Detection Based on a Window Hough Transform, CGIP, 2004. However, this method is not robust with respect to perspective distortions. It is therefore the object of the present invention to provide an efficient concept for

Detecting a perspective distorted polygonal structure in an image of a

To create identification document.

This object is solved by the features of the independent claims. Advantageous forms of further development are the subject of the dependent claims, the description and the attached figures.

The invention is based on the finding that, in the case of a perspective distortion of a polygonal structure, originally straight edges of the polygonal structure continue to run in a straight line. Due to the perspective distortion, however, the lengths of the edges are changed, for example, shortened or lengthened. Thus, an inclined recording of a rectangle in the shape of a trapezoid appears with a long base, a shorter top and two sides running in alignment. This finding allows a particularly simple detection of, for example, originally rectangular structures by detecting perspectively distorted quadrilateral structures whose edges run in a straight line within a tolerance range, for example +/- 5%. In this way perspective distorted polygonal structures can be detected particularly efficiently and, if necessary, equalized. According to one aspect, the invention relates to a method for detecting a perspective distorted polygon structure in an image of an identification document, with detecting edges in the image to obtain an edge image, detecting a plurality of polygonal edge structures in the edge image, determining a metric for any polygonal edge structure and selecting those polygonal ones

Edge structure as the perspective distorted polygonal structure, which has the largest metric. The metrics of perspective distorted polyhedra can be compared to capture the largest metric. The edge image is an image of edges detectable in the image and may have a plurality of polygonal edge structures, which may be, for example, the edges of the identification document or a passport side, or the edges of the geometric structures depicted therein. The respective edge structure can be given, for example, by an edge course, for example by a gray level edge course.

The identification document can be one of the following documents, with or without electronics: Identity document, such as identity card, passport,

Access control card, authorization card, company card, tax stamp or ticket, birth certificate, driver's license, motor vehicle pass or means of payment. The identification document can be one or more layers or paper and / or

be plastic-based. The identification document can be constructed from plastic-based films which are joined together to form a card body by means of gluing and / or lamination, the films preferably having similar material properties. The identification document may further comprise a chip for storing data.

According to one embodiment, edge detection is performed for detecting edges, in particular by means of the Canny algorithm. In addition to the Canny algorithm, any algorithms known per se for edge detection can be used. The edge structures can be determined directly by the edges detected in this way.

According to one embodiment, the edges are subjected to a transformation to obtain transformed edges, which can be represented by lines. The transformation can be the Hough transformation, by means of which defined line images are provided as edge structures. According to an embodiment, to detect each polygonal edge structure within a parallelism region, parallel edge pairs and / or converging or intersecting within a predetermined angular range

Edge pairs detected. According to one embodiment, parallel edge pairs with edges whose distance from each other does not exceed a predetermined threshold are detected. Of the

Parallelism range can specify, for example, an angular range within which adjacent edge pairs are considered to be parallel. Of the

Parallelism range may be, for example, a range of 0 °, +/- 1 °, +/- 2 °, +/- 5 ° or +/- 10 °. As a result, parallel edge pairs, such as the parallel sides of a trapezoid, are detected.

The predetermined angular range within which the edges of an edge pair intersect may include, for example, an angular range of 89 ° to 91 ° or to from 85 ° to 95 ° or 80 ° to 100 or 50 ° to 140 °. In this way, edge pairs are detected, which meet within the angular range. Thereby, it is possible to detect corners of the distorted polygon structure, whereby it can be determined whether the distorted polygon structure is, for example, a quadrangular structure resulting from a tilted reception of a rectangle. Further, by detecting the corners, it can be determined whether the distorted polygon structure is in a certain region (ROI: Region of Interest) of the image or the identity document.

According to one embodiment, the polygonal edge structures are dilated. In this way, for example, the edge structures are perspectively enlarged in order, for example, to enable a more accurate detection of the perspectively distorted polygonal structure.

According to one embodiment, to determine the metric, a number of pixels representing the respective edge structure is detected. The pixels representing the respective edge structure are gray scale values, for example. By prior dilation of the edge structure, an even more accurate determination of the number of pixels actually representing the respective edge structure can be detected. According to one embodiment, to determine the metric, a relative number of the pixels representing the respective edge structure is detected, the relative number of the pixels representing the respective edge structure being based on a ratio between the pixel actually representing the respective edge structure and a maximum number for displaying the respective edge structure possible pixels is calculated. The maximum number of pixels possible is the number of pixels that the

Edge structure in the assumption of an uninterrupted edge course would represent.

According to one embodiment, the metric of only those becomes polygonal

Edge structures determined, which are arranged within a predetermined image section, so-called ROI. In this way, it is ensured that the focus in detecting the perspective distorted polygonal structure on those

Image section of the image is directed, in which a polygonal structure is also expected.

According to one embodiment, the perspective distorted polygonal structure is equalized by means of at least one equalization parameter, in particular equalized in perspective. The equalization parameter indicates, for example, by what length amount the respective edge should be shortened or extended.

According to one embodiment, the at least one equalization parameter is calculated on the basis of average values of lengths of mutually opposite edges. In this case, for example, lengths of opposite edges are added and that

Addition result is halved to obtain the mean.

According to one embodiment, the perspective distorted polygonal structure is equalized by means of homography.

In one embodiment, the perspective distorted polygon structure is a perspective distorted rectangle, and the polygonal edge structures are quadrangular edge structures. According to one embodiment, the edge structure is filtered to text or

Remove letter or number elements from the image. Here can

For example, a high-frequency filtering or a pattern recognition or a gray scale detection can be performed. According to a further aspect, the invention relates to a device, in particular a smartphone, for detecting a perspectively distorted polygonal structure in an image of an identification document, having an optical pick-up device for picking up the image and a processor which is configured to perform a detection of edges in the image in order to obtain an edge image, to detect a plurality of polygonal edge structures in the edge image, to determine a metric for each polygonal edge structure, and to select that polygonal edge structure as the perspective distorted polygon structure having the largest metric. The optical recording device can be, for example, a camera of a smartphone. By way of example, the processor can be set up to carry out the method according to the invention for detecting a perspective-distorted polygonal structure in an image. In one aspect, the invention relates to a computer program having a

Program code for carrying out the method according to the invention, when the program code is executed on a computer.

Further embodiments will be explained with reference to the accompanying drawings. Show it:

Fig. 1 is a schematic representation of an identification document;

FIG. 2 shows a perspective distorted image of an identification document; FIG.

3 is a flowchart of a method for detecting a perspective distorted polygon structure; 4 is a flowchart of a method for detecting a perspective distorted polygon structure in an image; and

5 shows a flowchart of a text filtering with a threshold value formation. FIG. 1 schematically shows an identification document 101 with an image of a person 103 and a text field 105. The border of the identification document 101 is a polygonal structure which is predetermined by the edges of the identification document 101 and, for example, rectangular. This means that adjacent edges or their extensions meet vertically. The same applies to the edges of the image 103 of the person forming a polygonal structure. The text field 105 may be a text field of a machine-readable zone of the identification document 101, the edges of which form a polygonal structure, for example a rectangle.

In a recording of the identification document shown in Fig. 1, the image shown in Fig. 2 201 may arise, if that for this example recording

used smartphone is held inclined relative to the document shown in Fig. 1. This results in a perspective distorted, trapezoidal image of the identification document 101 with an edge image with edge structures 203, 205 and 207, which each result from a perspective distortion of the polygonal structures 101, 103, 105.

In the perspective distortion, the polygonal structures 101, 103, 105 are distorted in perspective and thereby converted into perspectively distorted polygonal structures, which are represented by polygonal edge structures 203, 205, 207. By an edge detection, the polygonal edge structures 203, 205 and 207 can be detected. In this exemplary embodiment, therefore, each polygonal edge structure 203, 205 and 207 corresponds to a perspectively distorted polygonal structure 101, 103, 105. In this case, for example, an ROI can be predetermined in order to match the respective perspective distorted polygonal structure, for example the

perspective distorted polygonal structure 101 to detect. However, the edge image can have further edge image structures, for example curved lines, which are also detected during edge detection, and which are not shown for reasons of clarity. Thus, the polygonal edge structure 203 is determined, for example, by the edges 209, 21 1, 213 and 215. The edges 209 and 21 1 run within one

Parallelism range, for example +/- 5 °, parallel to each other. However, they have different lengths. The side edges 213 and 215, however, are aligned. Corner points of the polygonal edge structure 203 are formed by the intersections of the adjacent edges. These corner points can therefore be detected based on the edge structure. Optionally, the edge structure 209 may be converted to a line image by the Hough transform to give a more accurate one

To get edge course. Similar explanations apply to the polygonal ones

Edge structures 205 and 207.

3 shows a flowchart of a method for detecting a perspective distorted polygon structure in an image of an identification document. The method includes detecting 301 edges in the image to obtain an edge image. In this case, the Hough transformation can be carried out. Subsequently, in the step of detecting 303, a plurality of polygonal edge structures are detected in the edge image. In this case, for example, the polygonal edge structures 205 and 207 can be detected. In the subsequent step of determining 305, a metric is determined for each detected polygonal edge structure. The metric can be determined, for example, by determining the respective number of pixels or the respective relative number of pixels for the pixels representing the respective polygonal edge structure.

In the step of selecting 307, the polygonal edge structure is selected as the detected perspectively distorted polygon structure having the largest metric, for example, the largest number of pixels. Thereafter, the detected perspective distorted polygon structure can optionally be equalized by a reverse perspective distortion.

4 shows a flow chart of a method for detecting a perspective distorted polygon structure in an image according to another embodiment. The method includes detecting edges 401 401 to capture an edge image having one or a plurality of polygonal edge structures. The edge image may, for example, be in the form of an edge folder, which can be opened by means of the Canny Edge detector can be provided. Such an edge detector is described in JS Canny, A Computational Approach to Edge Detection, PAMI, 1986. In this case, an automatic threshold value selection can also be carried out, in which, for example, only edges above a predetermined brightness threshold, for example on the gray scale image, are taken into account.

In the optional subsequent step of the filtering 403 textual structures are filtered or removed from the edge structure. The filtering 403 may be text filtering and is performed to remove, for example, high frequency structures that result in falsification of the line detection. Here, for example, the interconnected components of an adaptive

threshold-weighted image are calculated, as in the example

F. Shafait, D. Keysers, T.M. Breuel, Efficient Implementation of Local Adaptive Thresholding Techniques Using Integral Images, SPIE, 2008. Here criteria such as the aspect ratio, the relative height and the amount of pixels of the respective edge structure can be taken into account.

The filtering of the edge image can be done by local adaptive thresholding on the

Grayscale image and the labeling and evaluation of regions, e.g.

individual letters, which are used to determine the aspect ratio, degree of filling and relative length of the shorter side. As a result, the number of hypotheses, ie the edge structures to be considered, can be reduced, which increases the robustness of the algorithm and reduces its complexity.

After that, optionally, a line detection 405 can be carried out, in which, for example, the Hough algorithm is used. Here, for example, a plurality of lines within an ROI can be obtained. These lines or edges are grouped in pairs using a condition regarding their course of the course. So parallel edges or parallel lines are grouped in pairs. The same applies to edges or lines which meet or intersect within a predetermined angular range and thus represent corners of a polygonal structure. In this way, a plurality of polygonal edge structures, which as hypotheses, ie as a possible perspective distorted polygonal structures, or as Models can be saved in a list. The list can be provided, for example, by means of a database.

For example, in the optional subsequent step of selection 409, it is determined whether the vertices of the edge structures are within the ROI to those

To consider edge structures, which potential perspective distorted

Represent polygonal structures.

In an optional subsequent step, the edge image can be dilated in order, for example, to take account of a specific curvature fraction. This would not take into account completely weighted lines along the current hypothesis. To calculate the weight of each hypothesis, consider the connecting lines between the four points of a hypothesis. The relative number of pixels placed along these lines on the edge image (support) is noted down to the length of the hypothesis. The hypothesis with the largest support is output as a result of the localization and can be used to equalize the enclosed area if it is a rectangular structure. A hypothesis can be determined by intersections of two pairs of lines or edges whose angle lies within the angular range and differs by no more than thangle degrees and whose distance from one another is less than a threshold value thdistance. It can be assumed that the original image is an orthogonal, undistorted image of the wanted polygon structure.

For equalization, according to one embodiment, the desired length of the rectified rectangular structure may be specified or determined as the target length. Thus, pairs of lines can be considered taking into account the minimum distance and the

Directional deviation can be determined using the thresholds thdistance and thangle. For example, 90 ° <thangle <0 °. The threshold thdistance can be determined from the dimensions of the ROI. According to one embodiment, the intersection of two pieces forms one

Hypothesis when the four intersections are within the ROI. The hypotheses can be weighted by determining the respective metric. In a subsequent step, that polygonal edge structure is selected which most likely represents the perspective distorted polygon structure. In this case, for example, metrics can be determined, as described above. In a further, optional step of the equalization 41 1, the detected perspective distorted polygonal structure is equalized.

The method can according to an embodiment for the extraction of targets, so-called.

Targets, for mobile AR applications (AR: Augmented Reality) are used.

For the detection or equalization of a perspective distorted polygon structure, a known aspect ratio may be useful for the definition of the region relevant to the search, for example an edge structure, or replace the corresponding estimate in the equalization step.

According to one embodiment, a region of interest (ROI) is understood as meaning an image region in which a search is to be made, for example, for a quadrangular polygonal structure. As a result, the robustness can be increased and the shortening of the running time of the method can be achieved.

According to one embodiment, rectangular edge structures can be rectified by rectification. An estimate of the aspect ratio of the

Origin image by averaging the lengths for each of the 2 pairs of lines involved. By using the desired target length, the width is now also defined. This determines 4 point correspondences. The equalization can be carried out, for example, on the basis of a homography estimation and inverse mapping.

To check the equalization of rectangular edge structures, a

Comparison metric, i. a measure of error to be determined. This can be

Point correspondences formed between the reference quadrilateral rref and the extracted quadrilateral rex. The relative error e is given as the ratio of the maximum deviation of the correspondences with respect to the smallest side length dref_min of the reference quadrilateral: ex max

e

e nirt

By visual inspection, an upper limit of, for example, 3.5% was determined at which a reference to the quality of usable solution can be assumed. On the assumption that the detected polygon structure represents a region used as a tracking target, it can often be

It can be assumed that the perspective distorted polygonal structure is a perspective distorted rectangular structure. In this way, an automatic rectification, i. Equalization, be performed. Here, for example, undistorted corners can be determined by averaging over the pixel width and height on the basis of the corresponding hypothesis. With this information, homographic equalization can be performed for rectification.

According to another embodiment, the degree of perspective distortion may be determined from a model rectangle. In this case, the model rectangle can be distorted in perspective in such a way that a perspective distorted model structure results, which corresponds to the detected, perspective distorted polygonal structure. By means of a backward equalization applied to the perspectively distorted polygonal structure, this can be equalized.

The method described above can be carried out, for example, by means of a stack-based or a recursive method. To the

To further increase processing speed, look-up tables can be used to indicate forbidden and allowed directions of the edge pairs.

Due to the edge detection to be performed, it is not necessary to make any assumptions about a content of a picture background or the polygon structure to be extracted or detected. Because for edge detection it is sufficient to assume that the edges visually stand out from the background. According to one embodiment, the image may be a video frame or serve as a tracking target, for example, to enable an immediate contactless interaction.

According to one embodiment, it can be assumed that the

Identification document is planar and has a rectangular boundary, which results in a square edge image. In addition, it can be assumed that an aspect ratio is known. This aspect ratio can be determined by means of an estimate. As a result, an equalization of the perspective distorted

Polygon structure, which in the case is a quadrangular structure, be particularly easily performed. The identification document is localized by the determination of edges by an edge detection with an optional subsequent line detection by means of the Hough transformation. Here, four regions can first be processed according to the ROI. These four regions are an upper region, a lower region, a left region, or a right region. After this, a selection of parallel lines and subsequently of parallel pairs of lines or edges can be carried out. After a

The current hypotheses can be intersected based on features such as edge image support, connecting lines, aspect ratio, inner angle, and

Orientation be evaluated. The support can be determined, for example, by the number of pixels along the connecting lines. Here, a metric is calculated for the individual features whose weighted sum indicates the quality of the hypothesis. The hypothesis with the maximum sum is output in each case. This information can be used together with the current aspect ratio for extraction and equalization of the identification document image.

In Fig. 5 is a flowchart of a text filtering is shown with a

Threshold 501, which may be locally adaptive, a labeling 503 and a subsequent filtering 505 are performed in which, for example, a surface 507 or an aspect ratio 509 of a character are evaluated.

For example, text filtering is based on the assumption that text areas may vary in both their nature and their spatial arrangement. Especially at

Identification documents with text parts, which are approximately the entire length of the document, such as machine-readable lines

Text areas in the edge or line detection cause problems. Text filtering is an efficient measure that merely assesses the geometry of the regions of a threshold image to filter out the textual components. This may for example be based on the assumption that the text regions are approximately square and filled to a certain extent.

LIST OF REFERENCE NUMBERS

101 identification document

103 Illustration of a person

105 text box

201 picture

203 polygonal edge structure

205 polygonal edge structure

207 polygonal edge structure

209 edge

21 1 edge

213 edge

215 edge

301 Detect

303 Capture

305 Determine

307 Select

401 Detecting edges

403 text filtering

405 line detection

407 Modeling

409 selection

41 1 equalization

501 thresholding

503 labeling

505 filtering

507 area

509 aspect ratio

Claims

1 . A method of detecting a perspective distorted polygon structure in an image of an identification document, comprising:

Detecting (301) edges in the image to obtain an edge image;

Detecting (303) a plurality of polygonal edge structures in the edge image; Determining (305) a metric for each polygonal edge structure; and

Selecting (307) that polygonal edge structure as the perspective distorted polygon structure having the largest metric.

2. The method of claim 1, wherein for detecting (301) edges edge detection, in particular by means of the Canny algorithm, is performed.

3. The method of claim 2, wherein the detected edges by means of a

Transformation, in particular the Hough transformation, into transformed edges, in particular lines, are transferred.

A method according to any one of the preceding claims, wherein for detecting (303) a polygonal edge structure parallel within a parallelism region

Edge pairs and / or within a predetermined angular range converging or intersecting edge pairs are detected.

5. The method of claim 4, wherein only parallel edge pairs with edges whose distance from each other does not exceed a predetermined threshold, are detected.

6. The method according to any one of the preceding claims, wherein the polygonal edge structures are dilated to determine the metric.

A method according to any one of the preceding claims, wherein for determining (305) the metric a number of the pixels representing the respective edge structure is detected.

8. Method according to claim 1, wherein for determining (305) the metric a relative number of the pixels representing the respective edge structure is detected, the relative number of the pixels representing the respective edge structure being based on a ratio between the actual edge structure Pixels and a maximum number of possible to represent the respective edge structure pixels is calculated.

9. Method according to one of the preceding claims, wherein the metric is determined by only those polygonal edge structures which are arranged within a predetermined image detail.

10. The method according to any one of the preceding claims, wherein the perspective distorted polygonal structure equalized by at least one equalization parameter, in particular in perspective equalized, is.

1 1. Method according to Claim 10, in which the at least one equalization parameter is calculated on the basis of average values of lengths of mutually opposite edges.

12. The method according to any one of the preceding claims, wherein the perspective distorted polygonal structure is equalized by means of a homography estimation.

13. The method of claim 1, wherein the perspective distorted polygon structure is a perspective distorted rectangle and wherein the polygonal edge structures are quadrangular edge structures.

14. A device, in particular a smartphone, for detecting a perspective distorted polygonal structure in an image of an identification document, comprising: an optical pickup device for picking up the image; and a processor configured to perform detection of edges in the image to obtain an edge image, a plurality of polygonal ones

Capture edge structures in the edge image, one metric for each polygonal one

To determine edge structure, and that polygonal edge structure than the

perspective distorted polygonal structure, which has the largest metric.

15. A computer program with program code for carrying out the method according to one of claims 1 to 13, when the program code is executed on a computer.