Description
Method and Apparatus for determining the position of a noise-like pattern within an image
Technical Field
The invention relates to a method and to an apparatus for determining the position of a noise-like pattern within an image, based on cross-correlation using micro noise patterns.
Background
Documents, including pictures and in both electronic or printed form, can be authenticated or watermarked or certified or registered using two-dimensional patterns. These two-dimensional patterns may be, among others, a so-called symbology, e.g., a barcode or a Data Matrix code, a noise- like pattern, or an image. A noise-like pattern may be a pattern that has little or no discernible structure or information content to the observer without additional knowledge about the generation of the noise-like pattern, while an observer with such knowledge may be able to discern such structure or information from the pattern; this includes noise patterns in the strict sense.
Once such a pattern has been discriminated from other elements in the electronic document or a captured image of its printed form, it may be necessary to precisely locate some point or points of interest within or in the vicinity of the pattern in order to facilitate an accurate restoration and/or evaluation of the pattern. This process is called "fine registration". For example, such a pattern might be
rectangular in shape as is illustrated in Fig. 1, wherein a noise-like pattern has been arranged within a black frame, FR, conventionally required for registering the pattern.
The coarse position of a two-dimensional symbology is specified using four coordinate pairs as shown in Fig. 2: (xθ,yθ) top left corner, (xl,yl) top right corner, (x2,y2) bottom right corner, and (x3,y3) bottom left corner.
Following the coarse registration step/stage, a corresponding rectangular window W describes or defines the position of the cropped image containing the two-dimensional symbology.
The fine registration can be carried out by applying corner filters to local search areas around the coarse coordinate pairs. Such processing will result in four new coordinate 35 pairs {xθ',yθ'), (xl',yl'), (x2',y2') and (x3',y3'), which represent the (fine) coordinates of the symbology for referencing in subsequent image processing steps.
Summery of Invention
However, the accuracy of the coordinates determined in such way is not very precise due to e.g. rotations or, more generally, geometric distortions of the symbology.
A desire in the technical field concerned is to provide more accurate coordinates allowing to define the exact position of or within a two-dimensional symbology. According to an embodiment a method for determining the position of a noise- like pattern within an image, wherein said noise-like pattern can be distorted with respect to a corresponding predetermined original noise-like pattern, which original
noiselike pattern includes at predetermined positions a number of N micro noise patterns (MNP) , said method including the steps:
- carrying out within said image a coarse search (CS) for said noise-like pattern, resulting in corresponding coordinates (xθ,yθ; xl,yl; x2,y2; x3,y3) describing a window (W) for the coarse position of said noise-like pattern within said image;
- based on said window (W) , carrying out a fine search (FS) for said micro noise patterns (MNP) by shifting a small window containing said micro noise pattern over said window (W) and performing, for each candidate position of a micro noise pattern, a cross-correlation or a quantitative matching between the corresponding pixel areas, and by determining the N best matches from said cross-correlation or quantitative matching result values;
- selecting based on said N best matches the locations of said N micro noise patterns (MNP) ;
- based on the locations of said N micro noise patterns (MNP) , determining the exact position of said noise-like pattern within said image.
According to an embodiment an apparatus for determining the position of a noise-like pattern within an image, wherein said noise-like pattern can be distorted with respect to a corresponding predetermined original noise-like pattern, which original noise-like pattern includes at predetermined positions a number of N circular micro noise patterns (MNP) , said apparatus including:
- means (CS) being adapted for carrying out within said image a coarse search for said noise-like pattern, resulting in
corresponding coordinates (xθ,yθ; xl,yl; x2,y2; x3,y3) describing a window (W) for the coarse position of said noise-like pattern within said image;
- means (FS) being adapted for carrying out, based on said window (W) , a fine search for said micro noise patterns
(MNP) by shifting a small window containing said micro noise pattern over said window (W) and performing, for each candidate position of a micro noise pattern, a cross-correlation or a quantitative matching between the corresponding pixel areas, and by determining the N best matches from said cross-correlation or quantitative matching result values, and for selecting based on said N best matches the locations of said N micro noise patterns (MNP) , and for determining, based on the locations of said N micro noise patterns (MNP) , the exact position of said noise-like pattern within said image.
A predetermined original two-dimensional symbology is enriched by adding preferably at least three predetermined small noise-like patterns called micro noise pattern (denoted MNP) at predetermined positions, which MNPs are used for improving the symbology registration. The micro noise pattern may be a pattern significantly smaller in area than the hosting pattern, and whose image statistics have been adjusted such that the micro noise pattern is not easily discernible within the hosting pattern to the observer without detailed a priori knowledge of the micro noise pattern, for example knowledge of the generation of the micro noise pattern. The micro noise pattern may be part of the hosting noise-like pattern. One fiducial or a multitude of fiducials of hosting noise-like noise pattern may be integrated into the mirco noise patterns
of the hosting noise-like pattern. The fiducial or the multitude of fiducials may be formed such that the fiducial or the multitude of fiducials is/are nearly unremarkable within the overall impression of the hosting noise-like pattern for the observer, for example for the human eye. Advantageously, the black border or frame FR usually surrounding the symbology for supporting the registration is no more mandatory.
In principle, the inventive method is suited for determining the position of a noise-like pattern within an image, wherein said noise-like pattern can be distorted with respect to a corresponding predetermined original noise-like pattern, which original noise-like pattern includes at pre- determined positions a number of N circular micro noise pat terns, said method including the steps:
- carrying out within said image a coarse search for said noise-like pattern, resulting in corresponding coordinates describing a window for the coarse position of said noise- like pattern within said image;
- based on said window, carrying out a fine search for said micro noise patterns by shifting a small window containing said micro noise pattern over said window and performing, for each candidate position of a micro noise pattern, a cross-correlation or a quantitative matching between the corresponding pixel areas, and by determining the N best matches from said cross-correlation or quantitative matching result values;
- selecting based on said N best matches the locations of said N micro noise patterns;
- based on the locations of said N micro noise patterns, determining the exact position of said noise-like pattern within said image.
In principle the inventive apparatus is suited for determining the position of a noise-like pattern within an image, wherein said noise-like pattern can be distorted with respect to a corresponding predetermined original noise-like pattern, which original noise-like pattern includes at pre determined positions a number of N circular micro noise patterns, said apparatus including:
- means being adapted for carrying out within said image a coarse search for said noise-like pattern, resulting in corresponding coordinates describing a window for the coarse position of said noise-like pattern within said image; - means being adapted for carrying out, based on said window, a fine search for said micro noise patterns by shifting a small window containing said micro noise pattern over said window and performing, for each candidate position of a micro noise pattern, a cross-correlation or a quantitative matching between the corresponding pixel areas, and by determining the N best matches from said cross-correlation or quantitative matching result values, and for selecting based on said N best matches the locations of said N micro noise patterns, and for determining, based on the locations of said N micro noise patterns, the exact position of said noise-like pattern within said image.
Advantageous additional embodiments of the invention are disclosed in the respective dependent claims.
Drawings
Exemplary embodiments of the invention are described with reference to the accompanying drawings, which show in:
Fig. 1 two-dimensional symbology (noise-like pattern) with black frame;
Fig. 2 cropped image containing a two-dimensional symbology, following coarse registration;
Fig. 3 a sample micro noise pattern having a circular shape;
Fig. 4 circular noise-like pattern with embedded square noise-like pattern surrounded by four micro noise patterns;
Fig. 5 partial map of Europe noise-like pattern with three embedded micro noise patterns;
Fig. 6 noise-like pattern including multiple (distributed) micro noise patterns;
Fig. 7 simplified block diagram of the inventive apparatus;
Fig. 8 a flow-chart of a method for generating a noise-like pattern according to an embodiment.
Exemplary embodiments
According to the invention, the original two-dimensional symbology is enriched by adding preferably at least three small noise-like patterns called micro noise patterns MNP at known positions. A corresponding example MNP is shown in Fig. 3. It has a circular shape and contains a limited degree of noise so as to make it for humans less perceptible
or visible within the picture. It is not necessary that the MNPs are circular, but circular MNPs have the advantage that they are rotation-invariant and that their use facilitates a faster and easier processing according to the invention.
Non-circular MNPs may be used if only correspondingly limited types of picture distortions need to be compensated for.
Fig. 4 demonstrates the application of four MNPs within a circular noise-like pattern or symbology, and Fig. 5 demonstrates the application of three MNPs within a map- shaped noise-like pattern or symbology.
The predetermined noise-like pattern (i.e. the symbology) and the small image representing the MNP and the number of embedded MNPs are the mandatory inputs to the inventive noise-like pattern registration processing. The coarse registration processing uses methods known in the art, such as block matching or histogram matching or texture searching, for finding one or more rectangular windows W in which the noise-like pattern may be located. For example in block matching, a search is carried out for finding among all possible candidate positions the minimum sum of absolute (pixel) differences between the noise-like pattern and the corresponding section in the captured image. Regarding the fine registration processing, the captured image section (window W) containing the symbology is processed by (e.g. pixel wise and line wise) shifting a small window (denoted MNP window) containing the MNP over the entire image section in order to find corresponding matches. The size (width and height) of the MNP window is dependent on the size of the MNP. In case it is known that there is a higher probability of finding a match in pre-determined or known areas within
the image section, the shifting can begin in such areas. The fine registration processing calculates for each possible candidate MNP window position within the current window W a quantitative match value (e.g. by computing the cross- Correlation corrected for the average value of each of the two signals) of the MNP window with the corresponding candidate section of current window W as defined by the corresponding coordinate pairs (cf. Fig. 2) . The magnitude of each resulting match value is examined in order to find the best n matches (i.e. peaks in the match values in the match value histogram) , where N is the number of expected MNPs in the captured image .
In case of calculating match values by using cross- correlation, the magnitudes of the cross-correlation results can be used to support inverted symbologies, too (black-white and white->black) . Optionally, the signs of the cross- correlation values can be used to check for inverted symbologies .
The best N matches are returned as the precise coordinates of the MNPs.
Based on the predetermined positions of the MNPs in the original noise-like pattern, the regained or received noiselike pattern can be processed correctly. For example, it can be positioned correctly or distortions can be removed, so that the information or the data represented by the noiselike pattern can be reconstructed.
Fig. 6 illustrates an advantageous application of the MNPs. Multiple MNPs can be equally distributed over a noise-like pattern. For example, the outer MNPs are used to register
the noise-like pattern and the inner MNPs are evaluated for determining numeric or spatial inaccuracies during the restoration of a captured noise-like pattern to a regular grid, by using (the positions of the MNPs for) an affine/perspective inverse warp processing (known as quadrilateral to square' processing), i.e. an inverse mapping from the distorted picture to the undistorted picture based on the MNP ' s reference coordinates.
Usually, an inverse warp method processes a quadrilateral from the top, left to the bottom, right corner. The numeric or spatial inaccuracies will increase in the same direction, resulting in sub-pixel or even pixel shifts.
Once the current positions of all MNPs are determined by the cross-correlation processing described above, any (sub-)pixel shifts can be detected by examining (i.e. comparing with the original positions) the positions of the MNPs, which may be slightly shifted by the inverse warp processing, too. Furthermore, instead of using one inverse warp run over the entire image, several inverse warps can be applied to the tiles surrounded by MNPs in order to increase the numeric accuracy of the overall restoration.
In Fig. 7 the (scanned) image is stored in a memory MEM. Corresponding image data are fed to a coarse search or registration step or stage CS which searches as described above in the stored image data for the coarse position of the noise-like pattern. As a result, e.g. the four coordinate pairs (xθ,yθ), (xl,yl), (x2,y2) and (x3,y3) describing window W are used for feeding the data related to window W (i.e. basically the noise-like pattern data) from memory MEM to a fine search or registration step or stage FS for per
forming the cross-correlation processing with the MNP data as described above. The MNP data and the quantity of MNPs within the current noise-like pattern can be provided from memory MEM and/or from a controller step or stage CTRL that controls the operation of MEM, CS, CF and an optional in verse warping step/stage IW, which receives from FS and possibly from MEM the data required for performing the processing described in connection with Fig. 6.
Advantageously, use of a black border supporting the registration of the noise-like pattern is no more required in the inventive processing.
The invention facilitates: - increase of the image registration accuracy by using correlation with MNPs;
- removal of the black frame that is otherwise mandatory for filter-based registration;
- rotation invariance through the use of MNPs with circular shape;
- support for arbitrary shapes for the noise-like pattern to be registered, i.e. the noise-like patterns no more need to be rectangles or squares.
Figure 8 shows a flow-chart of an exemplary embodiment of a method for generating a noise like pattern. Other methods may by implemented as well.
In step 101 an input is provided. For example, the input comprises a sequence of random and/or pseudo-random integers or a pattern.
In step 102 the input is encoded, for example a key is used as a seed for encoding the input. A second sequence of random and/or pseudo-random integers is generated dependent on the key out of the sequence of random and/or pseudo-random integers or the pattern. In the exemplary embodiment, the generation of the second sequence may depend on e . g. ' randn ' (gaussian N (0, I)), 'rand' (equiprobable distribution) , ' randint (binary +1 or-1 distribution), or MD5, SHA algorithms (0-255 integer number) .
In step 103 the output of step 102 is provided, for example the second sequence of random and/or pseudo-random integers as a noisy sequence of integers.
In step 104 the output is converted into pixel values, for example gray-scale pixel values or colored pixels. A large part of the value of the pixels making up the pattern is apparently randomly determined. Thus, a noise-like pattern may be output .
The noise-like pattern comprises in at least a part of its frequency spectrum an equal distribution or a substantially equal distribution of the frequencies. There may be no significantly dominant frequency in at least said part of the frequency spectrum.
The generation of a mirco noise pattern may be carried out such that the image statistics of the mirco noise pattern and the image statistics of the hosting noise-like pattern are preferably similar. The form of the mirco noise pattern, for example ring-like or rectangular, may be set dependent on expected distortion during usage.