SE520474C2 - Methods and apparatus for identifying objects in digital images - Google Patents

Abstract

In a method for identifying objects in a digital image which is included in a sequence of images, luminance values of a current digital image are compared with at least one threshold value in order to create a current binarized image on the basis of the comparison. A quality measure for the current binarized image is also calculated, whereupon said at least one threshold value is updated on the basis of the quality measure for use in binarization of a subsequent image. The method can be realized in the form of a computer program, a computer program product, a device, and a hand -held apparatus for position determination.

Description

10 15 20 25 30 35 520 474 2 relevant structures, partly to reduce the amount of data processed in subsequent steps. Of course, it is desirable that the binaryization takes place with high accuracy, as errors are otherwise allowed to propagate in subsequent treatment steps. In most cases, unfortunately, high accuracy can only be achieved at the cost of relatively time- and memory-consuming calculations.

The above considerations must, for example, be made when calculating a position based on images of a pattern on a substrate. The pattern here contains mutually distinct symbols whose shape and / or mutual position encode said position. The images can, for example, be recorded optically by a sensor in a handheld device, for example with the shape of a pen. Such an apparatus for position determination is described, for example, in US-A-5,051,736, US-A-5,477,012, WO OO / 73983 and US-B1-6,208,771. The calculated positions reflect the movement pattern of the apparatus over the ground and can therefore be used to create an electronic version of handwritten information.

The above images can be processed in a data processing unit, such as a suitably programmed microprocessor, an ASIC, an FPGA, etc., which receives a sequence of digital grayscale images, binaryizes them for identification of the above symbols, and calculates a position based on each binary image. During binary, a threshold matrix is used that contains a threshold value for each pixel in the grass image. Each image typically contains at least about 100 x 100 pixels and typically has 8-bit luminance resolution. Registration of handwritten information should be done with a high time resolution, typically about 50-100 images per second, which is why it is difficult to combine the requirements for high accuracy in binary with the requirements for fast processing and low memory requirements, even in a specially adapted data processing unit.

In most cases, the images also contain disturbances, for example in the form of noise, blur, uneven lighting and. «.... lO 15 20 25 30 35 5 2 0 4 7 § 4: ï¿ 3 geometric distortion, which further complicates the identification of symbols or objects.

Summary of the Invention The present invention has for its object to at least partially overcome the above-related problems of the prior art.

More particularly, the invention aims to provide an improved image processing technique for identifying objects in a digital image included in a sequence of images.

It is also desirable to prescribe an image processing technique which is accurate but which can nevertheless be realized in a time and memory efficient manner.

It is also an object to teach an image processing technique that is tolerant of disturbances, such as variations in sharpness, variations in background luminance, variations in signal-to-noise ratio and perspective effects, both within each image and between different images.

These and other objects, which will become apparent from the following description, are achieved in whole or in part by a method according to claim 1, a computer program according to claim 19, a computer program product according to claim 20, a handheld apparatus according to claim 21, and an apparatus according to claim 22. Preferred embodiments are defined in the dependent claims.

The invention is based on the insight that the desired image processing technique can be achieved by a combination of a threshold algorithm and feedback control.

More specifically, a feedback control of at least one threshold parameter used at the threshold is executed to achieve a setpoint in the form of a quality measure of the images after the threshold. The existence of a feedback control makes it possible to achieve a desired resistance to the above-mentioned disturbances. Furthermore, a sufficient accuracy can be achieved even with the use of a relatively simple thresholding algorithm, which in itself can be realized in a time and memory efficient manner. According to a first aspect of the invention, this thus relates to a method of identifying objects in a digital image included in a sequence of images. The method comprises the step of comparing the luminance values of a current digital image with at least one threshold value in order to create a current binary image on the basis of the comparison. The method also comprises the steps of calculating a quality measure for the current binary image, and on the basis of the quality measure updating the said at least one threshold value for use in binary the following image.

The mentioned at least one threshold value is suitably updated on the basis of the difference between the quality measure, which forms an actual value, and a quality setpoint.

The quality measure reflects any desired property of the binary images, such as a certain size or shape of the objects. According to an embodiment, which may, among other things, be expedient in the above-mentioned position determination, the quality measure represents the area of the objects in the current binary image. Such a quality measure can for instance consist of the total object area, an object area mean value, an object area distribution, a relationship between object area and background area in the binarized image, etc. Such a quality measure, in the form of an aggregate value for a plurality of objects in the binarized image, is relatively insensitive to perspective-related image distortions. For example, a perspective means that the area of the objects increases in one part of the image and decreases in another; on average, however, the object area is essentially the same, regardless of perspective.

The above-mentioned resistance to interference is made possible by the use of a threshold matrix containing threshold values which are designated for different sub-areas of the image in question, each sub-area comprising a plurality of picture elements. Such a threshold matrix can be calculated in a fast and memory efficient manner by estimating a background luminance value and an object luminance value for each sub-area of the current image, and by updating the threshold values of the threshold matrix on the basis of said background luminance value. and said object luminance value. This threshold matrix is thus calculated on the basis of image statistics data for the sub-areas of the current image and thereby contains threshold values that are related to the overall luminance distribution in this image, both with respect to the background and the objects.

In such an embodiment, the threshold values of the threshold matrix can be updated on the basis of at least one contrast depth factor, which indicates the relative position of the threshold value between the background luminance value and the object luminance value. Binaryization is suitably regulated via this contrast depth factor.

In this case, the quality measure of the current binary image is used to determine the contrast depth factor before the next binary.

According to one embodiment, one and the same contrast depth factor is used for calculating the entire threshold matrix.

According to an alternative embodiment, a contrast depth factor is determined for each of a number of control areas in the current binaryized image, whereupon the resulting set of contrast depth factors is used to calculate the threshold matrix.

A preferred embodiment includes a control loop, in which each iteration comprises the steps of reading a current image; to estimate the contrast distribution of the current image; calculating the threshold values of the threshold matrix, based on said contrast distribution and a contrast depth factor calculated during the previous iteration; creating a current binaryized image based on the threshold matrix; to calculate a quality measure of the current binary image; to calculate an error between a setpoint and the quality measure; and calculating a new contrast depth factor based on the previous contrast depth factor and said error. The contrast distribution of the current image can, for example, be estimated on the basis of the above-mentioned background and object luminance values for different sub-areas. ... ». 10 15 20 25 30 35 520 474 6 The control loop is suitably designed to work with substantially constant poles. Such a control loop can be based on a model function which relates the quality measure to the previous contrast depth factor and which comprises at least one model operating point which is essentially common to all digital images in the sequence of images, ie for all operating conditions. Thus, a new contrast depth factor can be calculated by parameterizing the model function on the basis of the previous contrast depth factor and said model operating point. The parameters of the model function can thus be calculated for each iteration, whereupon the control parameters of the control loop can be adapted adaptively to achieve substantially constant poles. These poles can be selected to achieve the desired stability and / or response time of the control loop.

It should perhaps be pointed out that the common model working point does not have to be an actual operating point for the working control loop. The model operating point can thus be outside the control range of the control loop and only be used as a side condition for calculating the control parameters.

The model function is suitably defined at least around the setpoint. For control technical reasons, it is further preferred, but not necessary, that the model function is a linear function.

According to one embodiment, the quality measure is set equal to the setpoint when parameterizing the model function. Such an embodiment is preferable with respect to the stability of the control loop.

According to an alternative embodiment, the model function is parameterized using the quality measure calculated for the current binary image. Such an embodiment is preferable with respect to the response time of the control loop.

According to a further preferred embodiment, an intermittent update of the setpoint is also performed on the basis of the quality measure, preferably on the basis of the new contrast depth factor which is calculated on the basis of the quality measure. Such an update can take place based on a measured relationship between an optimal setpoint for different operating conditions and the associated contrast depth factor. The new contrast depth factor emitted by the control loop during each iteration can thus also be used to calculate an updated setpoint via this measured relationship.

To relate to the position determination initially discussed, the control loop can be designed to achieve a given area on the objects in each of the binary images. Such a setpoint is of course set with knowledge of both the imaging system and the original size of the objects on the substrate being imaged. In practice, however, it is difficult to translate the original size of the objects to an optimal object size in the images. For example, the image is affected by the quality of the substrate, for example with regard to the objects' absorbance / reflectance. Furthermore, the objects on the substrate may deviate from their intended original size due to errors in the printing or printing process. Alternatively, there is no unambiguous original size, for example if the substrate has been deliberately provided with position coding objects of different sizes, for example to embed the position coding pattern in graphic information, as described in the applicant's international patent application PCT / SEO1 / 71644, which is incorporated herein by this reference.

In all these cases, it is a great advantage to carry out the above-mentioned intermittent update of the setpoint, in that the control loop can be made to automatically adapt to the data.

Further aspects of the invention relate to a computer program, a computer program product, a handheld position determining apparatus, and a device for identifying objects in a digital image.

The advantages of the computer program, the computer program product, the handheld device and the device appear from the above description. The features described in connection with the method of identifying objects in a digital image are, of course, applicable to these further aspects as well.

Brief Description of the Drawings The invention is described below by way of example with reference to the accompanying drawings, which illustrate presently preferred embodiments.

Fig. 1 is a schematic view of a set of 4 X 4 objects in a position coding pattern.

Fig. 2 is a schematic view of a handheld sensor device that can be used to detect the position coding pattern of Fig. 1.

Fig. 3 is a block diagram showing parts of a device according to the invention, as well as the exchange of information between these parts.

Fig. 4 is a flow chart showing the overall steps of an inventive method of identifying objects in digital images.

Fig. 5 is a view schematically illustrating the division of a grass scale image into sub-areas for calculating a threshold matrix.

Fig. 6 is a block diagram for illustrating a control loop according to a first embodiment of the method according to the invention.

Fig. 7 is a flow chart illustrating in greater detail the first embodiment of the method according to the invention.

Fig. 8 is a diagram of the measured relationship between the mean object area Q and the contrast depth factor k for different substrates, and a model function that approximates the ratio around a setpoint value w.

Fig. 9 is a block diagram for illustrating a control loop according to a second embodiment of the method according to the invention.

Fig. 10 is a diagram of the measured relationship between optimal setpoint w @ x and associated contrast depth factor k for different operating conditions. Fig. 11 is a flow chart illustrating in detail an implementation of the second embodiment.

Description of preferred embodiments The following description is focused on position determination based on grayscale images of a position coding pattern. The position coding pattern can be of any kind, for example one of the patterns initially designated. In the following, however, the invention is exemplified in connection with the pattern described in the applicant's patent publications WO 01/16691, WO 01/26032 and WO 01/26033. with reference to Fig. 1.

This pattern is briefly described below. The position coding pattern comprises a virtual grid 10, which is thus neither visible to the human eye nor can be detected directly by a device which is to determine positions on the surface, and a plurality of markings 11, each depending on its location. , to "4". represents one of four values “1” The value of the mark 11 depends on where it is located in relation to its nominal position 12. The nominal position 12, which can also be described as a raster point, is represented by the intersection of the raster lines.

In one embodiment, the distance between the raster lines is 300 μm and the angle between the raster lines is 90 degrees.

Other raster distances are possible, eg 254 pm to suit printers and scanners, which often have a resolution that is a multiple of 100 dpi, which corresponds to a distance between points of 25.4 mm / 100, ie 254 pm.

In the example in Fig. 1, there are four possible locations, one on each of the raster lines starting from the nominal position 12. The displacement from the nominal position 12 is equal for all values. Each mark 11 is offset with its center of gravity relative to its nominal position 12, ie no mark is located in the nominal position. There is also a single mark 11 per nominal position 12. -q-n. In one embodiment, the markings 11 are offset 50 μm along the grid lines relative to the nominal positions 12. The offset is preferably 1/6 of the grid distance, since it then becomes relatively easy to determine which nominal position to which a certain mark belongs. The offset should be at least about 1/8 of the raster distance, as otherwise it can be difficult to determine an offset, ie the requirements for resolution become large. On the other hand, the displacement should be less than about 1/4 of the raster distance in order to determine belonging to the nominal position.

Each marking ll consists of a more or less circular dot with a radius which is about equal to the displacement or slightly smaller. The radius can be between 25% to 120% of the displacement. If the radius becomes much larger than the offset, it can be difficult to determine the grid lines. If the radius becomes too small, a larger resolution is needed to register the markings. However, the markings do not have to be circular or round, but can have any suitable shape, such as square, triangular, elliptical, filled, unfilled, etc.

The patterns described above can be designed to encode a very large number of absolute positions. For example, the pattern may be such that 6x6 adjacent markings together encode a position, in the form of an x-coordinate and a y-coordinate. If a subset of the pattern is applied to a product, an electronic representation of what is written or drawn on the product can be obtained with a pen by continuously determining the position of the pen on the product by reading the local combination of markings. This reading can be done by optical detection.

Fig. 2 shows a hand-held apparatus 20, hereinafter referred to as a pen, which is used for optical detection of the position coding pattern in Fig. 1. In the following, the main components of the pen are briefly described. For a more complete description, reference is made to the above-mentioned WO 01/16691, WO 01/26032 and WO Ol / 26033.

The pen 20 has a pen-shaped housing 21 which in its one short end defines an opening 22. The short end is intended to abut or be kept at a small distance from the surface on which the position determination is to take place.

One or more infrared LEDs 23 are arranged at the aperture 22 for illuminating the surface area to be imaged, and an IR-sensitive area sensor 24, for example a CCD or CMOS sensor, is arranged to register a two-dimensional image of the surface area.

The area sensor 24 is connected to a data processor 25 which is arranged to determine a position on the basis of the image registered by the sensor 14. The data processor 25 may include a processor means 25a which is programmed to process images from the sensor 24, or from a memory means 25b assigned to a sensor 24, for position determination on the basis of these images.

The processor means 25a may comprise a microprocessor, such as a CPU ("Central Processing Unit"), a DSP ("Digital Signal Processor") or any other programmable logic device, such as an FPGA. The processor means 25a may alternatively, or additionally, comprise a hardware circuit, such as an ASIC ("Application-Specific Integrated Circuit") and / or discrete analog and digital components.

The memory means 25b preferably comprises different types of memory, such as working memory (RAM), reading memory (ROM / FLASH) and write memory (FLASH). In a known manner, the working memory can store data while it is being processed by the processor means 25a, the read memory can store the program code executed by the processor means 25a in the working memory, and the write memory can store the result of the processing, such as position coordinates.

The pen 20 also has a pen tip 26 which deposits marking liquid on the substrate. Thus, the user can physically write on the substrate at the same time as what is written is digitally registered via optical detection of the positioning ... m ... n-_ 10 15 20 25 30 35 520 474 12 coding pattern. The marking liquid is suitably transparent for infrared light, while the markings 11 of the position coding pattern (Fig. 1) are absorbent for infrared light. This avoids that the marking fluid interferes with the detection of the pattern.

Thus, when the pen 20 is passed over the position coding pattern, the area sensor 24 registers a sequence of digital grayscale images which are transmitted to the positioning data processor 25. In one embodiment, the grayscale images contain 96 x 96 pixels whose luminance values are given with 8-bit resolution. To achieve adequate time resolution on the digitally registered information, images are read from the area sensor 24 at a frequency of approx. 100 Hz.

In the pictures, the markings 11 (fig. 1) appear as dark dots against a light background. Typically, each selection or object covers multiple pixels. The sharpness can vary within the image as a result of the pen 20, and thus the area sensor 24, being angled towards the substrate when writing down information. The contrast can also vary within the image as a result of uneven scattering properties of the substrate. In addition, the illumination of the surface may be uneven. Taken together, this leads to variations in sharpness, contrast, signal-to-noise ratio and illumination within each image. In addition, corresponding variations arise between different images, as the angle of inclination of the pen varies over time, and between different users and substrates, while recording information.

Fig. 3 shows a block diagram of relevant parts of the data processor according to Fig. 2. A grass scale image I is registered by means of the sensor 30 (corresponding to the area sensor 24 in Fig. 2) and transferred for storage to a memory module 31, for example the above-mentioned working memory or write memory. If necessary, a plurality of images can be buffered in the memory module 31 while waiting for decoding. A segmentation module 32 reads a grass scale image I from the memory module 31 and then thresholds it using a threshold matrix T obtained from a threshold calculation module 33. The segmentation module 10 here compares the luminance value of each pixel in the current pixel. with an associated threshold value in the threshold matrix T. If the luminance value is greater than the threshold value, the corresponding luminance value of the binary image is set to one (1), otherwise to zero (0). The resulting binary image B thus contains dark objects (value O), which ideally constitute the markings, against a light background (value 1). The binary image B is then saved in the memory module 31.

The binary image B is then read in by an analysis module 34, which calculates a quality measure Q in the image B, as will be described in more detail below. Finally, the binary image B is processed in a decoding module 35, which processes the information in the image B to decode position coordinates x, y on the basis of the positions of the objects in relation to the virtual raster. The decoding module 35 will not be described in detail herein, since the present invention is directed to the pretreatment step, more particularly the binaryization of the grayscale images I.

The incoming gray scale image I is also processed by a statistics module 36, which generates image statistics data S for given sub-areas or subregions of the current grass scale image I. This image statistics data S is conveniently stored in the memory module 31, from which the threshold calculation module it shall begin the calculation of a new threshold matrix T.

The data processor further comprises a control module 37, which reads in a setpoint w, control parameters ll, 12 and the quality measure Q calculated by the analysis module 34 and calculates a contrast depth factor k which in turn is used by the threshold calculation module 33 for calculating the threshold matrix T.

It should be understood that each of the above modules 32-37 may be implemented in the data processor 25, in the form of a software controlled processor, a specially adapted hardware circuit, discrete analog / digital components, or any combination. hence.

In the presently preferred embodiment, the threshold calculation, analysis, decoding and control modules 33-35, 37 are realized as a software-controlled microprocessor, while the segmentation and statistics modules 32, 36 are realized as an ASIC operating on large data quantities without loading the microprocessor. To further relieve the microprocessor, grayscale images are loaded from the sensor 30 to the memory module 31 via DMA (“Direct Memory Access”).

Fig. 4 shows overall process steps performed in the system according to Fig. 3. In the system, a current grayscale image #n is first retrieved, step 41. In step 42, reading of the quality measure Q (n-1) for a previous binary image and calculation of a current threshold matrix T (n) on the basis of this quality measure. In step 43, the segmentation module 32 then reads in the current threshold matrix T (n), compares this with the current gray scale image and creates a current binary image. In step 44, the analysis module 34 calculates a quality measure Q (n) of the current binary image.

After step 44, the execution returns to step 41.

In the following, the threshold calculation module 33, the analysis module 36 and the control module 37 will be described separately in greater detail.

The Threshold Calculation Module The Threshold Calculation Module 33 (Fig. 1) is designed to estimate the contrast in a number of sub-areas of the current gray scale image I and to calculate a threshold value per sub-area from the contrast. A detailed description of the threshold calculation module can be found in the applicant's Swedish patent application SE 0102254-0, which is incorporated herein by this reference. Below is a summary of the principles that form the basis for the threshold calculation module, followed by an exemplary embodiment.

The contrast within each sub-area is estimated as the difference between a background luminance value and an object luminance value, which in turn are estimated for each sub-area. Suitably, the background luminance value and the object luminance value are estimated on the basis of first-order statistics by the luminance values of the picture elements included in each sub-area. First-order statistics, for example including the smallest value, the largest value, the median value, the mean value and the sum of the luminance values of the picture elements within a sub-area, can be extracted computationally from the gray scale image I via the statistics module 36.

In the case of grayscale images with dark objects against a light background, the background luminance value can be estimated on the basis of the largest luminance value of the picture elements within each sub-area. Alternatively, the background luminance value can be estimated on the basis of the average value of the luminance values of the picture elements within each sub-area.

According to another alternative, the background luminance value is estimated on the basis of a percentile value, for example in the interval 80-95, for the luminance values within each sub-area.

Correspondingly, the object luminance value can be estimated on the basis of the smallest luminance value of the picture elements within each sub-area.

The above principle can be refined by estimating the background luminance values and the object luminance values for background sub-areas and object sub-areas, respectively, whose sizes are adapted for optimal estimation of the respective value, as will be shown in the examples below.

Fig. 5 exemplifies calculations of a threshold matrix according to the above-mentioned principles, based on grayscale images of 96 x 96 pixels. Each grass-scale image is divided into 64 (8 x 8) Islo object sub-areas, each containing 12 x 12 pixels, and 256 (16 x 16) background areas, respectively, Islb, each containing 6 x 6 pixels. The sub-areas Islo, Islb are delimited by thin lines in Fig. 5. This division is used partly by the statistics module 36 for generating image statistics data S, and partly by the threshold calculation module 33 for calculating the threshold matrix T. The division is adapted to the coding pattern in Fig. 1.

In this example, the threshold matrix is thus calculated on the basis of image statistics data for two different sets of sub-areas, partly object sub-areas and partly sub-sub-areas. The object sub-areas and the background sub-areas overlap and each cover the entire part of the image to be binaryized. The object sub-areas are so large that they certainly contain at least part of a selection. The upper limit for the size of the object sub-areas is given by the smallest acceptable resolution of the threshold matrix, which depends, among other things, on the spatial size of the luminance variations in the images. The background sub-areas, on the other hand, can be made smaller, as they only need to be so large that they certainly contain picture elements that are representative of the local background luminance of the image, ie they should be larger than each mark in the image. In doing so, any magnification as a result of perspective effects should be taken into account.

It should also be noted that the object sub-areas Islo in this example are dimensioned to comprise a whole number (in this case four) background sub-areas Islb, which facilitates the calculation of the threshold matrix T. (Fig. 1) to calculate a threshold value for each background sub-area. : Ti = bi - k * (bi - oi), where bi is the estimate of the background luminance within the background subarea Islb, and oi is the estimate of the object luminance within the larger object subarea IS, O which overlaps the current background subarea Islb. In this example, the background luminance is estimated as the largest luminance value within the background subarea and the object luminance as the smallest luminance value within the object subarea. Thus, in this embodiment, the statistics module 36 (Fig. 3) extracts image statistics data S in the form of the largest luminance value (max) and the smallest luminance value (min) within the sub-areas Islb and ISIO, respectively.

The contrast depth factor k (OS k S 1) controls the contrast depth at which the threshold value is to be set. For k = 0, the threshold value is set at the level of the background luminance value, and for k = l, the threshold value is set at the level of the object luminance value. The threshold calculation module 33 receives a current value of the factor k from the control module 37. In the described exemplary embodiment, this factor is assigned the same value for all sub-areas.

After the above calculation, the threshold matrix T contains a threshold value Ti per background sub-area Islb (Fig. 5).

The analysis module The analysis module 34 (Fig. 3) is designed to calculate the quality measure Q on the current binary image. The choice of quality measure of course depends on the type of object to be identified in the grayscale images.

When identifying the markings in the position coding pattern according to Fig. 1, it has proved appropriate to base the quality measure on the size of the objects in the binary image. The quality measure is suitably calculated for the whole image, or at least for an image area that certainly contains several objects. This minimizes the impact on the quality measure from the geometric distortion that arises when the pen, and thus the area sensor, is skewed relative to the position-coded substrate.

The analysis module 34 can thus be designed to sum the number of object pixels (value 0) in the binary image, for calculating the total object size.

The analysis module 34 may alternatively be designed to calculate an average object size in the image by summing the number of object pixels, identifying the number of groups of contiguous object pixels that can be assumed to form objects, and forming the ratio of these number values. ..... 10 15 20 25 30 35 520 474 fffçff 18 According to a further alternative, the analysis module 34 may be designed to identify all groups of contiguous object pixels that can be assumed to form objects, calculate the size of each group, and form a histogram of the size distribution of the groups or extract some measure thereof.

The control module The control module can be illustrated as part of a control system according to Fig. 6. The control module 60 (corresponding to the control module 37 in Fig. 3) operates on a process 61, which is executed in the threshold calculation module 33, the segmentation module 32 and the analysis module 34 in Fig. the module 37 is designed to calculate the contrast depth factor k on the basis of the difference e between the setpoint w and an actual value, in the form of the quality measure Q in a previous binary image. This contrast depth factor k is then used by the process 61 to calculate a current threshold matrix and with this create a current binary image from a current grayscale image.

Fig. 7 shows in more detail the process steps performed in the system according to Figs. 3 and 6. In a first step 71, image statistics S (n) are read for the current grayscale image. In step 72, which is performed in the threshold calculation module 33 as described above, a current threshold matrix T (n) is calculated based on a previous contrast depth factor k (n-1) and the image statistics S (n}. Then, in step 73, the current In the subsequent step 74, the average object size in the current binary image is calculated and used as the quality measure Q (n). In step 75, which will be discussed in more detail below, a new contrast depth factor is then calculated. k (n) Then the execution returns to step 71.

The above control system comprises a control loop of the first order, in that the new threshold matrix is calculated before the next grayscale image is binaryized. va ... lO 15 20 25 30 35 520 474 §: ;;; Fig. 8 shows the transfer function of the process 61 in Fig. 6. More specifically, a measured relationship between the average object size Q and the contrast depth factor k is shown for six different substrates with position coding patterns according to Fig. 1. Both paper quality and object size vary between the different substrates. In Fig. 8, the solid curves indicate the measuring points for the respective substrates, and the dashed lines indicate linear approximations to the measuring points. It can be seen that the transfer function can be approximated with a linear model function around a setpoint w: Q (n) = a - k (n-1) + C, where a and C are constants, and n is the time step.

In this case, the control loop can be expressed as a common PI loop: km) = al - (w (n) ~ om) + az - 2 (wrm) - omm), m = 0 where w (n) is the desired average object size, and al , as are control parameters.

By defining Ak (n) = k (n) - k (n-1) you can eliminate the summation by rewriting the control loop SOTH man) = al - (Awzn) - Adm) + az - (w (n) - Qfm) .

This results in AQm) = a-A1 <(n-1) = a-al- (Aw (n-1) -AQ (n-1)) + + a-a2- (w (n-1) -Q (n-1)) in which can be rewritten and transformed into a transfer function for a feedback system: (a-al + a-a2) -z'l-a-al-z "2 Gc (z ) = 1 + (a-al + a-a2-1) -z'l-a-al-z "2 'The poles of the transfer function become: Z = 1-a-al-a-a2iJ fi- a-al-a-azf + a-a1 2 4 The position of the poles affects the stability and response time of the control system. For example, it is a criterion for stability that all poles are within the unit circle.

It is thus possible to examine the poles with respect to the control parameters a1, az, for a given value of the constant a, and on this basis select values of the control parameters which give the desired performance of the control system.

In general, the control system should be as fast as possible, but not at the cost of too many oscillations.

A connection for the regulatory loop will now be established. If you enter e (n) = w (n) - Q (n) you get Ak (n) = k (n) - k (n - l) = al -Ae (n) + az -e (n) <ï > km) = km - 1) + a, - (em) - em - 1)) + az - em) The integration is implicitly included in the above relationship.

This avoids problems with integration distortion. In other words, if the output signal becomes saturated at time n, the saturated value can be used, and at time n + 1, the above relationship can be used again without regard to previous events.

To return to the location of the poles, this is as above dependent on the value of a, which in turn can vary greatly depending on the operating conditions of the sensor device, for example as a result of changes in properties of the substrate, such as object size, shape quality or absorbance . If al and az are set -.- .. 10 15 20 25 30 520 474 2l to constant values, these must be selected according to the highest possible value of | a | to ensure that oscillations do not occur under any conditions. Such a control loop could be useful in some cases, when one can tolerate a relatively slow response to rapid changes in the environment of the sensor device.

In an alternative embodiment, the control loop is designed to work with substantially constant poles, regardless of operating condition. This design takes advantage of the fact that the model function Q (n) = a -k (n-1) + C, for all relevant operating conditions, has a substantially independent operating point. Fig. 8 shows that all model function curves substantially converge towards an operating point (Pd: Qd, kd), in this example with an average object size around -1 for a contrast factor around 1.05. Since Q (n) = w = constant in "steady state", the value of a can be calculated instead of set to a constant: _ W_Qd In the light of the above, the control system, by this calculation of a, can be modified to have substantially constant poles : a'a1 = ß1 <:> a1 (n): ß1 'a och a az = ßz ca azü fl = ßè. kÛï; hñ: ¶'kd. d This gives the final version of the control loop for binaryization: lO 15 20 25 30 520 474 22 km) = km - 1) + alm) - (an) - em - 1)) + azm) - em) e (n) = w - Q (n) a fl n) = ßi 'k (n _ l) _ kd w - Qd az fi l) = ßz 'k (n _ 1) * kd w - Qd where ßl and ßz are selected according to the above discussions about stability and response time, w is the desired average object size after binaryization, and Q (n) is the average object size after binaryization of the image in question, and the setpoint w can be determined by testing, The optimal values of the control parameters B1, ßz as will be appreciated by those skilled in the art For example, successful tests have been performed with ß1 = 0, i.e. using a fully integrative rule erlOOp, ßz = ~ l, and w = 4, ie an average object area of 4 pixels.

As an alternative to setting the mean object size Q (n) equal to the setpoint w in the parameterization of the model function, to achieve substantially constant poles regardless of operating condition, one can use the actual mean object size after binary. This approach can potentially provide even faster rules. - ring of the current image, ie the actual value Q (n). system, however, at the cost of an increased risk of instabilities in the control system. However, in the example given above and illustrated in Fig. 8, the stability of the system does not deteriorate to any appreciable extent since the model function, even at a distance from the setpoint w, well reflects the true dependence between the mean object size Q and the contrast depth factor k.

In the embodiment according to Figs. 3-8, a setpoint is set which is assumed to be adequate for all operating conditions. However, it can be difficult to determine such a universal setpoint. In the example above, the object size in the binary images is used as a quality measure. The optimum object size, ie the setpoint, is set to .- »» -. 10 15 20 25 30 35 520 474 23 knowledge of the nominal size of the markings in the coding pattern (cf. Fig. 1) and possible magnification in the image from the substrate to the area sensor. However, it has been found that the optimal object size also depends on the operating conditions, especially properties of the substrate, such as the substrate's reflectance of IR light, the surface smoothness of the substrate, the ink absorption of IR light, the degree of blackening of the markings, etc. actual size deviate from its nominal value, for example due to disturbances in the application of the markings.

Fig. 9 shows an alternative embodiment which meets the above needs. The control system according to Fig. 9 contains a control module 90, which corresponds to the control module 60 in Fig. 6 and which operates on a process 91, which corresponds to the process 61 in Fig. 6. The control system further contains a calculation module 92 for setpoint, which intermittently updates the setpoint w based on the current contrast depth factor k output from the control module 90.

Namely, it has surprisingly been found that there is a relationship between the optimal object area in binary images for a given operating condition (with given properties of the substrate) and the corresponding contrast depth factor calculated by the control module according to the previous description. Optimal object area can refer to the setpoint that for a substrate type provides the best identification of the objects, seen over all permissible misalignments of the pen relative to the substrate. There are various possible criteria for the optimal object area, such as the least number of incorrectly identified objects, the greatest certainty when calculating positions, etc.

Fig. 10 shows a part of Fig. 8 in more detail. For each curve (solid lines), ie each substrate, the optimal working point (unfilled circle) is further indicated, ie the working point that gives an optimal object area for each substrate. Fig. 10 also shows a straight line which has been adapted to the optimal working points via .. ~ .-. 5 15 20 25 30 35 520 474 24 regression analysis and given by the function relation wqx = 5- k + y, where y = 1.4 and 6 = 4.8.

The calculation module 92 is thus based on this functional relationship in order to determine an optimal setpoint.

In some cases, for technical reasons, it may be appropriate not to update the setpoint during each iteration of the control loop. Thus, the contrast depth factor k from the control module 90 can be averaged over a number of iterations before an updated setpoint w is calculated and output to the control module 90. It may further be appropriate to allow updating of the setpoint only within a setpoint range.

It should also be pointed out that the stability criteria for the control system as a whole can be changed in relation to the control system in Fig. 6, which is why the values of the control parameters ah cb are probably different. In the case of a control system with constant poles, ßl = 0 and ßz = -0.2 have been found to give satisfactory results.

In the following, with reference to Fig. 11, an implementation of the alternative embodiment as described above is described.

First, an initiation 110 of the control module is performed. In step 111, the control module reads in start values, ie the current contrast depth factor k (O), the current error e (0) and the current setpoint w (0). Typical values are k (O) = 0.6; e (0) = O; w (O) = 4.28. The initial values k (O) and w (O) can be constant default values, or are updated each time the pen is switched off, for example with the most recently calculated value of the contrast depth factor and the setpoint, respectively.

In step 112, the control module reads in the control parameters ßh ßz- A parent counter n is reset to 1 in step 113, whereupon a main control loop is started in step 114.

In connection with this, in step 115, the threshold calculation module reads in image statistics S (n), which has previously been calculated by the statistics module for a current grayscale image I (n). Based on the image statistics S (n), the threshold calculation estimates a background matrix BG (n) and an object matrix O (n) containing background luminance values and object luminance values for given sub-areas in the grayscale image I, respectively. (n). In step 116, the threshold calculation module then calculates the threshold matrix T (n) based on the background matrix BG (n), the object matrix O (n) and the contrast depth factor k (n-1).

In step 117, the segmentation module reads in the grass scale image I (n), if it has not already done so before, and the threshold matrix T (n), whereupon the grass scale image I (n) is binaryized using the threshold matrix T (n). The result is a current binary image B (n).

Then, in step 118, the analysis module processes the binary image B (n) and calculates the average object size Q (n) within it.

In step 119, the control module reads in the calculated average object size Q (n) and calculates the difference, or error, e (n) between the current setpoint w (n-1) and the average object size Q (n). In step 120, the control modules al fi n) and 0 @ (n) calculate outputs from k (n-1), Q (n) and the control parameters ßl and ßz, respectively. In this example, Q (n) is thus used to parameterize the model function. Finally, in step 121, the control module calculates k (n) on the basis of k (n-1), a1 (n), a2 (n), e (n) and e (n-1).

The setpoint calculation module (reference numeral 92 in Fig. 9) then reads, in step 122, the updated contrast depth factor k (n) and calculates a new setpoint w (n), which is then read in by the control module.

Here, the calculation module can be designed to calculate the new setpoint as a function of an average of a number of previous contrast depth factors.

Finally, the counter n is incremented in step 123, whereupon the execution returns to step 115 for a new iteration.

Alternative embodiments The above description is only intended to give an example of how the invention can be realized within the scope of the scope of protection defined by the appended claims.

For example, the control loop can be implemented as a PID controller, ie a controller which, in addition to proportional (P) and integrative (I) control, also executes derivative (D) control. Furthermore, it is conceivable to use linear control systems of higher order than 1, as well as different types of non-linear control systems, for the control of the binaryization process.

Furthermore, it should be emphasized that the above-mentioned sub-areas can have any shape, such as square, rectangular, triangular, rhombic, hexagonal, etc.

In the above example, a contrast depth factor is calculated based on the entire binary image. However, it is conceivable to instead determine a contrast depth factor for each of a number of control sub-areas in the binary image, on the basis of a calculated quality measure for each control sub-area. The resulting set of contrast depth factors can then be used to calculate the threshold matrix.

With respect to the exemplary position coding pattern, it may be pointed out that the grid may have shapes other than orthogonal, such as a rhombic grid, for example at a 60 degree angle, a triangular or hexagonal grid, etc. Furthermore, the markings may be displaced in other directions than along the grid lines. .

However, the invention is not in any way tied to the described position coding pattern, but is also applicable to the identification and decoding of other coding patterns.

In the exemplary embodiment above, the pattern is optically readable and the sensor is thus optical. It is understood, however, that the images processed according to the invention may be generated in another way, for example by detection of chemical, acoustic, electromagnetic, capacitive or inductive parameters. It will also be appreciated that the invention may be used »..-. 520 474: riff 27 also for identifying light markings against a dark background.

Finally, it should be noted that the invention is generally useful for achieving accurate, fast and memory-efficient identification of objects in a digital image included in a sequence of images.

Claims (22)

10 15 20 25 30 35 520 474 28 PATENT REQUIREMENTS A method of identifying objects in a digital image included in a sequence of images, comprising the step of comparing the luminance values of a current digital image with at least one threshold value in order to create a current binary image on the basis of the comparison, k äI1r1e tec] to calculate a quality measure for the current binary image, and on the basis of said quality measure update the at least one threshold value for use in binary the following image. A method according to claim 1, wherein said at least one threshold value is updated on the basis of the difference between said quality measure and a quality setpoint. A method according to claim 1 or 2, wherein said comparison is performed on the basis of a threshold matrix containing threshold values designated for different sub-areas of the image in question. A method according to claim 3, comprising the steps of estimating for each sub-area of the current image a background luminance value and an object luminance value, wherein the updating of the threshold matrix threshold values takes place on the basis of said background luminance value and said object luminance value. A method according to claim 4, wherein the threshold values of the threshold matrix are updated on the basis of at least one contrast depth factor, which indicates the relative position of the threshold value between the object luminance value and the background luminance value. A method according to claim 5, wherein the contrast depth factor is determined on the basis of said quality measure. A method according to claim 5 or 6, wherein the quality measure is calculated for the control sub-areas of the current binary image, and wherein the contrast depth factor is determined for each of the control sub-areas. A method according to any one of the preceding claims, wherein the quality measure represents the area of the objects in the current binary image. ... n- 10 l5 20 25 30 35 520 474 29 A method according to any one of claims 3-8, comprising the step of forming a control loop, each iteration (s) of the control loop comprising: loading a current image (I), estimating the contrast distribution of the current image (I). calculating the threshold values of the threshold matrix (T), based on said contrast distribution and a contrast depth factor calculated during the previous iteration (k (nl)), to create a current binary image (B) on the basis of the threshold matrix (T), to calculate a quality measure (Q) on the current binary image (B), to calculate an error (e (n)) between a setpoint (w) and the quality measure (Q), and to calculate a new contrast depth factor (k (n)) based on the previous contrast depth factor (k (n-1)) and said error (e (n)). 10. lO. A method according to claim 9, wherein the control loop operates with substantially constant poles. 11. ll. The method of claim 10, wherein the control loop is based on a model function that relates the quality measure (Q) to the previous contrast depth factor (k (n-1)), which model function comprises at least one substantially common model working point (Pd) for all digital images in the sequence. of images, wherein the step of calculating a new contrast depth factor (k (n)) comprises the step of parameterizing the model function on the basis of the previous contrast depth factor (k (n-1)) and said model operating point (Pa) - 12. l2. A method according to claim 11, wherein the model function is at least defined around said setpoint (w). The method of claim 12, wherein the parameterizing the model function comprises setting the quality measure (Q) equal to the setpoint (w). ..... 5 10 15 20 25 30 520 474 30 A method according to claim 12, wherein the quality measure (Q) calculated for the current binary image (B) is used in the parameterization of the model function. A method according to any one of claims 11-14, wherein the model function is a linear function. A method according to any one of claims 11-15, wherein the control loop is given by: km) = km-1) + a, (n) - (e (n) -e (n-1)) + a2 (n) em) e (n) = wQ (n) kfn-u-kd w = -----_ aln 'ßl Qnv-Qd k (nl) -lm = --_- azn ß? own-od where ß fi, ßß are constants, and k¿, Qd are the values of the contrast depth factor and the quality measure in the said model working point (Pd). A method according to any one of claims 9-16, comprising the step of intermittently updating the setpoint on the basis of said quality measure. A method according to any one of claims 9-17, comprising the step of intermittently updating the setpoint (w) on the basis of the contrast depth factor (k (n)). 19. Computer program with program code which, when executed in a processor-equipped unit, causes it to perform binaryization of digital images, is characterized in that said program code during execution executes a feedback control of at least one threshold parameter used in the binaryization to achieve a setpoint in the form of a quality measure of the images after binaryisation. A computer program product that is readable for a processor-equipped unit and comprises a computer program with instructions for causing the unit to perform a method according to any one of claims 1-18. A hand-held position determining apparatus, comprising a sensor (24) for producing a sequence of images of a surface with a position coding pattern, and a processor-provided processing unit (25) arranged to calculate a position from the position coding pattern in the image , the processing unit comprising a computer program according to claim 19. An apparatus for identifying objects in a digital image included in a sequence of images, comprising a segmentation means (32) configured to read in at least one threshold value and compare luminance values of a current digital image with said at least one threshold value. to create, on the basis of the comparison, a current binary image, which is designed to calculate a quality measure of the current binary image and to update said at least one threshold value, the segmentation means (32) being arranged to be read in. the updated threshold for use in binaryization of a subsequent image.