GB2272285A

GB2272285A - Determining the position of edges and corners in images.

Info

Publication number: GB2272285A
Application number: GB9311732A
Authority: GB
Inventors: Stephen Mark Smith
Original assignee: UK Secretary of State for Defence
Current assignee: UK Secretary of State for Defence
Priority date: 1992-06-10
Filing date: 1993-06-07
Publication date: 1994-05-11
Anticipated expiration: 2013-06-07
Also published as: GB9311732D0; GB2272285B

Abstract

This invention relates to a method for digitally processing an image in the form of a digitised black and white video picture in order to locate the edges and/or corners of objects in the image. The method works by comparing, for each point or element in the digitised image or array, the brightness of elements located within a predetermined region (eg 2, 4, 6 etc) centered on the chosen element with the brightness of the element itself to determine those elements having substantially equal brightness. All of these elements are regarded as belonging to the same surface in the image. The method then locates those elements which are associated with a minimum number of equally bright elements within their local region and these located elements will lie on or close to an edge of the image. Once the edge elements have been located in this way, standard methods of non-maximum suppression and edge thinning can be applied to find the position of the edge more accurately. Corners appear as elements associated with lower minima because there are fewer equally bright elements in the vicinity of a corner than around an edge.

Description

Method for digitally processing images to determine the position of edges and/or corners therein The present invention relates to a method for digitally processing an image to locate the edges of objects in the image and to an apparatus for doing the same. A second aspect of the present invention relates to a method for digitally processing an image to locate the corners of objects in the image and to an apparatus for doing the same. In particular this invention relates to a method for digitally processing images which can be used in a method for reconstructing a 3-D description of a scene from a moving image of the scene for the guidance of unmanned vehicles and mobile robot systems.

Throughout this document "corners" are considered to fall within the scope of the term "edges", a corner being a part of an edge.

A great deal of vision research lies in the field of "early vision" which has as its purpose the reduction of the amount of data contained in an image so that the data becomes more manageable and more immediately useful. The amount of data can be reduced by extracting data from the image relating to the positions of edges or corners of objects in the image. One aim of this research is to use such data obtained from an image created by a detector located on a moving vehicle to guide the moving vehicle in real time.

There are four main properties that suitable edge and corner finders have to possess in order to be useful in real time applications, and these are as follows: (a) The edge/corner finders should locate the edges/corners of objects in an image quickly enough to be used in real time applications.

(b) The edge/corner finders should produce a highly stable output.

That is, the number of correct edges/corners detected should be as high as possible and the number of false edges/corners detected should be at a minimum.

(c) An edge/corner should be located continuously as the image moves.

(d) The edge/corner finder should locate the edges/corners in an image as accurately as possible. That is, an edge/corner should be located at precisely the right place in the image.

In addition to this an edge finder should exhibit complete edge connectivity at junctions. ie. edges detected by the edge detector should join where one or more edges meet at junctions.

Many different methods of locating the edges in an image have appeared in literature covering this area of research. The majority of these methods work by taking image brightness differences between neighbouring image positions to create an "edge strength" image. However these methods can become unstable and inaccurate when the image noise is high.

Similarly many different methods of locating the corners in an image have appeared in literature covering this area of research. The different known corner finders each have a different definition of a corner which is translated into a method of processing the image, generally in accordance with a mathematical algorithm. These methods generally involve the use of image brightness spatial derivatives in the algorithm, particularly first and second derivatives. The problem with using spatial derivatives is that again they are affected by noise in the image and to reduce the effect of noise, the image or the derivatives have to be smoothed and this leads to a reduction in accuracy.

Known edge/corner detectors can also suffer from the disadvantage that they are too slow, when embodied in software, to be used in real time applications. One way of overcoming this disadvantage to some extent is to develop dedicated hardware for the edge/corner detector in order to implement the method more quickly. However, designing such dedicated hardware can be costly and time consuming.

The present invention seeks to overcome at least some of the aforementioned disadvantages by providing a method of digitally processing images in order to locate the edges and/or corners therein which can locate the edges and/or corners in an accurate and stable manner, quickly enough to be used in real time applications, when embodied in software.

According to a first aspect of the present invention there is provided a method of digitally processing an image, to locate the edges of objects in the image, comprising the steps of; (a) representing the image by a first two dimensional data array of elements comprising image values, (b) operating on each element in the first array, to determine the number of elements, n, located within a predetermined region centred on the element being operated on which have substantially the same image value as the element being operated on, and (c) locating the elements associated with local minima in the said numbers n.

The method according to the present invention does not employ any image brightness spatial derivatives. It does not therefore incorporate a smoothing process and so retains accuracy. Also, it can locate edges up to 10 times more quickly than known methods.

In the method according to the present invention each element in the image (except those which are so close to an edge of the image that predetermined regions cannot be centred on them) is operated on, either sequentially or simultaneously. When an element is operated on, a predetermined region centred on the element being operated on (hereinafter referred to as the nucleus element) is interrogated to determine the number n of elements within that region having a similar image value to the nucleus element.

The n elements which have a similar image value are then assumed to be part of the same surface in the image. The area within the predetermined region which comprises these n elements with a similar image value is hereinafter referred to as the univalue area. It can be seen immediately that the univalue area will decrease from its maximum (which is the area of the predetermined region) when an image edge is approached. If a straight edge is approached, then the univalue area will decrease to half of the area of the predetermined region as the predetermined region is moved towards and onto the edge. As the predetermined region is moved past the edge, the univalue area will start to increase again. Thus elements associated with local minima in the numbers n will be the elements located on or near to edges in the image. This is how an edge is found according to the present invention.The image value of an element will generally be the brightness of the part of the image represented by the element.

Preferably step (c) includes locating the elements associated with numbers n which are less than a geometric threshold g. If the univalue area covers a large proportion of the predetermined region then the element being operated on will not generally be located on or near an edge.

Therefore these elements can be disregarded by setting g to a suitable proportion of the number of elements in the predetermined region. The processing involved in locating the edge elements is therefore very simple. Once the edge elements have been located in this way standard methods of non-maximum suppression and edge thinning can be applied to pin point the position of the edge more accurately. Preferably, the geometric threshold g is between 65% and 85% of the number of elements in the predetermined region, because within this range very few non-edge elements are located as edge elements and very few edge elements are not located as edge elements.

For many applications of the present invention, for example, for reconstructing a 3-D description of a scene from a moving image of the scene, the edge direction associated with each edge element must be identified. The edge direction associated with an edge element is the direction of the tangent to the edge which passes through the edge element. The edge direction must also be found when non-maximum suppression is to be performed and when it is necessary to apply edge-thinning techniques to locate the position of the edge to sub-element accuracy.

The method according to the present invention allows edge direction to be determined quickly and accurately by the addition of the following steps; (d) operating on each element of the said first array which is located in step (c) (hereinafter referred to as an edge element) by comparing the number n associated with the edge element being operated on with a preset threshold, (e) operating on each edge element associated with a number n which is greater than the preset threshold by locating the centre of gravity of the elements within the predetermined region centred on the edge element being operated on which have substantially the same image value as the edge element being operated on and designating a direction perpendicular to a line connecting said centre of gravity and the centre of the edge element being operated on as the edge direction for the edge element being operated on, and (f) operating on each edge element associated with a number n which is less than the preset threshold by locating a longest axis of symmetry for the predetermined region centred on the edge element being operated on and designating the direction of said longest axis of symmetry as the edge direction for the edge element being operated on.

For a sharp edge which comprises elements of a first brightness to one side of the edge and elements of a second, different brightness to the other side of the edge, step (e) is followed to find the edge direction.

In this case the edge direction is simply perpendicular to a line joining the centre of gravity of the univalue area and the centre of the nucleus element. For a blurred edge where there is a thin band which has a brightness roughly half way between the brightnesses of the two regions which generate the edge, step (f) is followed. Clearly the univalue area of an edge element on a sharp edge will be larger than the univalue area of an edge element on a blurred edge, as the latter will generally comprise a line of elements which passes through the nucleus element.

Hence the preset threshold can be used to distinguish between these two types of edges. Preferably the preset threshold is approximately the diameter of the predetermined region in elements, ie. the number of elements that lie on a diameter of the predetermined region.

In borderline cases between a sharp and a blurred edge there is a tendency for step (e) to be followed when it would be more appropriate for step (f) to be followed. To remedy this a further step can be included which comprises comparing the distance between the centre of gravity and the centre of the edge element being operated on in step (e) with a threshold distance and if the said distance is less than the preset threshold distance the rest of step (e) is not followed and step (f) is followed instead. The threshold distance is preferably set to approximately one sixth of the diameter of the predetermined region.

The method according to the present invention can be adapted easily to locate the corners of objects in the image. According to the second aspect of the present invention, for locating corners of objects in the image, a number n is a local minima if it is less than all the numbers n associated with the elements within a predetermined minimising area centred on the element associated with the said number n. Using this definition of a local minima restricts the elements located by the method to edge elements that are located at corners of objects in the image, ie. corner elements. The univalue area of an edge element located at a corner is generally smaller than that of an edge element not located at a corner. In the neighbourhood of a corner there will generally be a small grouping of elements that have relatively small univalue areas associated with them.The exact position of the corner can be found by locating the element within each such grouping which has the smallest univalue area. This is how a corner is defined according to the present invention.

In the method for locating a corner element step (c) preferably includes the steps of comparing the said numbers n with a geometric threshold g which is less than or equal to half the number of elements in the predetermined region and operating only on the elements associated with numbers less than g to locate the elements associated with local minima in the said numbers n. If the area of the univalue region is more than half the area of the predetermined region then the element being operated on will not generally be located on-a convex edge of a surface. Elements that are not located on a convex edge of a surface will not be located at a corner, therefore such elements can be disregarded. More preferably the geometric threshold g is exactly half the number of elements in the predetermined region.This threshold is preferred because it will allow the maximum range of types of corners to be reported. When a straight edge includes the element being operated on the number of elements in the univalue area will always be greater than half of the elements in the predetermined region, see Figure 1. A smaller value of g results in only "sharper" corners being detected.

Preferably the predetermined minimising area is a square of 5 by 5 elements. A minimising area of this size has been found to be large enough to detect only true corners and yet small enough for all corners discernible to the eye to be located. Also, the larger the minimising area the longer the processing takes.

Sometimes the method for locating corners according to the present invention will report an element as a corner when in fact it is not. This occurs particularly where there is a slight blurring of the image. In a blurred image at a common boundary where two regions of different brightness meet there may be a thin line of elements of a third intermediate brightness. This line is often broken up and only one element thick and so it may cause corners to be wrongly reported.

This problem can be substantially eliminated according to a preferred embodiment of the present invention which comprises in addition to steps (a) to (c), the steps of; (d') operating on each element in the said first array which is located in step (c) (hereinafter referred to as a corner element) by locating the centre of gravity of the elements within the predetermined region centred on the corner element being operated on which have substantially the same image value as the corner element being operated on, and (e') disregarding the corner element being operated on as a corner element if the centre of gravity is within a predetermined distance from the corner element being operated on.

The univalue area will have a centre of gravity that is not close to the element being operated on for a true corner, see Figure 6a. False corners are rejected because a thin line passing through the element being operated on will have a short distance from the centre of gravity to the element being operated on, see Figure 6b. The preferred method with additional steps (d') and (e') does not significantly reduce the speed of the method according to the present invention as only elements previously reported as corners in step (c) are operated on.

The method for locating corners according to the present invention may also report false corners when the image contains a considerable amount of fine structure or image noise. The false corners reported for such images can be substantially eliminated by forcing the univalue area to be, to some extent, a single block of elements of similar brightness as opposed to elements of the same brightness scattered throughout the predetermined region.

A further preferred embodiment of the present invention, has been developed to substantially eliminate this problem, which comprises in addition to steps (a) to (c), (d') and (e') the steps of; (f') operating on each corner element not disregarded in step (e') by determining whether a predetermined number y of elements which extend from the corner element being operated on in the direction of the centre of gravity have substantially the same image value as the corner element being operated on, and (g') discounting the corner element being operated on as a corner element if one or more of the said y elements does not have the same value as the corner element being operated on.

Again the preferred method which additionally comprises steps (f') and (g') does not significantly reduce the speed of the method according to the present invention as only elements that are corner elements are operated upon.

Preferably the predetermined region is substantially circular and more preferably comprises a square of 5 by 5 elements with three elements added onto the centre of each edge of the square. If the region is smaller than this then the results generally tend to become unstable and if the region is greater the results tend to omit the edges associated with the smallest structures in the image that are identifiable by the eye. Furthermore a larger region reduces the speed of the method according to the present invention. In methods for locating edges a predetermined region of 3 by 3 elements is preferred if the image contains multiple edges which run very close to each other.

When dealing with simple greyscale images (where the image value has just a single variable associated with it) the image value of an element will generally be the greyscale brightness of that part of the image represented by the element. Parts of an image, with a size of the order of the predetermined region, which are part of the same surface generally have substantially the same brightness. This is the case for both colour and monochrome images. It may be possible to use other image values derived from the original image, for example, the image brightness gradient, without degrading the performance of the method according to the present invention. In the more complex case of vector images (where each image value has more than one variable associated with it) then the full vector can be used as the image value without necessarily affecting the way the rest of the method works.

Preferably an element is considered to have substantially the same image value as the element being operated on if it differs in brightness from the element being operated on by 50 greyscale units or less. This brightness threshold does not affect the quality of the results but does affect the number of edges that are reported, because it determines the allowed variation in brightness within the univalue area. A reduction in this threshold causes the method according to the present invention to pick up more subtle variations in the image and report a correspondingly greater number of edges. This threshold must be set according to the contrast in the image and can be tuned to give the required density of reported edges.A fixed value of around 25 greyscale units is suitable for almost all real images and if low contrast images are encountered the brightness threshold can be set automatically, simply by varying it to give the required density of reported edges.

According to a third aspect of the present invention there is provided apparatus for digitally processing images in order to locate edges comprising a computer set up to run a computer program according to the methods described above. Preferably the apparatus additionally includes data input means for receiving the image and relaying image values derived from the image to the computer. Preferably the data input means comprises a video camera and a digitising means for converting the output of the video camera into a digital form that can be input into the computer.

Embodiments of the present invention will now be described by way of example only with reference to the following drawings in which: Figure 1 is a schematic representation of a first two dimensional data array of elements comprising image values according to the present invention, with four predetermined regions and respective nucleus elements superimposed thereon.

Figure 2 is a schematic representation of a second two dimensional data array which corresponds to the first array shown in Figure la and which contains edge strengths (g - n), with g set at 27.

Figure 3a shows part of a first two dimensional data array that has a sharp diagonal edge lying across it.

Figure 3b shows the predetermined region centred on element 40 of Figure 3a.

Figure 4a shows part of the two dimensional array that has a blurred diagonal edge lying across it.

Figure 4b shows the predetermined region centred on element 44 of Figure 4a.

Figure 5 is a schematic representation of a second two dimensional data array which corresponds to the first array shown in Figure la and which contains edge strengths (g - n), with g set to 18.

Figures 6a to 6c show three predetermined regions with respective nucleus elements and centres of gravity superimposed thereon.

Figure 7 shows apparatus according to the present invention.

One embodiment of the method according to the present invention will be discussed with reference to Figures 1 and 2.

Figure 1 shows a first two dimensional array 10 of elements 20 which contain image values. The image comprises a dark region which has image values of approximately 75 (these image values are shaded over) and a light region which has image values of approximately 25. Four predetermined regions 2, 4, 6 and 8, each comprising 37 elements are shown with nucleus elements 12, 14, 16 and 18 respectively. The univalue area within each predetermined region comprises the elements within the region that have substantially the same image value as the nucleus element. In the region 2 the univalue area contains 13 elements ie. n = 13, in region 4, n = 22, in region 6, n = 17 and in region 8, n = 37. The geometric threshold g is set at 27 which is just under three quarters of the number of elements in the predetermined region. If the univalue area comprises a number of elements n less than or equal to g then an edge strength g - n is placed in a second array 30, in Figure 2, in the position corresponding to the nucleus element. Clearly, of the above, only the regions 2, 4 and 6 comprise a number n of elements within the univalue area that is less than or equal to 27.

Referring now to Figure 2 which shows the second two dimensional array, or edge strength image 30 of elements which corresponds to the first array 10. The n = 13 associated with the region 2 in the first array 10 produces an edge strength g - n = 14 which is inserted in the second array 30 in the position 32 corresponding to the position of the nucleus element 12 in the first array. The n = 22 associated with region 4 produces an edge strength 5 which is located in the second array in the position 34 corresponding to the position of the nucleus element 14 in the first array. Similarly the n = 17 associated with the region 6 produces an edge strength 10 which is located in the second array in the position 36 corresponding to the position of the nucleus element 16 in the first array.In this way all values of n associated with regions in the first array which are less than 27 are represented in the second array by an edge strength g - n in a position corresponding with their associated nucleus elements. If n is greater than g, a zero edge strength is generally inserted in the second two dimensional array, however for clarity the elements with a zero edge strength in Figure 2 have been left blank. In Figure 2 this results in groups of numbers clustered around the positions associated with edges of the dark region in the image 10.

A list of edge strengths and their corresponding element positions from Figure 2 can be used to find the edge directions associated with edge elements and this in turn can be used for non-maximum suppression in the direction perpendicular to the edge direction in order to locate the edge position more accurately. For example, in Figure 2 the straight edge 38 produces two rows of edge strengths 5 one either side of the edge 38, whereas the actual position of the edge 38 lies between the said two rows.

Non-maximum suppression can be used to find the actual position of the edg 38, once the edge directions are known.

Figure 3a shows part of a first two dimensional data array which has a sharp edge with elements with image values of around 25 to one side of the edge and image values of around 75 to the other side of the edge. The edge direction of the edge shown in Figure 3a is shown by the arrow 46.

Figure 3b shows the predetermined region centred on the element 40 of Figure 3a. The element 40 would be located as an edge element in the method described above. The edge direction of the element 40 is calculated in the following way. Firstly, the position of the centre of gravity of the elements in the univalue area (ie. the unshaded elements) is located, by the standard mathematical method. The position of the centre of gravity is shown at 48. Then the edge direction of the element 40 is the direction perpendicular to the line 50 which joins the centre of gravity 48 and the centre of the edge element 40.

Figure 4a shows part of a first two dimensional data array which has a blurred edge with elements with image values of around 5 to one side of the edge, elements with image values of around 95 to the other side of the edge and a thin line of elements with image values of around 55 lying on the edge. The edge direction of the edge shown in Figure 4a is shown by arrow 52.

Figure 4b shows the predetermined region centred on the edge element 44 of Figure 4a. Clearly the method described above with reference to Figure 3a and 3b cannot be used to find the edge direction for edge elements like element 44. Instead the edge direction can be found by finding the longest axis of symmetry.

The method according to the present invention can determine which calculation applies to find the edge direction of a particular element by comparing the number n associated with that element with a preset threshold. This threshold is set to approximately the number of elements that span a predetermined region, in the present case approximately 7.

Clearly, predetermined regions of the type shown in Figure 4b will generally have numbers n (in this case n = 5) which are less than 7 and the predetermined regions of the type shown in Figure 3b will generally have numbers n (in this case n = 21) which are greater than 7.

Referring now to Figures 1 and 5, the second aspect of the present invention for the detection of corners will now be explained.

For the detection of corners the geometric threshold is set to approximately a half of the elements in the predetermined region, in this case it is set to 18. Referring back to Figure 1, only predetermined regions 2 and 6 will have numbers n associated with them (13 and 17 respectively) which are less than 18. Figure 5 shows the edge strengths (g - n) that are inserted in the second two dimensional data array 30' when g = 18. In Figure 5 this results in four groups of numbers clustered around the positions associated with corners of the dark region in the image 10. The positions of the maximum numbers in each said group correspond to the positions of the corners of the dark region in the image 10.

Referring now to Figures 6a to 6c which show three predetermined regions 34, 36 and 38 with nucleus elements 44, 46 and 48 respectively, the centre of gravity of the univalue areas (shown shaded) can be located by the standard method and are shown by crosses 54, 56 and 58 respectively.

Assume that nucleus elements 44, 46 and 48 have been reported as corner elements according to the method discussed above in relation to Figures 1 and 5. Then according to one embodiment of the present invention elements reported as corner elements are discounted as corner elements if the centre of gravity of the associated univalue area is less than a distance equal to 1 in units of elements from the centre of the corner element.

The shaded univalue area in the predetermined region 34 is the same as that corresponding to corner element 12 in Figure 1. The centre of gravity 54 is more than the said distance away from the centre of the nucleus element 44 and so the nucleus element 44 will still be reported as a corner. The univalue area in the predetermined region 36 is of the type expected to result between two areas of differing brightness in an image that is blurred and comprises a thin line of an intermediate brightness between two said areas. The nucleus element 46 does not represent a corner and so it would be undesirable for it to be reported as a corner.

According to the criterion above nucleus element 46 will be discounted as a corner because the centre of gravity 56 is less than the said distance away from the centre of the nucleus element 46.

The shaded univalue area in the predetermined region 38 shows the type expected when there is a lot of fine structure or noise in the image.

Fine structure or noise in the image will result in a large number of corners. Clearly in such a case it is preferred that corners of the type in the predetermined region 34 are reported in preference to corners of the type in the predetermined region 38. According to a further embodiment of the present invention a corner element is discounted as a corner element if one or more of the three elements extending from the corner element in the direction of the centre of gravity have a different brightness to the corner element. Clearly the univalue area in the predetermined region 38 would fail this test because element 47 has an image value that differs greatly from that of the nucleus element 48 and would be discounted as a corner.

Figure 7 shows apparatus for carrying out the above methods and comprises a video camera 62 which records images and feeds them to a digitiser 64, for example a Datacube Digimax board which feeds the digitised images to a computer 66, for example, a Sun workstation or a Meiko transputing surface. The computer 66 runs according to a computer program which operates according to the method discussed above to locate the corners and/or edges in each image.

One embodiment of a computer program for detecting edges according to the present invention that can be used in the computer 66 comprises the following steps: 1. Setting up a first and a second two dimensional data array The two dimensional data arrays are set up with elements arranged in rows and columns which elements correspond to the pixels in the image being processed. The elements are identified by two dimensional cartesian coordinates (x,y). The said data arrays will generally be 256 elements by 256 elements.

array The digital brightness in greyscale units of a pixel in the image at position (x,y) is assigned to the corresponding element in a position (x,y) in the first two dimensional data array. This is repeated for each pixel in the image.

3. Determining the number n Each element in the first array is then operated on in turn except those within three elements of the outer edge of the data array.

When a particular element is being operated on all the elements in the array are disregarded except those that lie within the predetermined region centred on the element being operated on. The region is chosen to be a square 5 elements by 5 elements with 3 elements added onto the centre of each edge. This shape approximates to a circle.The digital brightness value of each element in the predetermined region is then "compared" with that of the nucleus element according to the following equation;

where #(x,y) is the position of the element being operated on in the array, #r(x,y) is the position of the element within the predetermined region being "compared" with the element being operated on, I(r) is the digital brightness value of the element in position z(x,y) in greyscale units, t is the brightness difference threshold and c is the output of the "comparison". c(r,~) will be approximately equal to 100 if the brightness value of the element in position r(x,y) differs from the brightness value of the element in position ro(x,Y) by less than t units and will be approximately zero otherwise. The brightness threshold t is set to 25 so that elements with brightness values within 25 greyscale units of the nucleus element will have c(#,#) approximately equal to 100.

The value c(s,#) is found for each element within the region including the nucleus element itself and a running total, n, of the outputs c is made, according to the equation;

over predetermined region This total n(x,y) is therefore 100 times the number of elements in the predetermined region that have substantially the same brightness as the nucleus element.

4. Comparison of n(x.v) with the geometric threshold g Each value n(x,y) is then compared to a fixed geometric threshold.

The geometric threshold g is set to 100 times 72% of the number of elements in the predetermined region, ie. 100 times 72% of 37 which is approximately 2650. If n(x,y) is less than the geometric threshold then an edge strength [ g - n(x,y) ] is assigned to the element position (x,y) in the second array corresponding to the position of the element being operated on in the first array. If n(x,y) is greater than g then a zero is assigned to the element in a position (x,y) in the second array.

A list of the elements in the second array, identified by their position in the array #' (x,y), and the edge strength values associated with them indicates the position and strength of edges in the image. Such a list can be used in further standard processing methods to locate edges more accurately.

5. Finding the ed#e direction associated with ed#e elements Each element in the first array that is associated with a number n which is less than the geometric threshold g, ie. all edge elements, are operated on to find the edge direction associated with them in the following way.

The number n(x,y) associated with the element being operated on ro(x,y) is then compared to a preset threshold, p. In this example, p = 700, which is 100 times the diameter of the predetermined region in elements.

If n(x,y) > p then the centre of gravity of the univalue area associated with the element being operated on is found. This is found by the standard method, with each element in the predetermined region weighted by the value c(r,;) given above in equation 1. The centre of gravity is therefore found using the following equation;

where zcg(Z;) is the position of the centre of gravity of the univalue area formed when the predetermined region is centred on The distance between the centre of gravity zcg(x;) and the element being operated on E; is then calculated.This distance is simply the euclidean distance, d, in elements, using the equation;

If d is more than 0.35 times the mask radius in elements, ie. 0.35 times 3.5 which is approximately 1.2, then the edge direction associated with the element being operated on is perpendicular to the line connecting the centre of gravity rcg(ro) and #. If d < 1.2 then the edge direction is found by finding the longest axis of symmetry.

Similarly if n(x,y) < p the edge direction is found by finding the longest axis of symmetry over the predetermined region. In practice, the longest axis of symmetry is found by taking the sums;

over predetermined region

over predetermined region Then the ratio of y2 to x2 is used to determine the orientation of the edge and the sign of xy is used to determine whether a diagonal edge has a positive or negative gradient.

6. Non-Maximum Suppression The list of edge strengths [ g - n(x,y) ] with associated element positions #(x,y) and edge directions can then be processed further by non-maximum suppression. Non-maximum suppression is the suppression of non-maximum ed strengths Eg - n(x,y) ] which lie near to an edge. The edge strengths will be a maximum at the actual position of an edge and to either side of the edge strengths will be smaller, but not necessarily zero.

Non-maximum suppression removes the smaller edge strengths by finding the maximum edge strength in the direction perpendicular to the edge direction. This is a standard method that is commonly used in edge finding algorithms.

A computer program for detecting corners according to the second aspect of the present invention that can be used in the computer 66 comprises the steps 1 to 3 of the program above for locating edges, then the following steps are followed: 4'. Comparison of n(x.y) with the geometric threshold g Each value n(x,y) is then compared to a fixed geometric threshold g.

The geometric threshold g is set to 100 multiplied by half the number of elements in the region ie g = 1850. If n(x,y) is less than the geometric threshold then a value [ g - n(x.y) ] is assigned to the element in the position (x,y) in the second array corresponding to the position of the nucleus element in the first array. If n(x,y) is greater than g then a zero is assigned the element in a position (x,y) in the second array.

5'. Locating local maximum values in the second array Each element in the second array which does not have a value equal to zero is then operated on by comparing the value of the element being operated on (referred to as a central element) with the values of all the other elements located within a region of 5 by 5 elements centred on the central element. If the value of the central element is greater than those of all the other elements in the said region of 5 by 5 elements then the corresponding element in the first array is reported as a corner.

6'. Locating centre of gravitv The centre of gravity of the univalue areas associated with elements reported as corners (corner elements) are then calculated by the standard mathematical method, as above in equation 3, with each element within the predetermined region weighted by the value c(r,~) given above in equation 1.

7'. Discounting corners when univalue areas have a centre of gravitv close to the corner element The distance between the centre of gravity zcg(Z;) and the nucleus element rO is then calculated. This distance is simply the euclidean distance, d, in elements, using equation 4. If d is less than 1 then the corner element r is discounted as a corner element.

8'. Discounting corners that have discontinuous univalue areas.

The brightness value of each remaining corner element is compared with the brightness values of the three elements that extend from the corner element in the direction of the centre of gravity of the univalue area associated with the corner element. If one or more of the three elements has a value of c(r,r~) which is at or near zero then the corner element is discounted as a corner.

This entire process can be run on a RISC processor, using a 256 by 256 pixel image takes about 1/3 of a second for an average scene.

In step 3 of the computer program the equation 1 can be replaced by the following equation: c(r,Eo) ={ 100 if |I(r) - o)l < or = t { O if |I(Z) -- I(ro > t, However using equation ss instead of equation 1 gives less stable results.

Equation 1 allows an element's brightness to vary slightly without having a large effect on the value of c, even if it is near the threshold value.

Equation 1 was chosen because it gives a similar response to equation ss but with slightly rounded corners. Other equations with similar responses can be used in place of equation 1. Equation 1 can be implemented as a look up table for speed.

The main application of the present invention is in a two-dimensional image segmentation system which will track the corners found by the method according to the present invention over time and allow analysis of the scene in view so that separately moving objects may be identified. A video camera feeds images to a digitiser which feeds the digitised image data to the computer. This computer then runs the computer program described above over each image relayed to it from the video camera in real time. The speed of the method according to the present invention has made this possible for the first time. This allows subsequent segmentation of the image into separately moving objects, by virtue of the variation in the motion of the corners, using a 2-D segmentation system.

A second computer program may also run in the computer, for example the DROID 3-D vision system developed by Plessey, which tracks the corners through time in order to construct a 3-D description of the world. The corners in the successive images are followed over many images by the computer. The results of the 3-D structure from motion program can allow a vehicle to avoid static obstacles, whilst the 2-D segmentation system can allow a vehicle to avoid or follow moving objects or vehicles.

Claims

1. A method of digitally processing an image, to locate the edges of objects in the image, comprising the steps of; (a) representing the image by a first two dimensional data array of elements comprising image values, (b) operating on each element in the first array, to determine the number of elements, n, located within a predetermined region centred on the element being operated on which have substantially the same image value as the element being operated on, and (c) locating the elements associated with local minima in the said numbers n.

2. A method of digitally processing an image according to claim 1 wherein step (c) includes locating the elements associated with numbers n which are less than a geometric threshold.

3. A method of digitally processing an image according to claim 2 wherein the geometric threshold g is between 65% and 85% of the number of elements in the predetermined region.

4. A method of digitally processing an image according to any one of the preceding claims which comprises the additional steps of; (d) operating on each element of the said first array which is located in step (c) (hereinafter referred to as an edge element) by comparing the number n associated with the edge element being operated on with a preset threshold, (e) operating on each edge element associated with a number n which is greater than the preset threshold by locating the centre of gravity of the elements within the predetermined region centred on the edge element being operated on which have substantially the same image value as the edge element being operated on and designating a direction perpendicular to a line connecting said centre of gravity and the centre of the edge element being operated on as the edge direction for the edge element being operated on, and (f) operating on each edge element associated with a number n which is less than the preset threshold by locating a longest axis of symmetry of image values over the predetermined region centred on the edge element being operated on and designating the direction of said longest axis of symmetry as the edge direction for the edge element being operated on.

5. A method of digitally processing an image according to claim 4 wherein the preset threshold is approximately the diameter of the predetermined region in elements.

6. A method of digitally processing an image according to claim 4 or claim 5 wherein the distance between the centre of gravity and the centre of the edge element being operated on in step (e) is compared with a threshold distance and if the said distance is less than the threshold distance the rest of step (e) is not followed and step (f) is followed instead.

7. A method of digitally processing an image, according to claim 1, for locating the corners of objects in the image, wherein a number n is a local minima if it is less than all the numbers n associated with the elements within a predetermined minimising area centred on the element associated with the said number n.

8. A method of digitally processing an image according to claim 7 wherein step (c) includes the steps of comparing the said numbers n with a geometric threshold g which is less than or equal to half the number of elements in the predetermined region and operating only on the elements associated with numbers n which are less than g to locate the elements associated with local minima in the said numbers n.

9. A method of digitally processing an image according to claim 7 or claim 8 wherein the predetermined minimising area is a square of 5 by 5 elements.

10. A method of digitally processing an image according to any one of claims 7 to 9 which comprises the additional steps of; (d') operating on each element in the said first array which is located in step (c) (hereinafter referred to as a corner element) by locating the centre of gravity of the elements within the predetermined region centred on the corner element being operated on which have substantially the same image value as the corner element being operated on, and (e') disregarding the corner element being operated on as a corner element if the centre of gravity is within a predetermined distance from the corner element being operated on.

11. A method of digitally processing an image according to claim 10 which comprises the additional steps of; (f') operating on each corner element not disregarded in step (e') by determining whether a predetermined number y of elements which extend from the corner element being operated on in the direction of the centre of gravity have substantially the same image value as the corner element being operated on, and (g') discounting the corner element being operated on as a corner element if one or more of the said y elements does not have the same value as the corner element being operated on.

12. A method of digitally processing an image according to claim 11 wherein y is 3

13. A method of digitally processing an image according to any one of the preceding claims wherein the predetermined region is substantially circular.

14. A method of digitally processing an image according to claim 13 wherein the predetermined region is a square of 5 by 5 elements with 3 elements added onto the centre of each edge of the square.

15. A method of digitally processing an image according to any one of claims 1 to 6 wherein the predetermined region is a square of 3 by 3 elements.

16. A method of digitally processing an image according to any one of the preceding claims wherein the image value is greyscale brightness.

17. A method of digitally processing an image according to claim 16 wherein elements are considered to have substantially the same brightness if they differ in greyscale brightness by 50 greyscale units or less.

18. A method of digitally processing images substantially as hereinbefore described with reference to the accompanying drawings.

19. Apparatus for digitally processing an image for locating the edges of objects in the image comprising a computer set up to run a computer program for carrying out a method according to any one of the preceding claims.

20. Apparatus for digitally processing an image according to claim 19 wherein the apparatus additionally includes data input means for receiving the image and relaying image values derived from the image to the computer.

21. Apparatus for digitally processing an image according to claim 20 wherein the data input means comprises a video camera and a digitising means for converting the output of the video camera into a digital form that can be input into the computer.

22. Apparatus for digitally processing images substantially as hereinbefore described with reference to the accompanying drawings.