GB2225852A - Object recognition system - Google Patents

Abstract

A system for the automatic recognition of parallel lines in a perspective image of a viewed object (8) comprises a recognition system (7) which detects edge lines of the image and recognises as parallel lines those lines which have a common vanishing direction (17). An algorithm generates a set of planes (14, 15, 16) for the image lines, together with a set of initial pairings between these lines. The quality of fit is decided by the deviation of the end points of the lines in a least squares fit to the common vanishing direction.

Description

Object Recognition This invention relates to a system for detecting parallel lines in a perspective image of a viewed scene. It is intended for use primarily, but not exclusively, in a system for recognising an object, such as a pallet, which is to be picked up by an autonomous guided vehicle.

In a system as described in British Patent: 2,143,395, a vehicle can accurately determine its position and heading in a work space, such as the interior of a warehouse, by combining odometry with triangulated range measurements using reflecting bar-coded targets placed at known, fixed positions around the work space.

Since the vehicle can compute its position, it can move along software-computed paths between known end points, such as conveyors or machine tools. However, in such a system a vehicle cannot locate objects, such as pallets, which may be at random positions in an area, such as a loading bay.

Work has been done on systems which, by use of computer vision, attempt to recognise the shapes of objects.

In general, parallel lines in a viewed scene do not project parallel lines in an image; more often, they converge on a vanishing point. Determination of the vanishing point is the usual method used by object recognition systems for characterising the projected parable0 lines. However, this is a very unstable representation: the error-statistics are highly anisotropic, with a serious singularity when the real 3-dimensional parallel lines are nearly parallel to the image plane. In this case, the lines appear nearly parallel in the image and the vanishing point lies a very great (in the extreme, infinite) distance away from the centre of the image plane.

It is an object of the present invention to provide an improved system for the automatic recognition of parallel lines in a viewed scene.

According to the invention there is provided a system for the automatic recognition of parallel lines in a viewed scene, the system comprising means to produce a perspective image of the object; and means to detect lines in the perspective image and to recognise as parallel lines those lines which have a common vanishing direction.

An embodiment of the invention will now be described, by way of example, with reference to the accompanying drawings, in which Figure 1 is a schematic block diagram of a vehicle navigation system incorporating the invention.

Figure 2 illustrates the concept of vanishing direction, Figures 3, 4, 5 and 6 show, respectively, a pallet image, a corresponding edge map, a set of recognised parallel lines, and an orthogonal set of recognised parallel lines, and Figures 7, 8, 9 and 10 show, respectively, an image of two pallets with boxes thereon, a corresponding edge map, a set of recognised vertical lines, and a set of recognised horizontal lines.

Referring to Figure 1 of the drawings, a vision system 1, comprising a video camera 2, is mounted on a vehicle 3, which is controlled for general movement around a warehouse or other site by signals from a base station 4, fed to the vehicle navigation system 5 over a radio link 6.

The vision system 1 is coupled to a data processing system 7 for processing data obtained from the vision system. The operation of the system will be described in relation to the viewing of a scene including one or more pallets 8, but it is applicable to the viewing of scenes including other objects having edges which comprise parallel lines.

In accordance with the present invention, the recognition of parallel lines is carried out on the basis of a concept of "vanishing direction", which alleviates the above-mentioned problems associated with vanishing point determination.

Referring to Figure 2, the vanishing direction might be defined simply as the direction V from the focal point 10 of the camera system 1 to the vanishing point 11 of a scene comprising, for example, two parallel, but apparently converging, lines 12 and 13.

However, this definition is not sufficient, as it degenerates just as the definition of the vanishing point does. Figure 2 serves to provide another, more useful definition. The lines 12 and 13 in the image, along with the focal point 10, define a set of planes 14, 15 and 16 in space. If these lines all go through the vanishing point 16, then the common intersection of the corresponding planes is a line 17 in space joining the focal point to the vanishing point.

This construction can be carried out even if the lines appear parallel in the image; the vanishing direction is then the direction of a line in space which is parallel to the image plane and to the observed lines in the image plane.

The vanishing direction has a further advantage. If we have an accurate knowledge of the position of the focal point 10 (from an appropriate calibration procedure) then the vanishing direction coincides with the direction of the parallel 3-dimensional lines the images of which are used in the construction. The reasoning is as follows: the real lines are parallel, and lie in the same plane as their projection in the image plane. If these lines never meet, then to lie in these planes they must be parallel to the intersection of the planes and therefore parallel to the vanishing direction.

A difficulty which arises is to find the 'best' vanishing direction, given a set of lines which are approximately converging.

In order to do this, a model of the way in which the positional errors in the image can occur is needed. This model will then allow quantification of the fit between the image lines and the estimated vanishing point. The model used is relatively simple: the centre-points of the lines are assumed to be correct, while the endpoints are allowed to deviate. The measure of fit is chosen to be the square of the endpoint deviation (in order to facilitate a least-squares fit). While this measure is fairly accurate for longer line segments, it allows excessive rotation of the shorter ones, the orientation data of which is usually more reliable than the measure would suggest.

The processing system 7 (Figure 1) operates to find the vanishing direction in accordance with an algorithm as defined below,and which includes the following steps: 1. the equation of the plane defined by the image line and the focal point is parameterised in terms of the endpoint deviation of the line.

2. given this equation, a measure of fit between a set of image lines and an estimated vanishing direction is calculated.

3. an iterative least-squares algorithm is then developed to find the optimal vanishing direction.

The following conventions are observed, namely that the focal centre is chosen to lie at the origin of the co-ordinate system of the cameras, and the image plane is parallel to the x-y plane.

The vanishing direction is represented by a vector p; planes (which in this case all pass through the origin) are represented by their normal vectors; the vectors and are respectively the vectors from the focal centre to the midpoint of an image line, the direction vector of the line, and the normal to that line (chosen such that (t, , k) is right-handed).

The line given by#v,#t generates a plane with normal v At thus given an endpoint deviation#the plane normal becomes: v A(t + 27 n) where 1 is the length of the line. A constraint is # that the vanishing direction p should lie in the corresponding plane:

This is required for each #, #, ni, tj.

From the above, the endpoint deviation required for a given plane to contain the vanishing direction can be determined:

Given this, we then need a method of choosing p such that the combined measure of fit is minimised: Fe fj2, is chosen such that the fit is a least-squares fit. The next section describes a method of iteratively determining the optimal direction p.

A least-squares fit for p is not trivial, essentially due to the non-linearity of the problem, and to the fact that the variation occurs only orthogonally to#. The present approach to the problem is as follows: consider the function in cartesian co-ordinates of the plane orthogonal to the initial estimate of p; approximate this function by a second-order polynomial in two variables, the stationary point of which can then be found analytically.

Given:

we find:

All of the \tj values are independent of the magnitude of p so all of the Vryj values lie in the plane orthogonal to p. Hence, we can consider the two components of VF in this plane as two functions, the zero-crossing of which is to be found. Two unit vectors a and b in the plane orthogonal to p are chosen such that:

Then gradients of both components are taken:

Linearising the gradient, we know that x = p - s (s is the solution) must be orthogonal to F7(b. VF), as the component of VF in thel-direction is already zero, as we wish it to be. It is also required that 7. Y( . 7 F) = Iv Fl .The required direction is given by u = V( 7F)Ar; so the final result is:

From this, the new estimate p of the vanishing direction is#' =p - x.

The above algorithm applies to finding the vanishing direction based on a given set of lines. The lines in an image must then be grouped into these parallel sets. This is effected as follows. The line segments extracted from the image are sorted in descending order of length and the shortest are discarded on the basis of a fixed threshold. Then, lines which are within 100 of each other are paired up. These pairs are used as starting points for finding parallel sets.The following steps are then carried out: 1. an initial vanishing direction is determined from the pair; 2. the values of7/are calculated for every line, based on the estimated value of p; 3. lines for which 7 is below a given threshold are included in the set, while the others are excluded from the set; 4. if the set has not changed, then the cycle has been completed; 5. otherwise perform a single iteration of the least-squares algorithm described above, and return to step 2.

Figure 3 shows an image of a pallet, produced by the vision system 1, and Figure 4 shows edge lines 20 which are found by the processing system 7, by use of any known edge detection method (Canny, etc). From these detected edges, the system 7 operates in accordance with the invention to detect sets of parallel lines 21,22 in one horizontal direction (Figure 5) and in the orthogonal horizontal direction (Figure 6), using the vanishing direction concept.

Figure 7 shows an image of a complicated scene, in which confusion is caused by the presence of two pallets, each of which is partly obscured by boxes which are resting on the pallets. Figure 8 shows the corresponding highly-complicated edge map derived by the processing system, and Figures 9 and 10 show vertical and horizontal sets of parallel lines 23,24 recognised by the present system.

It is envisaged that two operations might then be performed on the parallel lines detected by the system in accordance with the invention. Firstly, the sets of parallel lines might be investigated to check whether there are cyclically spaced groups of lines amongst them, indicating, for example, the presence of slats of a pallet or ribs on a container. Secondly, the sets of parallel lines might be compared with stored models, to determine whether there is sufficient correspondence to be able to decide that the object being viewed is a pallet or is another particular object for which data are stored.

If the system were able to determine that the object being viewed is indeed the object which is to be picked up, the navigation system would then drive the vehicle to the correct pick up position in front of the object.

Claims (4)

1. A system for the automatic recognition of parallel lines in a viewed scene, the system comprising means to produce a perspective image of the object; and means to detect lines in the perspective image and to recognise as parallel lines those lines which have a common vanishing direction.

2. A system as claimed in any preceding claim, wherein the means to produce a perspective image comprises a video camera system.

3. A system for the automatic recognition of an object, substantially as hereinbefore described with reference to the accompanying drawings.

4. A guidance and control system for a guided vehicle, including a recognition system as claimed in any preceding claim.