WO1992009031A1

WO1992009031A1 - Apparatus and method for digitising coordinates

Info

Publication number: WO1992009031A1
Application number: PCT/GB1991/002036
Authority: WO
Inventors: Alan R. Turner-Smith; Steve White
Original assignee: Isis Innovation Limited
Priority date: 1990-11-16
Filing date: 1991-11-15
Publication date: 1992-05-29
Also published as: GB9024951D0

Abstract

An apparatus and method for digitising the coordinates of an item (1), such as a map or X-ray image, are described. The item (1) is placed on the bed of a digitising tablet (2) and a mouse (3) is movable over the surface of the item. The mouse incorporates a television camera (6) which outputs an image signal to an associated computer (11). The image may be displayed on computer screen (10) simultaneously with a computer-generated cursor (14) which latter may be moved over the screen display to select a desired point on the image to be digitised. Positional and orientation information about the position of the mouse, and hence the scene being viewed, at any one time is determined by a pair of pick up coils(4, 8), the coordinates of whose positions are sent ot the computer. By computer analysis of the positions of the two coils (4 and 8), the position and orientation of the image on the screen can be related to the physical position of the scene being viewed, and hence the position of the cursor (14) at the selected point can likewise be determined, and stored in permanent storage.

Description

"APPARATUS AND METHOD FOR DIGITISING COORDINATES"

This invention relates to an apparatus and method for digitising coordinates and in particular to a system including a manually movable cursor pointer operable to enable the generation, in an associated computer, of digital positional coordinates, enabling the analysis of a scene.

Such systems commonly utilise a flat plate, known as a digitising tablet, on the bed of which the item to be analysed is placed. One type of digitising tablet comprises a plurality of wires, crossing at right angles to form an X-Y matrix, and along which current is passed. The cursor pointer is generally mounted on a mouse, or similar, so that it can be readily moved across the surface of the item being analysed_,. The mouse incorporates a pick-up coil which receives signals, by electromagnetic induction, from the crossed wires of the tablet. These signals, together with those from the electronic circuitry controlling the digitising tablet, are passed to an associated computer for analysis. From the signals it receives, the computer is able to calculate the position, in digital form, of the cursor pointer.

Such image processing systems have a variety of uses, for example in cartography for the analysis of maps. In the present case, the particular use envisaged is in the analysis of X-ray images although it --^{s not} intended that the invention be limited to such use. For analysis of transparent items, such as X-ray films, it is desirable that the digitising tablet is itself transparent.

Digitising tablets themselves are well known, and will not be described further. It is envisaged that the present invention may be utilised with any type of digitising tablet.

Conventional cursor pointers for digitising tablets use cross wires or a similar mark to enable the operator to place the mouse accurately over the point whose position is to be digitised. Particularly with large items, the resolution and accuracy of the tablet usually exceeds the precision with which the operator can place the cursor. To improve the precision, an optical magnifier is sometimes fitted. Higher resolution is achieved under higher optical or television magnification by controlling the movement of the cursor or item to be analysed on a carriage by screws or electric motors.

The system of the present invention achieves the maximum precision of digitising by high magnification through use of a television camera but without the need to place the camera or item to be analysed accurately.

The camera, which is preferably of the miniature type, is mounted on the mouse and so may be moved by the operator about the surface of the item to be analysed. The video signals from the camera are displayed on a screen within the operator's view. Conveniently, this may be the same screen as that used with the associated computer. It is not necessary to have a cursor pointer on the mouse, although this may be retained for convenience. Instead a cursor may be generated by the computer, and superimposed on the screen on top of the image from the camera. This will be described in more detail later.

As already mentioned, the mouse carries a pick-up coil which, in association with the digitising tablet and its electronics, provides positional information to the computer. In an embodiment of the present invention, the mouse is provided with two such pick-up coils, spaced apart across the surface, and both able to pick up signals from the digitising tablet by electromagnetic induction. The use of two coils enables not only positional information to be sent to the computer, but also a signal representative of the orientation of the mouse on the tablet.

In an embodiment of the invention the top side of the digitising tablet, or a separate transparent plate to be placed above or below the item to be analysed, is formed with a matrix of reference marks, for example reseau marks, in such a way that the marks will be viewed by the camera, and will thus appear on the screen simultaneously with the image of the item. The matrix of marks does not necessarily have to be aligned in any particular way with reference to the digitiser co-ordinates because any misalignment can be ascertained and taken into account within the computer. However, the use of such marks can enhance the accuracy and resolution of the system because the reference marks can be applied very accurately and, with magnification onto the screen, the position of a particular point between adjacent marks ascertained very accurately - to better than 30 microns accuracy, which is better than the accuracy of the digitising tablet itself. In order that the invention may be better understood, an embodiment thereof will now be described by way of example only and with reference to the accompanying drawings in which:

Figure 1 is a diagrammatic perspective view of a system constructed in accordance with the invention; Figure 2 is a more detailed and part-exploded view of the system of Figure 1 ;

Figure 3 is a spatial view of certain components within the mouse of Figure 2; Figure 4 is a circuit diagram of part of the switching circuitry for the mouse pick-up coils; Figure 5 is a flow diagram illustrating the procedure for capturing an image;

Figure 6 is a diagram illustrating image frame coordinate to digitiser coordinate mapping; and Figure 7 is a diagram illustrating a portion of the digitising tablet over which is placed a transparent plate containing a matrix of reference marks.

In the embodiment illustrated, the item 1 to be analysed is placed on the bed of an electromagnetic digitising tablet 2. Movable over the surface of the item 1 is a pick up unit in the form of a mouse 3. The mouse comprises a housing in which are positioned a positional pick-up coil, represented by a circle 4, a lens 5, a CCD or similar television camera 6 and a keypad 7. The keypad is intended for various normal mouse functions, as well as some of the camera controls. Also within the mouse housing is an orientation pick-up coil, represented by a circle 8, spaced from the coil 4. Both of coils 4 and 8 are positioned so that, with the mouse placed as shown, they are able to couple electromagnetically with the digitising tablet. A first lead 9 takes the video signals from the camera to a screen 10 belonging to the associated computer 11. A second lead 15 enables data and other signal transfer between the mouse and the computer. Finally, although not shown, the system includes the electronic circuitry associated with the digitising tablet 2, which incorporates leads for applying the necessary signals to the tablet, as well as connections to the computer for control signal and data transfer.

The system is intended to enable a particular feature from item 1 to be identified, and its position and orientation to be accurately obtained and digitised, the digitised coordinates normally then being stored in memory. In order to digitise some feature 12, the camera is placed roughly to view the feature, and the image 13 captured by the computer and displayed, as shown, on the screen. The operator then moves a computer generated cursor pattern 14 over the image on the computer screen to enable the digital coordinates of a point or points on the feature to be taken. To achieve accuracy, the cursor 14 may not only be moved about the screen, but may also be altered in size so as to enable it to be scaled to more accurately match the feature being examined. In the case of the particular cursor 14 illustrated, the two parallel lines may be movable towards and away from one another or rotated, for example, and the diameter of the circle altered.

Once the operator is satisfied with the pos¬ ition of cursor 14 with respect to the displayed image of the feature he may then press a key on the keypad 7 or other key on the computer keyboard to enter that feature as having particular digitised coordinates. The actual positional and orientation information of the mouse at any one time is received from the digitiser tablet electronics and the mouse. The computer combines this information with the position of the cursor on the screen at the time that the key is pressed to calculate the digitised position of the feature point.

Reference is now made to Figures 2 to 4 which show in more detail a particular embodiment of the invention. In particular, the components of the mouse 3 are shown exploded in Figure 2 to assist clarity. The output signal from the camera 6 is taken on lead 9 to an image capture unit 16 and thence via lead 17 to the computer 11. The image capture unit comprises an analogue to digital converter which converts to digital format the analogue output signal from camera 6, and a frame store having a video output for direct displays if the computer 11 is not used. Such items are available commercially and may take the form of a card for use in the computer. An example of a suitable unit is the Virtuoso Image capture unit, part No. 92400X, made by Primagraphics Ltd., of Melbourne, Hertfordshire. This unit provides a parallel 32 bit output to the computer 11.

The digitiser tablet 2 may likewise comprise a commercially-available item, an example being taken from the range of digitiser tablets marketed by GTCO Corporation of Colombia MD21046, USA. We have used their 16" x 24" Translucent digitiser tablet, part No. 1624T. The electronic circuitry associated with the digitiser tablet is shown under reference 18 in Figure 2 and may comprise GTCO's type T5 controller unit, part No. 21A71D4.

Also available from GTCO is a 16-button high accuracy cursor for use with the above digitiser tablet and controller which we have adapted to form part of the mouse 3. In particular, the keypad 7 and associated circuitry is take from the GTCO cursor. The circuitry is modified as shown in Figure 4 to adapt the system for two pick-up coils 4 and 8. Referring to Figure 4 there is shown the keypad and coil switching circuitry, shown under reference 30 in Figure 2. The block 19 represents the unmodified keyboard circuitry supplied by GTCO for their 16-button cursor. The pin numbers on the GTCO circuit board have the following meanings: Pin 1 = zero volts Pin 7 = status LED

Pins 6 and 8 = connections to pick-up coil Pin 7 of the GTCO board, which is used to drive a status LED, is further used to drive a relay comprising a relay coil 20 and change-over contacts 21. Diode D1 is an integral part of the relay. As can be seen, the function of the relay is to switch over the pins 6 and 8 from coil 4 to coil 8, according to whether the relay coil is energised or not. The manner of operation of the circuit is explained below. Referring now to Figure 3, there is shown the spatial configuration of certain components of the mouse 3, namely the optical arrangement of the camera 6, and the pick-up coils 4 and 8. The CCD image sensor forming part of camera 6 is shown under reference 22. All dimensions, which are given by way of example only, are in millimetres. The top surface of the digitiser tablet is represented by the dotted line 23. The lens 5 is mounted in a lens holder 24 made of plastics material such as black acetyl. The lens itself is a bi-convex type, having a focal length f given by the lens formula:

1 ⁼ 1 ⁺ 1 ^{f f}1 ^f i.e. f = 30 mm

The sequence of operations in the process of capturing an image of a portion of an item 1 placed on the digitising tablet is illustrated in Figure 5, which is largely self-explanatory. Once the mouse 3 has been moved to the desired position, the operation to capture the image then being viewed by the mouse camera commences by the actuation of a particular key on the keypad 7. This causes the video signal from camera 6 to be passed to the image capture unit, which converts the analogue video signal representative of the image into a digital form. The digital signal is then passed to computer 11 where it is stored in RAM. The computer now sends a command along lead 25 to the digitiser control box 18 which returns a signal in ASCII format representative of the then X-Y coordinates of coil 4 on the digitiser tablet. The coordinates are likewise stored in RAM. Next, the computer sends a command to the digitiser control box 18 which causes the LED status line to be set, thus causing pin 7 of the keyboard circuitry 19 to be set and thus energising the relay coil 20. This in turn switches over the pins 6 and 8 from the positional pick-up coil 4 to the orientation pick-up coil 8. There is now a short delay, to allow the relay to act, and then the computer sends a further command to the digitiser control box to return the X-Y coordinates of coil 8 on the digitiser tablet. These coordinates are also stored in RAM. Finally, the digital representation of the image as stored in RAM, together with the two sets of coordinates associated with that image, are stored to more permanent storage, such as a hard disc. The keypad 7 is once again enabled to allow the procedure to start again with a new image. In this way, the operator can rapidly select and store large numbers of images simply by moving the mouse 3 and pressing a single button on keypad 7. Control may, of course, be transferred to the computer keyboard if this is more convenient. The whole operation described with reference to Figure 5 is carried out automatically, once the initial key has been actuated, under the control of a software programme running on computer 11. At each position of mouse 3, the operator is presented on the screen with a magnified view of the portion of the item 1 being viewed and he/she can therefore readily select particular images for storing. Furthermore he/she does not need to be accurate in the placing of the mouse to capture a particular image because the work of locating, and storing the coordinates of, a particular desired point on the image is carried out later on the computer, and using a computer-generated cursor in the manner outlined above.

In normal usage, it is anticipated that a number, possibly a large number, of points on an item 1 will need to be identified, and their coordinates stored. With the present invention, this operation is carried out in two stages: firstly the operator moves the mouse around the item, while viewing the magnified image on the screen. When a desired image has been identified - that is one which contains one or more of the points to be digitised - the operator holds the mouse stationary, and presses the key to commence capture of the image on the computer in the manner described above with reference to Figure 5. Exact positioning is not required at this stage, so the operation can be carried out very quickly. The operator repeats this operation to store the images, and associate two sets of coordinates, for each desired image on item 1.

The second stage in the operation is to extract from storage each of the images and associated coordinates in turn, and analyse them, using a suitable computer-generated cursor to establish, for each desired point on the displayed image, a second set of coordinates representative of the position of the point with respect to a second set of X-Y axes within the image frame. When the cursor is correctly positioned on the screen, a keyboard button is pressed to enter the image frame coordinates into the computer. Software in the computer then mathematically combines the image frame coordinates with the already-stored coordinates of that image with respect to the digitiser tablet X-Y axes to establish a third set of coordinates representative of the position of the desired point with respect to the digitiser tablet X-Y axes.

This method, although carried out in two stages, has been found to be quicker, more accurate and less fatiguing than the conventional method using an unmodified mouse with an inbuilt cursor.

Reference is now made to Figure 6 in order to illustrate the mapping of the coordinates of an exemplary point O from the image frame axes to the digitiser axes. The X and Y axes of the digitiser tablet are indicated by the references X_D and Y_ respectively; those of the image frame by the references X. and Y. respectively. The points A and B are the physical positions of the geometric centres of the pick-up coils 4 and 8 respectively on mouse 3. The image frame being considered is represented by the rectangle 26 and the apparent movement of the object in the image frame from an initial position 27 to a final position 28 is illustrated in the inset to Figure 6.

To illustrate the movement of an object in the image frame, when the cursor is moved on the digitiser tablet, Figure 6 shows the coils positioned at AB and A^'B^' with an object visible in both image frame areas. In order to map the image frame coordinates to digitiser coordinates the positions of A and A' are required together with the angles of AB and A'B^' relative to the digitiser Y-axis and the relative scale between the two coordinate frames.

Having the X-Y coordinates of both of points A and B for each image enables the computer to calculate the angle which the line A-B makes with the Y-axis Y_n of the digitiser tablet. The image frame Y-axis Y_τ, however, may not be parallel with the line joining points A and B, because of a rotational misalignment of the camera with the line joining the centres of coils A and B. In this case all coordinates taken from the image should be rotated about the position of the centre of coil A in the image frame by the misalignment shown as angle theta in Figure 6. Following this rotational correction the relative coordinates of the object to the point A are calculated by subtraction. Using a previously calculated scaling factor for image coordinates to digitiser coordinates the relative coordinates of the object to point A in the image frame are scaled and then added to the coordinates of coil A, given by the digitiser, to form an intermediate coordinate. The coordinates of the centres of coil A and coil B, given by the digitiser, are then used to calculate the angle, alpha, that the line joining A to B makes with the digitiser Y-axis Y_D. Finally, the intermediate coordinate point is rotated about A by -alpha to give the mapped position of the object on the digitiser tablet. The techniques for achieving this within the computer are well known, and will not be described.

The normal limit to the accuracy of the digitising apparatus is set by the accuracy and resolution of the digitising tablet since it is this which effectively determines how well the television images can be mapped onto the digitiser coordinates.

Figure 7 illustrates an embodiment of the invention in which the accuracy and resolution can be enhanced over that of the basic digitising tablet used, by forming a matrix of reference marks of higher accuracy than the digitising tablet on the surface of the digitising tablet in such a way that the marks, or such of them as appear within the view of the camera, will appear on the screen 10 simultaneously with the image of the item being analysed. Where the item to be analysed is transparent, the marks can be made dir¬ ectly on top of the digitiser tablet and their position and orientation with respect to the digitiser coordin- ates is fixed. Alternatively a separate transparent plate of approximately the same size as the digitising tablet can be placed under or, more likely, over the item. If such a plate is marked over its surface with the above-mentioned matrix of reference marks then such marks as appear in the view of the camera will appear in the magnified image on the screen. In this connection it will be noted that the electromagnetic coils 4 and 8 will operate satisfactorily even though spaced a few millimetres above the bed of the digitising tablet. These ideas are represented diagrammatically in Figure 7 in which the lines 31, 32 show an X-Y array of lines intended to represent the digitiser coordinates (which are not normally, of course, visible), the cross marks 33 represent the reference marks forming the matrix which overlies the digitiser coordinates, and reference 34 represents a typical image as seen by the camera 6, with the reference marks 33 appearing simultaneously with the image of the item being analysed. It will be noted that the three coordinate systems illustrated - that of the digitising tablet, the matrix of reference marks and the television image need not necessarily be in alignment: any misalignment between the coordinates of the digitising tablet and those of the matrix of reference marks can be ascertained and taken into account by the computer; misalignment between the coordinate systems of the image frame and the digitising tablet is discussed in some detail above. The position of the reference marks may be measured in the television image on the screen by any known technique, for example by automatic pattern recognition software, to thereby enable the TV image coordinates of the marks to be mapped into the coordinates of the digitising tablet. The mapping of the image frame coordinates of the reference marks into the digitising tablet coordinates has only to be accurate enough to identify the marks, not measure them. Thereafter, other image frame coordinates, such as cursor 14, can be mapped onto the coordinate system defined by these reference marks, each at a known position, and will adopt the accuracy of the reference marks and television camera, rather than that of the digitising tablet.

The advantages of the system over conventional digitising methods are:- 1. It is not necessary to position the digitiser cursor accurately, but simply to place it near the feature to be digitised.

2. The computer-generated cursor may be modified interactively to suit the particular characteristics of the feature to be digitised.

3. Computer "mouse" or direction keys are easy to use and can position the computer cursor with great precision on the computer screen.

4. If many features are to be digitised on the object, multiple television images may be captured quickly and the computer screen images digitised in comfort.

Other possible implementations of the principle are:- a. The television image is not captured in a computer but a suitable electronic cursor superimposed on the television image from the cursor camera. b. Any type of digitised tablet other than an electromagnetic one may be used, providing the orientation as well as position of the camera cursor can be recorded. c. Automatic image processing methods may be applied to the computer or television images to extract the position of features, which may then be related to the digitiser coordinates.

It is anticipated that the techniques of this invention could be applied to both two dimensional (as described above) and three dimensional digitising of coordinates.

Claims

1. Apparatus for digitising coordinates, said apparatus comprising a digitising system for digitising the coordinates of an item to be analysed positioned within an area of investigation, a pick up unit incorporating means for detecting the orientation and position of the pick up unit within said area of investigation, said pick up unit further including a camera operable to obtain an electrical signal representative of the image of a portion of said item, display means for displaying said image, means for generating a cursor and displaying said cursor on said display means simultaneously with the display of said image, means for determining the position of said cursor at any one time and storage means for storing coordinates representative of the position of the image within said area and of the cursor.

2. Apparatus as claimed in claim 1 wherein said digitising system comprises a digitising tablet having a surface on which said item to be analysed is placed, and wherein said detecting means comprises at least one coil which is electromagnetically coupled to the digitising tablet so that, during use, the pick up unit is moved over the item to be analysed.

3. Apparatus as claimed in claim 2 wherein said pick up unit incorporates a further coil for picking up positional information from said digitising tablet, said further coil being spaced from the first, and means^' for analysing the positional information from the two coils to thereby ascertain the orientation of the pick up unit.

4. Apparatus as claimed in claim 3 including switch means for switching between the two coils in order to select from which coil the positional information is obtained, and means for controlling said switch means in such a way as to extract information from one of said coils and then the other, in sequence.

5. Apparatus as claimed in any one of claims 1 to 4 further including means for magnifying the image taken by the camera for display on said display means.

6. Apparatus as claimed in any one of the preceding claims further comprising a matrix of reference marks applied either to the top surface of the digitising tablet, or to a separate sheet of transparent material positioned on top of said digitising tablet either above or below the item to be analysed, in such a way that the displayed image contains a section of the matrix, and means for measuring the position of one or more of the displayed marks, to thereby enable the position of the cursor with respect to such mark or marks to be determined.

7. Apparatus as claimed in any one of the preceding claims comprising a computer which includes said cursor generating means and cursor position determining means and said storage means, and further including an analogue to digital converter for digitising the analogue output signal from said camera for application to the computer.

8. Apparatus as claimed in claim 7 wherein said computer incorporates a display means which comprises the aforesaid display means for displaying said image and said cursor simultaneously.

9. Apparatus as claimed in any one of the preceding claims wherein said pick up unit further comprises a keyboard comprising a plurality of keys operable to control certain functions of the apparatus, in particular a key to commence the operation of obtaining the digital coordinates of an image.

10. Apparatus as claimed in any one of the preceding claims wherein said storage means is further operable to store a digital representation of the image itself, associated with the stored positional information relating to that image.

11. A method of digitising coordinates obtained from an item to be analysed placed on a digitising tablet, said method comprising: a) moving about the item until a desired position is reached, a pick up unit comprising at least one coil for picking up positional information from said digitising tablet and a camera operable to obtain an electrical signal representative of the image of a portion of the item; b) displaying the image generated by said camera; c) generating and displaying simultaneously with said image a cursor; d) moving said cursor about the image displayed on the screen until a desired particular position is reached; e) recording and storing the coordinates of the cursor and of the image.

12. A method as claimed in claim 11 further comprising initially storing a digital representation of a plurality of images, together with their associated positional coordinates and thence, as a separate and later operation, reading said images from storage, displaying said images as aforesaid and thence carrying out said steps (d) and (e) in respect of each displayed image in turn.

13. A method as claimed in either one of claims 11 or 12 wherein the pick up unit is moved about the surface of the item until a desired position is reached and, at this position, the operator actuates a sequence of operations as follows:

(a) the camera takes a picture of the scene currently in view, and outputs an analogue electrical signal representative of the image of said portion of the item;

(b) the analogue output signal from the camera is converted into digital form and the resultant digitised image signal is stored in RAM;

(c) a digital signal representative of the coordinates of the coil is output from the pick up unit and is stored in RAM; (d) the stored image signal and stored coil coordinates corresponding thereto are transferred from RAM to permanent storage in such a way that the two corresponding stored items may be associated with one another; (e) the pick up unit resets ready for a further identical sequence of operations at a further position.

14. A method as claimed in -claim 13 wherein included within the aforesaid sequence of operations between steps (c) and (d) is a further step as follows:

(d) a digital signal representative of the coordinates of a further coil incorporated in said pick up unit is output from the pick up unit and is stored in RAM, said further coil being spaced from the first and being operable to pick up positional information from said digitising tablet; and wherein, in the aforesaid step (d) , the stored image signal and the stored coordinates from both said coil and said further coil are transferred from RAM to permanent storage in such a way that all three corresponding stored items may be associated with one another.