Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
  • G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
  • G06F3/044—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by capacitive means
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
  • G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor

Definitions

• the present invention relates to methods for determining the shape of an object from electrostatic sensing, to methods for controlling a device or apparatus in dependence on said determined shape, and to apparatus comprising processing means and electrostatic sensing elements for performing said methods.
• the present invention has particular, but not exclusive, application to computer systems, displays, consumer electronic devices and other interactive devices where object shape determination may be used for gesture recognition, for example, thereby providing touchless control and interaction.
• a first transmitting electrode is excited by application of an alternating voltage which, due to capacitive coupling sets up electric field lines with a second receiving electrode and hence induces a displacement current in the second electrode.
• an alternating voltage which, due to capacitive coupling sets up electric field lines with a second receiving electrode and hence induces a displacement current in the second electrode.
• some of the field lines are terminated by the object which leads to a decrease in the capacitive current.
• the presence of an object in close proximity to the electrodes may be sensed if the current is monitored.
• the patent US6, 025,726 discloses an electric field sensing arrangement which is intended for computer and other applications.
• the position of a user's finger or hand or whole body may be sensed, depending on the intended application.
• the circuitry and arrangement as described in US6,025,726 is somewhat bulky and complex which does not lend itself well to integration into a laptop or smaller handheld device with a display.
• the method disclosed to determine approximately the position of an object is based on the incorrect physical assumption that the electric field between transmitter and receiver electrodes is described by a dipole field and also on the incorrect assumption that this field is not perturbed by the presence of the object. Resolving the 3D shape or outline of an object is problematic with such a system and method. There therefore arises a need for an electric field sensing method and apparatus that is able to determine the three dimensional shape of an object to enable, for example, control via gesture recognition.
• a method for determining the shape of an unknown object placed in an electrostatic sensing region extending from sensor apparatus comprising a plurality of electrostatic receiving means and at least one electrostatic transmission means, the method comprising measuring the change in charge induced by said unknown object on each receiving means and storing said charge measurements as a first charge image dataset Cf , retrieving a predetermined shape dataset (PSD) representing point co-ordinates which define said predetermined shape, determining for the first charge image dataset C f a charge distribution dataset q 1 representing the charge distribution required on the predetermined shape to substantially result in the measured charge image C f , calculating the electrostatic potential distribution in the sensing region corresponding to said charge distribution dataset q', generating a new shape dataset having co-ordinates corresponding to points in the sensing region where the electrostatic potential distribution is close to zero, and determining the new shape dataset as representing the shape of the unknown object in the sensing region by comparing the difference between the new shape dataset and the predetermined shape dataset against a
• a method for controlling a device comprising determining an object shape in accordance with the first aspect as described above, selecting from a list of control actions associated with predefined shapes a control action associated with said determined object shape, and executing said selected control action.
• a system for determining the shape of an unknown object placed in an electrostatic sensing region extending from sensor apparatus comprising at least one electrostatic transmission means and a plurality of electrostatic receiving means, storage means for storing a predetermined shape dataset, and processing means for determining the shape of an unknown object placed in an electrostatic sensing region as mentioned in the first aspect above.
• the aforementioned method aspect in general determines the shape of an unknown object by incrementally altering the shape of a predetermined known object in dependence on a calculated electrostatic potential contour.
• the altering of the predetermined shape is guided by calculating a (fictitious) charge distribution on the predetermined shape, the calculation being guided by considering the predetermined object as a grounded conductor which enables the use of a regularisation procedure such as Tikhonov regularisation.
• the apparatus comprises a computer system having a microprocessor operable to perform the aforementioned method aspects, the processor and computer being linked to an electrostatic sensing plate comprising several receiver electrodes and several transmission electrodes.
• memory storage for storing the predetermined shape in the form of peripheral point co-ordinates onto which triangular elements are mapped.
• the processor monitors the charge received on each receiver, maps a suitable fictitious or hypothetical charge distribution onto the known object shape stored in memory, calculates the zero electrostatic potential contour and alters the position of each triangular element towards this contour, guided by the regularisation procedure, until a satisfactory fit has been found.
• Said processor may determine the fit as being satisfactory by comparing the sum of the squared differences between the measured charge at the sensor plate and that which would be generated and therefore measured due to the fictitious charge dataset, and terminating the process when this sum is below a predetermined threshold of 5% difference for example.
• a predetermined shape for example a sphere
• the unknown shape for example a hand in a thumbs up gesture
• the apparatus may comprise display means and a graphical user interface in which a control action associated with the determined shape is performed when said shape is determined.
• program code comprising instructions that instruct a processor to perform the aforementioned method are provided, as is a program code carrier having said instructions provided thereon.
• a method of controlling a device is provided, the control depending on the determination of the shape of an object as described in the first aspect.
• Figure 1 shows a personal computer linked to an electric field sensing sensor plate
• Figure 2 shows the sensor plate in more detail
• Figure 3 is a flow diagram illustrating steps of a method according to the present invention.
• Figure 4 is a flow diagram illustrating steps of a control method according to the present invention.
• Figure 5 is a screen shot of a display of a computer implementing a method according to the present invention.
• the embodiment shortly to be described comprises a system having an IBMTM compatible personal computer (PC) connected to sensing apparatus in the form of an electric field sensing plate having integrated field sensing electrodes and circuit components.
• the PC in usual fashion, has processing means in the form of a general purpose microprocessor. Storage in the form of magnetic hard disk drives, memory and optical disk drives are provided for storing programs which, when loaded into memory and executed by the microprocessor cause aspects of the present invention to be performed.
• the PC may reside remote to the electric field sensing plate with only a connecting link therebetween being necessary to provide the PC with sensed measurements.
• the link may be provided by direct means (for example serial, USB or firewire implementations) or indirect means (such as network links over ethemet, either wired or wireless).
• FIG. 1 is a schematic illustration of a system embodying the present invention.
• the system comprises a PC 10 connected by serial RS232 data link 11 to electric field sensing apparatus (S) 20.
• the PC 10 has an IntelTM Pentium class microprocessor 12 (or another suitable alternative such as those produced by AMDTM) which is connected via data-buses 13 to storage 14 (RAM), hard disk drive (HDD) 16 and optical disc drive 17.
• RAM storage 14
• HDD hard disk drive
• Method aspects of the present invention are embodied as software code 17b supplied originally on compact disc media 17a.
• the code 17b is copied to the hard disk drive 16 for local storage and execution.
• the code may be provided on other carrier media such as flash ram cards or floppy disks, or may be carried as a signal over the internet and hence downloaded to the PC on demand of the user.
• the code 17b, once supplied to the PC 10 will typically be installed onto the hard disk drive 16 for execution by microprocessor 12.
• Operating System code such as Microsoft Windows XPTM
• FIG. 2 illustrates the sensing apparatus (S) 20 in more detail.
• the plate 20 comprises many rows of electric field receiving electrodes ( R) 21a, 21b, 21c ... , and transmitting electrodes (T) 31a, 31b, 31c..., the electrodes being arranged in a checkerboard or alternating pattern.
• the sensor may comprise as many as 800 x 600 individual electrodes, arranged in a matrix. For the sake of clarity only a few rows and columns are shown in the Figure.
• the configuration, circuit layout and circuit constituents of such a sensing plate 20 having a sensing array may optionally be integrated in for example a poly- silicon based thin film transistor liquid crystal display, with electrodes integrated with individual display pixels.
• sensing plate technology and example sensing plate arrangements and circuits are more fully described in Applicants co-pending international application WO 02/103621 having the same inventor, published on the 27 th December 2002, the contents of which are herby incorporated, and to the which the reader is now directed.
• the plate 20 measures changes in charge induced on the receiving electrodes by the presence of an object, said changes in charge being provided to the PC 10 via serial link 11 for analysis via program 17b, the program having geometry information relating to the configuration and type of sensing plate attached either input by the user at initial installation, or automatically detected in an initial plug and play type of exchange between the program/PC 10 and the sensor plate 20.
• V alternating potential voltage
• This potential in turn induces a charge on the surface of the object such that the potential resulting from this charge distribution is equal, but opposite in sign to the transmitter potential so that the net potential at the object surface is zero.
• the charge induced on the object causes a change in the charge distribution at the sensor plate 20, this change being measured by the receiver electrodes (R) 21a,b,c.
• This charge change induced at the sensing plane is sometimes called a "charge image" in the technical art of electric field or electrostatic (capacitive) sensing.
• the measured charge change by plate 20 is usually of the order of only a few to one femtoFarads (1fF) per receiving electrode and hence inherently sensitive to noise.
• the uncertainty introduced into the measured charge image by the noise has, to the best of the inventor's knowledge, prevented any algorithm (and hence computer implementation of such) from converging on a meaningful solution which determines the probable shape of the object in the sensing region.
• the problem may be regularised by assuming that the charge distribution on a known hypothetical object which is an insulator will be similar to that which would result on the known object if it is treated as a conductor.
• FIG. 3 starts at step 40 wherein the computer 10 monitors the charge output by plate 20.
• a user then, for example, places an object such as his hand above the plate 20 and the change in charge is supplied from each receiver electrode 21a,b,c, to the computer which stores said charge as a first charge image dataset C f in memory 14.
• the program 17b then causes processor 12 to then perform step 42 in which data representing a predetermined shape is retrieved from memory.
• applicants chose to store the shape as a sequence of triangles having position information relating them to the sensor plate.
• the shape may be manipulated by moving individual triangles of the shape.
• the starting shape may be for example a sphere suspended a few centimetres above the plane defined by the sensing plate, and having dimensions within the x and y limit of said plate (say 20 x 20 cm for example).
• the triangular elements are mapped onto the object surface so as to form the 3D predetermined shape, and hence the predetermined shape is stored in hard disk drive 16 as a dataset describing the point co-ordinates of said elements making up said shape.
• the program Having retrieved the predetermined shape dataset (PSD) from storage 16 and passed to memory 18 for processing, the program then causes processor to move onto step 44 where the charge distribution q' required on the predetermined shape to result in the measured charge image C f is calculated.
• G is a matrix dataset stored in computer memory comprising elements describing the electrostatic exchange potential between each point co-ordinate of the predetermined shape and each other point of the predetermined shape and t is a vector dataset describing the electrostatic exchange potential between each point co-ordinate of the predetermined shape and the point co-ordinates of the transmission electrodes 31a,b,c.
• the computer program generates the system of N linear equations from, by example, considering a disk like transmitter and dividing the shape surface itself into N triangle elements with mid-points p h ⁇ ⁇ and z,-, each with a constant charge q ⁇ , and defining the vector t :
• the computer program using numerical computation techniques such as those provided by NAGTM or MATLABTM generates in memory datasets corresponding to G and t.
• the plate is discretised into segments ⁇ Rj, each with a capacitance c, and wherein a matrix having elements My is defined as:
• the program then causes processor to move onto step 46 wherein the potential distribution in the space above the ground plane is calculated according to:
• the program then, at step 48 alters the position of the triangular elements making up the predetermined shape so as to more closely align them with the position co- ordinates corresponding to the zero potential contour. Hence a new shape is generated.
• ⁇ is an attenuation factor introduced to avoid overshoot.
• each triangular element is repositioned so as to closer approximate the positions corresponding to the zero potential contour in the sensing region, thereby generating a new shape at step 48.
• Various threshold criteria at step 50 may then be used to compare (COM) the new shape with the predetermined shape in order to decide whether to replace in memory 14 the predetermined shape with the new shape (step 54 SET) and at step 56 (REP) repeat steps 42 to 50 or whether to, from step 50, determine that the new shape corresponds closely to the unknown object and hence to stop the iteration via path 58 and determine (DET) the new shape as representing the unknown object shape at step 60.
• the new shape element positions may be compared with the predetermined or previous shape element dataset and if the difference is smaller than a difference threshold of say 5%, the program moves to step 60 wherein the new shape dataset is determined to be that of the unknown object shape.
• More complex threshold calculations may be programmed, in which the sensor plate readings (charge image) C f ' that would be produced by the new shape having charge distribution q' are calculated. This dataset C f ' is then compared with the measured charge image dataset C f and if the least squares difference between corresponding values is below say a threshold of 0.1 % then a satisfactory fit is determined and the process moves to step 60 where the new shape dataset is returned as the unknown object shape and the iteration process ends.
• the parameter q 0 was calculated according to equation (1) and subsequently used to regularise the determination of q' from equation (5).
• a heuristic approach to determining qo is applied. This approach involves, for the system of Figure 1 , simulating or measuring the actual charge distributions that are obtained for a variety of different shaped objects of similar size. These charge distributions are then averaged for each position in the sensing region, and the resultant data set used as q 0 as before to regularise (i.e. apply equation (6)) the determination of q' from equation (5)
• the aforementioned method aspect is employed in a system ( Figure 1) to exert control according to the determined object shape.
• the PC 10 stores 16 a list of predetermined object shapes such as hand gestures (for example thumbs up, a fist, and so on) which are associated with a control action.
• a hand in the shape of the thumbs up gesture may be associated with an OK or approval control action
• a hand in the shape of a closed fist may be associated with a stop or no approval control action.
• a user interacting with the system may receive a query from the system.
• the user wishing to respond in the affirmative, does so by placing his hand in a thumbs up configuration in the sensing region.
• the processor 12 determines the object shape as hereinbefore described, selects from the list the control action associated with the shape most closely resembling the determined shape, and performs the control action.
• gesture recognition enabling control is provided.
• Figure 4 illustrates steps according to this embodiment, where at step 62 (DETOS) the object shape is determined, following which at step 64 (RCL) the control list is retrieved and consulted at step 66 (SELCA) to select a control action to be performed, following which the selected control action is executed at step 68 (EXCA).
• Figure 5 is a screenshot of an example computer implementation of the above to determine the shape of an unknown object.
• the method calculations were performed with a single transmitter 31a in the centre of the sensing area 20 and an array of 16x16 receiver electrodes 21a,b, distributed in a rectangular grid over the sensing area 20.
• the parameter ⁇ was 0.02.
• the predetermined shape was a spheroid object 70 and the actual unknown object 72 is, for the sake of clarity only, shown in the Figure.
• the result 74 after only sixty iterations is shown inset in the Figure.
• the predetermined shape sphere
• the predetermined shape has morphed to approximate the shape of the unknown object, thereby showing that shape information is efficiently reconstructed according to the methods hereinbefore described.
• an unknown object shape is computed from electrostatic calculations on a predetermined shaped object and sensed measurements.
• the predetermined object is morphed in shape towards the unknown object shape at each iteration, until the object shape is determined.
• the system advantageously enables control and interaction with a device.
• consumer devices such as television displays, set top boxes, Hi-Fi audio devices and automobiles may be equipped with processing means, program code and sensing apparatus to enable control of the devices based on object shape determination in accordance with the invention.
• Security and access control systems may similarly be equipped, with hand gesture interaction being used to, for example, gain access.

Landscapes

• Engineering & Computer Science (AREA)
• General Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• Human Computer Interaction (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• User Interface Of Digital Computer (AREA)
• Measurement Of Length, Angles, Or The Like Using Electric Or Magnetic Means (AREA)

Priority Applications (3)

Applications Claiming Priority (2)

Publications (1)

Family

ID=9959103

Family Applications (1)

Country Status (7)

Cited By (2)

Families Citing this family (13)

Citations (5)

Family Cites Families (6)

• 2003
  • 2003-05-31 GB GBGB0312516.8A patent/GB0312516D0/en not_active Ceased
• 2004
  • 2004-05-21 EP EP04734326A patent/EP1634156A1/en not_active Withdrawn
  • 2004-05-21 US US10/557,656 patent/US7330035B2/en not_active Expired - Fee Related
  • 2004-05-21 JP JP2006530706A patent/JP4474416B2/ja not_active Expired - Lifetime
  • 2004-05-21 CN CNB2004800149364A patent/CN100432907C/zh not_active Expired - Fee Related
  • 2004-05-21 WO PCT/IB2004/001778 patent/WO2004107157A1/en not_active Ceased
  • 2004-05-21 KR KR1020057022926A patent/KR20060023979A/ko not_active Ceased

Patent Citations (5)

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events