WO2009077962A2 - Method and apparatus for tomographic touch imaging and interactive system using same - Google Patents
Method and apparatus for tomographic touch imaging and interactive system using same Download PDFInfo
- Publication number
- WO2009077962A2 WO2009077962A2 PCT/IB2008/055281 IB2008055281W WO2009077962A2 WO 2009077962 A2 WO2009077962 A2 WO 2009077962A2 IB 2008055281 W IB2008055281 W IB 2008055281W WO 2009077962 A2 WO2009077962 A2 WO 2009077962A2
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- panel
- touch
- signals
- signal
- tomograph
- Prior art date
Links
- 230000002452 interceptive effect Effects 0.000 title claims description 29
- 238000000034 method Methods 0.000 title claims description 26
- 238000003384 imaging method Methods 0.000 title description 6
- 230000008859 change Effects 0.000 claims abstract description 15
- 230000004044 response Effects 0.000 claims description 8
- 238000012545 processing Methods 0.000 claims description 7
- 230000001276 controlling effect Effects 0.000 claims description 6
- 238000005070 sampling Methods 0.000 claims 3
- 230000002596 correlated effect Effects 0.000 claims 1
- 230000003993 interaction Effects 0.000 description 63
- 239000011159 matrix material Substances 0.000 description 33
- 238000005259 measurement Methods 0.000 description 33
- 238000004422 calculation algorithm Methods 0.000 description 21
- 230000000694 effects Effects 0.000 description 15
- 239000000463 material Substances 0.000 description 7
- 238000004364 calculation method Methods 0.000 description 6
- 238000001514 detection method Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 6
- 238000003325 tomography Methods 0.000 description 6
- 230000009471 action Effects 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 5
- 230000008901 benefit Effects 0.000 description 4
- 238000002591 computed tomography Methods 0.000 description 4
- 230000009466 transformation Effects 0.000 description 4
- 230000004913 activation Effects 0.000 description 3
- 230000004075 alteration Effects 0.000 description 3
- 238000004519 manufacturing process Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000001131 transforming effect Effects 0.000 description 3
- 238000005033 Fourier transform infrared spectroscopy Methods 0.000 description 2
- NIXOWILDQLNWCW-UHFFFAOYSA-N acrylic acid group Chemical group C(C=C)(=O)O NIXOWILDQLNWCW-UHFFFAOYSA-N 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 239000004020 conductor Substances 0.000 description 2
- 238000009792 diffusion process Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000005693 optoelectronics Effects 0.000 description 2
- 239000000758 substrate Substances 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 241000699670 Mus sp. Species 0.000 description 1
- 238000010521 absorption reaction Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 210000000988 bone and bone Anatomy 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000005094 computer simulation Methods 0.000 description 1
- 230000003750 conditioning effect Effects 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 230000001186 cumulative effect Effects 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 238000000151 deposition Methods 0.000 description 1
- 238000000295 emission spectrum Methods 0.000 description 1
- 239000003292 glue Substances 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000000704 physical effect Effects 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000035935 pregnancy Effects 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 229910052704 radon Inorganic materials 0.000 description 1
- SYUHGPGVQRZVTB-UHFFFAOYSA-N radon atom Chemical compound [Rn] SYUHGPGVQRZVTB-UHFFFAOYSA-N 0.000 description 1
- 230000035807 sensation Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
- 239000010409 thin film Substances 0.000 description 1
- 210000001519 tissue Anatomy 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
- G06F3/0412—Digitisers structurally integrated in a display
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
- G06F3/042—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means
- G06F3/0421—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means by interrupting or reflecting a light beam, e.g. optical touch-screen
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
- G06F3/042—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means
- G06F3/0428—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means by sensing at the edges of the touch surface the interruption of optical paths, e.g. an illumination plane, parallel to the touch surface which may be virtual
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
- G06F2203/041—Indexing scheme relating to G06F3/041 - G06F3/045
- G06F2203/04104—Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
- G06F2203/041—Indexing scheme relating to G06F3/041 - G06F3/045
- G06F2203/04109—FTIR in optical digitiser, i.e. touch detection by frustrating the total internal reflection within an optical waveguide due to changes of optical properties or deformation at the touch location
Definitions
- This invention relates to a method and apparatus for tomographic touch imaging sensor and interactive system using same. More particularly, the invention relates to a method and apparatus for touch sensitive devices for computing systems or for controlling actions of associated devices or equipment, and still more particularly, to devices that detect and/or process simultaneously multiple touch interactions (by fingers or other objects) at distinct locations on a touch-sensitive surface.
- Background Art
- Touch screens are becoming increasingly popular because of the ease and versatility of operation, as well as, their declining price.
- Touch screens can include a touch panel, which can be a clear panel with a touch- sensitive surface.
- the touch panel can be positioned in front of a display screen so that the touch-sensitive surface covers the viewable area of the display screen.
- Touch screens can allow a user to make selections and move a cursor by simply touching the display screen via a finger or stylus.
- the touch screen can recognize the touch and position of the touch on the display screen, and the computing system can interpret the touch and thereafter perform an action based on the touch event.
- the present invention proposes to provide a method and apparatus for achieving, for example, by providing a pixilated image representative of all the touch areas and their shapes. This ability of providing a full representation of all touch interactions is sometimes called 'true-touch'. Accordingly, it is a principal object of my invention to provide a method and apparatus for tomographic touch imaging and interactive system using same, especially for computing systems but also usable to control actions of other devices, and still more particularly, to devices that detect and/or process simultaneously multiple touch interactions (by fingers or other objects) at distinct locations on a touch-sensitive surface. The foregoing is accomplished by combining a touch panel with a tomograph in a unique way, and by using this combination as a touch imaging sensor and/or as an interactive system in a computer system.
- the present invention applies techniques from tomography field of science, but instead of using it for medical purposes, the tomographic techniques are used for making a computer input device.
- the input device is a touch device capable of multi-touch operations, oriented to control the forthcoming new generation of software applications that propose a new paradigm for Human Computer Interaction.
- tomography is used to determine inner information from the borders of a panel, and because invisible signals are used, it is possible to make a transparent touch panel with almost perfect light transmission properties.
- the transparent properties make the touch sensor useful to work with a display, as a touch screen.
- Other such input devices need to put complex elements on a panel that obstruct vision. As these other such input devices need to have at least some transparency, but not 100%, this requirement makes it difficult to manufacture, affecting costs.
- the present invention can be implemented with a very clear and homogeneous material, like a simple sheet of acrylic. Also, because the panel can be very thin, a slim and small system including a flat display can be made, thus enabling the system to be portable.
- a touch sensitive device that can detect a zero-pressure contact.
- a touch sensitive device that can discriminate regarding more than one level of pressure on possible different touch interactions that occurs simultaneously.
- a touch sensitive device that can be robust to some external unwanted interfering signals.
- a touch sensitive device that can have a transparent touch area of quite low absorption with respect to visible light, useful to make the power requirements of a possibly attached display less demanding.
- a touch sensitive device that can be constructed with elements relatively easy to manufacture.
- Figs. IA and IB show, respectively, a top view of the basic tomographic touch device of the present invention, and a cross sectional view taken along line A— A of Fig. IA.
- Fig. 1C shows selected imaginary or notional regions or areas of the touch panel shown in Figs. IA and IB illustrating where touch interactions may occur or be estimated.
- Fig. 2 shows an annular shape representing a particular 'test' touch interaction made on a rectangular touch panel.
- FIG. 3 shows components of a typical tomograph as used in the present invention.
- FIG. 4A shows one simple way of defining geometries for the tomography apparatus according to the invention.
- FIG. 4B shows a very useful way of defining geometries for the tomograph.
- Figs. 5A and 5B show, respectively, a top plan view and a cross sectional view of a touch device taken along line B-B of Fig. 5 A that includes a display and a computer as part of an interactive system according to the present invention.
- Figs. 6 A to 6B show, respectively, a top view of a similar touch device combined with an electrical impedance tomograph and a cross sectional view taken along line C-
- FIG. 7 shows an embodiment of the present invention that uses just an air gap to optically couple emitters and detectors to a panel, and the effect of signal reflection on a panel's edge.
- Fig. 8 shows two typical emission patterns and its associated detectors, and the signal alteration level measured on each detector after passing through an annular phantom.
- Fig. 9 illustrates a flowchart of the operation of an embodiment of the touch device as part of an interactive system.
- Fig. 1OA illustrates a flow chart of the tomographic touch sensor operation.
- Fig. 1OB illustrates a flow chart of the tomographic touch screen operation.
- Fig. 11 illustrates a flow chart of the interactive system using a tomographic touch sensor, a data processing element, and an output device.
- Fig. 12 illustrates a flow chart showing an example embodiment for the acquisition of a tomogram representative of touch interactions.
- Fig. 13 illustrates a flow chart showing an example embodiment for the acquisition of an unreconstructed tomogram representative of touch interactions.
- Fig. 14 illustrates a flow chart showing an example embodiment for the reconstructed tomogram calculation.
- Fig. 15 illustrates a flow chart showing an example embodiment for the definition of parameters for tomogram acquisition.
- a touch device is illustrated in Fig. IA (top view) and Fig. IB (cross sectional view).
- the touch device as shown, consists of a thin, flat, circular panel 10 of light conductive material, such as a suitable plastic material known for this purpose.
- the panel 10 due to its properties is adapted to conduct signals, introduced into the borders of the panel, through the panel.
- the panel can be opaque or transparent and can be rigid or flexible.
- the panel can also be resilient and/or compressible.
- the signals may be any radiation or wave energy that can pass through the panel from border to border or edge to edge including without limitation, light waves in the visible or infrared or ultraviolet regions of the spectrum, electrical energy, electromagnetic or magnetic energy, or sonic and ultrasonic energy or vibration energy.
- the light signals 20 are introduced through the edge or border 22 of the panel 10, as shown in Fig. IB.
- the shape of the panel 10 is not limited to any particular shape and may be irregular or regular, such as, circular, elliptical, square, rectangular or polygonal.
- the panel has a refraction index greater than air so it can act as a waveguide by the effect of TIR (Total Internal Reflection).
- Touch interactions at discrete locations on the panel surface 12 can interfere with, disrupt or frustrate the TIR effect so that a portion of the signal (light) is lost by escaping the waveguide of the panel.
- This is indicated in Fig. IB by the reference numeral 14 showing the light exiting form the panel, i.e. light loss.
- the light conductive properties of the panel are altered in those places where the touch interactions occurs, as indicated in Fig. IB.
- tomograph 30 Surrounding the conductive panel 10, there is an optical tomograph 30.
- tomograph means, as usually defined, a device that can image from the borders.
- the tomograph is controlled to emit light signals 20 via light emitters 34, through signal flow ports 18, which also detects light signals.
- signal flow port refers to both emitters and receivers (detectors), and the signal flow port physically acts either as emitter or receiver.
- an electrode in the electrical part of the embodiment can be a unique device that could serve both purposes emitter and/or receiver.
- an optoelectronics devices like a LED can be used as a detector, although with less performance. So a signal flow port here isn't anything material and thus, isn't either an element that can house an emitter or detector.
- a signal flow port is a term used to refer interchangeably to an emitter, or to a detector, or to a device that may accomplish both roles.
- the port houses an emitter and/or detector and may be virtual or real.
- the signal flow ports 18 are positioned at the edge 22 of the panel 10.
- the light signals 20 penetrate or pass through the panel 10, in the manner described, and are detected by detectors of the signal flow ports 18 after circulating or passing through the panel 10.
- the tomograph 30 generates a tomogram representative of the panel conductivity, which is a two dimensional representation of the calculated or estimated con- ductivity on many imaginary or notional regions 40 defined on the panel surface 12, as shown by the array on Fig. 1C (top view).
- the resulting tomogram is thus a calculation or an estimative representation of the touch interactions, including its shapes and details.
- the tomogram may be calculated to represent the conductivity change produced by touch interactions, by comparing it with the conductive properties of the panel when no touch interactions occur.
- the resulting tomogram can be provided as digital information aimed to control or operate a digital device.
- the digital device may include an output device, such as a display, and a processing element to transform the tomogram and provide output to a user as feedback of touch interactions, thus forming an interactive device.
- FIG. 2 An exemplary shape 50 of an interesting and quite challenging touch interaction is shown on Fig. 2 (top view). Shown is an annular shape that could be an approximation of the contact area produced by the palm of a hand touching a rectangular touch panel. As will be appreciated, the resulting tomograph can represent any complex figure or shape depending on the touching interactions. As a parallelism with medical X-ray tomography, the annular shape shown could be an approximation of the cross section of a hollow bone. It will be appreciated that CT or computerized tomography is a well known technology and is understood by those of ordinary skill in the art.
- the tomograph 30 of the touch device as illustrated on Fig. 3 may typically comprise a set of signal flow ports 18.
- Each signal port may be an emitter and/or a detector for the signals, where the signal flows through the port.
- An emitter 34 is typically an electronic device, for example an IR LED (Infrared Light Emitting Diode) or other light emitting device.
- a detector 32 is typically an electronic device, such as a photo transistor, photo diode, Charge Coupled Device (CCD) and other like devices.
- the signal flow ports 18 are typically located around the periphery or perimeter (edges 22) of the conductive panel 10 and spaced apart, usually at regular intervals.
- the emitters 34 are optically coupled to the panel, in a way that at least a portion of the emitted signal penetrates the conductive panel 10 and circulates or passes through by effect of TIR.
- the detectors 32 are optically coupled to the panel 10 in a way that each detector 32 receives the emission signals 20 emanating from one or more of the emitters 34 or ports 18.
- the ports 18 may be coupled to the conductive panel by via a coupling object that may be optical glue, or just direct contact, or a waveguide like an optical fiber, a diffuser or just an air gap, or by any other known means provided the light from the emitters 34 is introduced properly into the panel 10 and the light is detected by the detectors 32.
- a coupling object may be optical glue, or just direct contact, or a waveguide like an optical fiber, a diffuser or just an air gap, or by any other known means provided the light from the emitters 34 is introduced properly into the panel 10 and the light is detected by the detectors 32.
- the tomograph 30 also includes an emission pattern controller 60 operatively coupled to each emitter 34, for example, using any conventional set of communication channels 62, shown in Fig. 3 as wires.
- the tomograph 30 further includes a signal sampler 64 operatively coupled to each detector 34.
- the tomograph 30 includes a data processor 66 operatively coupled to the pattern controller 60, and to the signal sampler 64.
- the emission pattern controller 60 can energize, upon requirement or according to any predetermined program, any desired combination or set of said emitters 34. Each emitter 34 can be energized at any desired emission level of at least two possible values. The configuration of a set of emitters and its levels defines an emission pattern.
- a representative emission pattern, according to the invention, can be to have one determinate or preselected emitter 34 active while leaving the rest inactive, and then serially process through the emitters 34, one at a time.
- the signal sampler 64 can measure, upon requirement or according to any predetermined program, the signal level received on each detector 32.
- a representative signal sampler 64 can have one or more Analog to Digital Converters (ADC) and also may have one or more analog and/or digital multiplexers.
- ADC Analog to Digital Converters
- the data processor 66 can be incorporated in any known computer system, and is adapted to command the emission pattern controller 60 to energize the emitters 34 using a desired sequence of patterns. Also the emission pattern controller 60 and/or signal sampler 64 may function based on a determinate or preselected program, without being commanded by the data processor, for example a digital logic circuit.
- a set of emission patterns has to be defined, for example, a sequence of patterns may be to activate, in turn or serially, different emitters 34. Each emission pattern is active at different times, forming a sequence. While each emission pattern is in effect, an associated set of detectors 32 have to be defined to measure the effects of the active emissions. Then, the data processor 66 acquires from the signal sampler 64, the measured values from the associated set of detectors 32 via signal flow ports 18, the set being typically all or a substantial subset of the detector 32 signal flow ports 18. The measured values form part of the unreconstructed tomogram. Thus after applying all emission patterns and obtaining measurements from the detectors, the unreconstructed tomogram is obtained.
- An example of two emission patterns, its associated detectors, and a representation of the signal alterations, shown in graph of Fig. 8, received for the annular phantom, is shown also in Fig. 4B.
- the data processor 66 may be a micro controller, microprocessor, FPGA, ASIC or special purpose electronic circuit or even part of a complete computer system. Although it may be desirable for the data processor 66 to include a data memory containing a computer program, it is possible to practice the invention performing most computations with just a digital logic circuit. For example, a tomographic reconstruction calculation, as known in the art, can be executed just with an integrated circuit. That integrated circuit can lack a specific 'memory', because computation can be hard wired.
- the invention contemplates a model using a programmable logic circuit (FPGA) that has a memory, but without any 'computer program' as such stored in the memory.
- FPGA programmable logic circuit
- the data processor 66 commands the emission pattern controller 60 to activate one determinate emitter 34 via its associated signal flow port 18, then, acquires measurements from all detectors 32, then, repeating these steps serially for the rest of the emitters 34, one by one.
- N 6 the number of emitters 34
- N d the number of detector ports
- a vector of N 6 by N d elements is acquired.
- This acquired vector is sometimes called in the context of Computer Tomography (CT) the unreconstructed tomogram or sometimes sinogram.
- CT Computer Tomography
- a reconstructed tomogram can be calculated by the use of a Tomographic Reconstruction algorithm.
- the reconstructed tomogram, or just tomogram is the representation of the features of the tomographed phantoms, a phantom being a particular 'test' or 'active' touch interaction.
- a linear or nonlinear model of the tomographic system may be used to transform the unreconstructed tomogram. If a linear model is chosen, the system may be modeled by matrix equations and solved by linear matrix operations in a known manner.
- the variable b is the un- reconstructed tomogram, a vector containing the measured projection values reaching the detectors 32 for each emission pattern.
- the system matrix in the context of CT, represents the transformation that the typical linear tomograph system accomplishes to generate the projection data.
- a typical linear tomograph thus works by transforming b into x taking into account matrix A as a transformation operator.
- the regions 40 are sometimes called reconstruction regions. In a typical embodiment, the regions 40 are rectangular and are disposed next to each other in an appropriate imaginary or notional grid of pixels.
- the regions have to be geometrically defined in a way that each region is affected by more than two non-parallel line integrals; each line integral being defined as the path of an actually measured signal between a determined pair of emitter 34 and detector 32 of the signal flow ports 18. This is to accomplish, for each region, adequate angle density as is usually called.
- line integrals may be geometrically not similar to a line, but to a sort of 'field' as shown on Fig. 6A.
- a determinate signal corresponding to a line integral may be altered in more than one point of its path without being substantially obstructed by a single touch, as depicted in Fig IB. That physical property of the panel with respect to conduction of signals is an important factor determining ability of permitting adequate tomographic reconstruction. For example if the touch interaction substantially would obstruct the signal, and another touch in other part of the signal path wouldn't further alter the signal, the annular phantom as shown on Fig. 2 would be easily confused, by that hypothetic system, with a filled circle, because that system would be unable to adequately estimate the interior properties of the phantom. That property of possible successive alterations of the signal along a line integral, configures an integral function as sometimes is called. Also, to be able to adequately measure the possible effects of touches over an emitted signal, measurements by the use of analog to digital conversion techniques are important.
- FIG. 5A and 5B may include a display 70 beneath the touch panel 72 preferably transparent, to form a touch screen.
- the touch screen is bent at its edges and the signal flow ports 18 for the emitters 34 and the detectors 32 are arrayed around the bent down perimeter of the device. Light rays 20 pass through the touch screen as described.
- the touch screen may make available the information related to touch interactions generated by the tomograph, to be interpreted by a digital system, and then the digital system may control the display to show information related to the touch interactions, to be seen through the clear panel.
- the panel 10 could be convexly curved, and that curved panel could serve to house the display 70 and also to permit the ports 18 to be easily put or mounted on a substrate like a printed circuit board (PCB).
- Interactive touch device embodiment could be convexly curved, and that curved panel could serve to house the display 70 and also to permit the ports 18 to be easily put or mounted on a substrate like a printed circuit board (PCB).
- PCB printed circuit board
- FIG. 5B Another embodiment of the touch device, as illustrated on Fig. 5B, may include the output device 70 operatively coupled to the data processor 66 or to an alternative data processor to form an interactive system.
- the interactive system transforms the tomogram information representative of touch interactions produced by a user, into information to be represented in the output device 70 as feedback to touch interactions.
- Touch user interactions in the interactive system may be used to modify the state of the software running in the processor, and when the software reflects its modified state in an output device, such as a display, the user receives feedback related to the touch in- teractions.
- the typical embodiment of the interactive system may be a cell phone including the touch sensitive area and displaying the images on its user interface.
- the transformation of the information representative of touch interactions into information for the output device may be of the type commonly used in Graphical User Interfaces (GUI).
- GUI Graphical User Interfaces
- the transformation could be to show an icon on the display with an alternative color when a touch interaction event is detected on an area inside the icon, so a user can have the sensation of touching the icon.
- the output device 70 may typically be a display such as a Liquid Crystal Display
- LCD liquid crystal display
- touch sensor could be used without a display, for example, just to generate sounds.
- the tomograph representative of regions where touch interactions occur may be like a pixilated image representing the conductivity on each region, but an event based GUI may be designed to just accept coordinates of touch interactions or more complex gestures instead of level of activity on determinate regions. So an algorithm to translate activity on each region to coordinates of touch interactions or gestures is useful and contemplated by the invention.
- the algorithm for determination of touch events often called Blob Detection and Tracking and is well known in the art. It is used by some systems that calculate and track touch locations by first using computer vision over an image of the touch regions, and using the image captured with a video camera. Electrical impedance touch sensor embodiment
- a conductive panel 80 may be rendered electrically conductive using the signal flow ports 88, which can be simple electrodes 82, and the signals 84 may be electric currents.
- the electrically conductive panel 80 may be of a material as shown in US patent 4448837. which is conductive and transparent. Another way to implement the electrically conductive panel 80 is by depositing a thin film of transparent conductive material over a non-conductive transparent substrate, so the touch of a finger makes some part of the currents normally circulating on the panel, to circulate through the skin thus lowering the resistance on the contact points.
- a display is mounted beneath the touch screen as previously described.
- the panel may be of a material acting as a dielectric
- the tomograph may be a Capacitance Tomograph, where the touch of a finger or other object can alter the dielectric properties of the panel.
- the panel may be of a material acting as a vibration conducting medium, and the tomograph may have acoustic or piezoelectric transducers as ports, in a similar way as an ultrasonic tomograph for pregnancy is implemented.
- the acoustic impedance of the panel can be altered by the touch of a finger or other object. Operation of interactive touch device embodiment
- Steps Sl to S7 carried out as described in the blocks of Fig. 9 define and calculate parameters for posterior tomograph acquisition and reconstruction. These parameters can be calculated during the interactive operation of the system or device, or optionally the information can be calculated and stored in memory for later recall.
- Steps S8 to S22 carried out as described in the blocks of Fig. 9 are a group of steps that repeats indefinitely while a user is interacting with the interactive device.
- Steps S9 to S14 obtain an unreconstructed tomogram; steps S15 to S17 calculate a reconstructed tomogram representative of detected or sensed touch interactions.
- This is carried out by a suitable algorithm as detailed in the blocks of Fig. 9 for achieving tomographic reconstruction, but other well known algorithms for tomographic reconstruction can be used. Some of these algorithms are well explained in the book Principles of Computerized Tomographic Imaging by Kak and Slaney, available on the web, see http://www.slaney.org/pct/pct-toc.html.
- Step S18 to S22 are carried out as described in the blocks in Fig. 9 for processing touch information to control an application and provide feedback to the user.
- Step Sl is for defining positions and shapes of the imaginary or notional reconstruction regions over the sensitive surface.
- the regions forms typically a pixilated grid but each region can have its own shape, that can be polygonal, or inclusive defining a surface of more than one disjoint shape.
- Step S2 is for defining the set of emission and detector ports to be involved in unreconstructed tomogram acquisition, and also for selecting its positions.
- the port positions may be typically the surroundings of the sensitive panel, for example alternating an emitter with a detector at regular intervals. Also, the positions may be near just some portions of the panel's surroundings, for example on just a subset of edges of a polygonal panel. Where there are not ports, the panel may have reflective materials so signals can bounce there (be reflected) and reach other ports.
- FIG. 4A (top view) is a quite unuseful way of define regions 40 and port 18 locations.
- Step S3 is for defining a sequence of emission patterns, where the sequence typically contains a linear independent set of emission patterns, for example, the activation in sequence of just one emitter.
- Another set of patterns may include patterns interpreted as a matrix that has a substantially high rank so different emission patterns can be discriminated. Matrix rank is well known to those of ordinary skill in the art.
- Another example of patterns can be Code Sequences as normally used for CDMA, a well known technique for multiplexing. The objective is to permit multiplexing and adequate discrimination of the different patterns.
- Step S4 is for defining for each emission pattern a set of associated detector ports.
- the signal reaches a set of detector ports and some of them are selected for measurement.
- the set of ports associated with an emission pattern may be selected taking into account the level of the signal normally received.
- Step S5 is for taking a reference unreconstructed tomogram b0 that will be used as reference to calculate the differences on conductivity when any touch is occurring.
- the panel has default conductivity when no touch is affecting its conductivity, and the measurements are for determining the changes produced by touch interactions.
- Step S6 is for calculating the system matrix A, assuming a linear model of the tomographic system.
- This procedure as explained above, will be readily evident to a person of ordinary skill in the art.
- Step S7 is for calculating its pseudo inverse A' by using a Truncated Singular Value
- TSVD Decomposition
- the parameter that controls the truncation may be used to adjust the results provided by the reconstruction algorithm, for example to make a better estimation of the conductivity values by reducing resulting reconstruction noise by the use of that truncation parameter.
- a way of estimating the truncation parameter may be to simulate the system with respect to its noise performance, and optimizing the parameter trying to minimize the noise figure.
- the definition of parameters for tomographic reconstruction that includes the definition of regions, position of ports, and sequence of emission patterns with its associated detectors, needs to be selected in a way to permit adequate reconstruction.
- a way of evaluating suitability of parameters is by calculating the associated system matrix A, assuming a linear model of the tomographic problem, and then evaluating characteristics of the matrix A. For example, a criterion could be to evaluate if the matrix rank is close to number of defined regions, and with a condition number that characterizes a well-conditioned problem. If after evaluating suitability of tomographic parameters, they don't meet the defined criteria, the parameters can be redefined until the criteria are met. For example, less and bigger regions may be defined. Conditions number, rank, and matrix conditioning are concepts well understood to those skilled in the art.
- Step S8 represents the actions of a user that perform control operations through one or more touch interactions that can occur simultaneously over the sensitive surface.
- Step S9 is for initializing a vector b for holding the posterior acquired unreconstructed tomograph.
- Step SlO is for energizing emitter ports by configuring the pattern controller with an emission pattern.
- each emission pattern is selected, in sequence, from the set of defined emission patterns and applied.
- Step SIl is for measuring the signal levels reaching the set of detector ports associated with the active emission pattern.
- a double reading can be done. For example, the signal levels reaching detectors with all emitters off can be subtracted from the value measured when emission pattern is active.
- An embodiment can discard some of the acquired values with a criterion, or just not measure the value. For example, emitting from a determinate emitter and measuring from a detector in close proximity, may be of little use. Other criteria to discard measurements, may include discarding emitter-detector combinations that involves a weak signal between them, because, for example, their relative angle of incidence. Most emitters and receivers respond weakly to signals at high angles with respect to the perpendicular. Also, the measured values can be weighted by associating weighting factors with each measured value, being that operation useful to reduce overall recon- struction noise.
- Step S12 is for subtracting to this value, the corresponding reference values taken from vector b ⁇ . So when no touch is active, the resulting value will be zero plus the possible measurement noise. And in Step S13, the obtained values are appended to the vector b. As shown on Step S14, the steps from SlO to S13 are repeated until unreconstructed tomograph is completely acquired.
- Step S15 is for preprocessing the unreconstructed tomogram, for example to linearize it by applying for example a nonlinear operation, in a similar way that most X-ray tomography devices first apply a logarithm to linearize the cumulative effect of successive attenuations that tissue imposes to X-rays.
- Step S17 is for post processing the reconstructed tomogram, for example with a noise gate that assign zero to values that are below some threshold. The threshold can be set just above normal peak noise values.
- Step S18 is for recognizing touch interactions taking into account the reconstructed tomogram and generate touch events.
- a set of previously acquired tomograms may be also used.
- Algorithms of recognition of touch events and gestures are well known in the art and are sometimes called Blob Detection and Tracking, and may be based on algorithms such as the Watershed algorithm.
- the kind of events recognized can be a new touch appearing, a touch disappearing, an existing touch moving, or other gestures.
- a gesture can be a pinch gesture that is produced by touching the panel in two locations, and separating apart the fingers. That pinch gesture can be used for example to control the zoom factor in a map application.
- a touch event can be associated with a coordinate that can have more resolution than the provided by the reconstructed tomogram.
- a set of values around an area where the touch is occurring can be processed to obtain a sub-pixel coordinate. For example, having a 3x3 pixels area, the coordinate of each pixel center on that area can be averaged by weighting its coordinate with the value associated with each pixel, obtaining a coordinate similar to each pixel but with sub-pixel resolution.
- touch device operations may skip touch the events detection and generation, transforming the reconstructed tomogram by DSP algorithms or other methods to control the output device.
- the output device may be a sound generation device.
- the application to be controller may be a synthesizer, using touch interactions to alter synthesizer parameters.
- interactions on the horizontal axis may be mapped to pitch and interactions on the vertical axis, to harmonics levels, thus forming a multi spectral instrument.
- Step S19 is for modifying the state of a user interface program in response to the touch events. For example if the user interface has a model of a virtual button, that button state becomes active if a touch interaction is detected on the area within the virtual button perimeter.
- Step S20 is for generating the output information representative of the altered state of the UI program.
- the output information is one that maps to different colors according to the button activation state.
- Step S21 is for showing output information on the display adapted to that information through a transparent implementation of the touch sensitive panel.
- Step S21 is represented by the action of the user receiving feedback for its touch control operations by seeing results on the display. Further interactions from the user can be processed by repeating steps from S8 to S22. Additional examples of touch device operation
- Fig. 1OA illustrates a flowchart of tomographic touch sensor operation.
- step IOOA a tomogram representative of touch interactions over the sensitive surface of the panel is acquired from a tomograph, using as an example the embodiment in Fig.12.
- step IOOB the tomogram output of step IOOA is made accessible as digital information used to control a digital device as a result of the touch interactions.
- Fig. 1OB illustrates a flow chart of tomographic touch screen operation in which step
- step 105A acquires, when needed, a tomogram representative of touch interactions over the sensitive surface from a tomograph and provide it to a computer using possible embodiment in Fig. 12, and in step 105B access, when needed, image information generated by the computer, and display result, to be seen through the sensitive surface as feedback of touch interactions.
- Fig. 11 is a flow chart of an Interactive system using a tomographic touch sensor, a data processing element, and an output device.
- a data processor accesses the tomogram representative of touch interactions produced by a user over the sensitive surface.
- the tomogram is processed with DSP algorithms.
- step HOC a decision is taken whether it is an events-based interactive system? If YES, the program advances to step HOD where touch interactions from the tomogram are recognized and touch events are generated.
- step HOF the touch events in step IOOD are received and a user-interface software program running in the data processor modifies is state in response to the touch events.
- step HOE a user- interface software program running in the data processor transforms the tomogram and modifies its state.
- the output of steps HOE and IOOF feed to step HOG where the user interface software provides information representative of its modified state to the output device as feedback of the touch interactions.
- Fig. 12 is a flow chart showing an example embodiment for the acquisition of a tomogram representative of touch interactions.
- step 120A parameters are defined for tomogram acquisition, including emission pattern sequence, set of detector ports associated with each emission pattern, and amount, position and shape of each reconstruction region, using possible embodiment in Fig.15.
- step 120B an unreconstructed tomogram representative of the touch interactions over the sensitive surface is acquired by estimating and measuring the ports around the sensitive surface, using possible embodiment in Fig.13.
- the reconstructed tomogram is calculated by transforming the unreconstructed tomogram with a tomographic reconstruction algorithm, using possible embodiment in Fig.14.
- Fig. 13 is a flow chart illustrating an example embodiment for the acquisition of an unreconstructed tomogram representative of touch interactions.
- step 130A configure a pattern controller with an emission pattern selected in turns from the set of defined emission patterns to energize emitter ports.
- step 130B access through a signal sampler, to the measurement values from the set of detector ports associated with the selected emission pattern.
- step 130C append the measurement values to the partially acquired unreconstructed tomogram.
- the output is put to a decision in step 130D regarding is any pattern of the set of defined emission patterns not yet applied to have complete the unreconstructed tomogram acquisition? If YES, then the program jumps or loops back to step 130A; if NO, the program outputs.
- Fig. 14 illustrates a flowchart for an example embodiment for the reconstructed tomogram calculation.
- step 140A the system provides matrix A, obtains its pseudo inverse A' by truncated SVD
- step 140B the unreconstructed tomogram is optionally preprocessed.
- step 140Cx: Kb is calculated, where x is the reconstructed tomogram and b is the unreconstructed tomogram.
- step 140D the unreconstructed tomogram is optionally post-processed.
- Fig. 15 illustrates a flowchart of an example embodiment for the definition of parameters for tomogram acquisition.
- step 150A the amount, positions, and shapes of imaginary or notional reconstruction regions on the sensitive surface are defined.
- step 150B amount of emission ports to be involved in unreconstructed tomogram acquisition is defined, and positions selected.
- step 150C a linearly independent set of emission patterns is defined.
- step 150D amount of detector ports to be involved in unreconstructed tomogram acquisition is defined, and positions selected.
- step 150E for each emission pattern a set of associated detector ports is defined.
- step 150F the system matrix A is calculated assuming a linear model of the tomographic system.
- step 150F The output of step 150F is passed to a decision in step 150G, namely, is the rank and condition number of the system matrix A enough to allow an adequate posterior reconstruction? If the answer is YES, the signal is output; if the answer is NO, the signal is looped back to step 150A.
- FTIR based interactive touch device embodiment A preferred embodiment contemplated for practicing the present invention, is an interactive system with the optical FTIR based embodiment. This embodiment permits the use of a panel with almost perfect light transmission properties, the detection of touch interactions with zero-pressure and also with above-zero pressure, and the use of a very low-cost panel, and other manufacturing advantages, like ease of mounting. This embodiment is detailed as follows.
- a conventional LCD display is used as an output device.
- On top of the display there's a clear panel made of a sheet of acrylic of less than a few millimeters thick, where infrared light passes from the borders.
- the objective is to make the screen visible through the panel.
- the panel is in mechanic contact with the screen by using two-sided tape put on the borders of the screen top, out of its active visible area.
- the tape is reflective or mirror-like.
- the tape is thick so if mechanic deformation occurs when pressure is applied with the fingers, the panel doesn't touch the screen.
- the panel shape is similar to a rectangle, but octagonal with short diagonal corners similar to the panel shown on Fig. 5A.
- the panel borders are unpolished so to prevent total reflections on the borders and also to provide some diffusion when light enters or exits the panel's border.
- LEDs used as emitters and photo transistors used as detectors, touching the panel' borders, so infrared light can enter and exit from the panel.
- the LED and photo transistors are alternating, each emitter between two detectors and vice versa.
- the emitters and detectors are spaced apart at regular intervals on each panel edge, separated by about one millimeter.
- the emitters and detectors are soldered on different Printed Circuit Boards, one for each panel edge.
- the emitters are connected to a circuit that works similar to a LED Matrix controller that permits to activate any LED.
- the possible activation levels for each LED are two: no current or the maximum current allowed for the emitters. Taking into account each LED is active just a short period of time, more than the nominal continuum rate of emission is possible, like in conventional LED Matrices.
- the photo transistor detectors have infrared filters matching the emission spectrum of emitters. Each photo transistor makes its current flow through a resistance to convert the current in voltage.
- the detectors are grouped in groups of eight elements, and each group connected to an 8-to-l analog multiplexer.
- Each analog multiplexer is associated with a 14-bit Analog to Digital Converter, and each digital output of the ADCs are connected to a digital multiplexer, so each ADC conversion value can be addressed.
- the digital multiplexer is a bus-based one, so each non-selected ADC is in high impedance state while the addressed one ADC is in low impedance with respect to the bus.
- the LED matrix controller and the digital multiplexer are connected to a FPGA that controls them and is able to execute a Tomographic Reconstruction Algorithm.
- the FPGA is connected to a CPU adapted to be able to run an application with an User Interface.
- the CPU is also connected to the display.
- a pixilated matrix is defined on the panel's top surface.
- the total area matches the screen's visible area.
- the pixels are square with a side size matching emitter and detectors separations.
- a vector representing an unreconstructed tomogram is acquired by energizing in turn each emitter, and while each emitter is active, taking a measurement from all detectors.
- all ADCs are commanded in parallel to take the measurements selecting a multiplexer channel , repeating this for the eight acquisition channels. The measurements are sent to the FPGA through the bus.
- a base measurement representative of the tomogram acquired when no touch interactions occurs is taken at factory and stored with the device on a provided memory. That base measurement is called b ⁇ .
- This measurement is taken in every touch device produced, to account possible variations on each specific device properties. For example, not all LED in a batch produces the same level of light emission for same conditions. So this measurement accounts for that possible variation.
- a linear model of the tomographic system is assumed.
- the system matrix A is also calculated at factory by methods used to achieve Algebraic Reconstruction Methods.
- Each column of matrix A is the unreconstructed tomogram taken when a determinate 'unity' touch interaction is in effect.
- This unreconstructed tomogram is the change with respect to the base tomogram b ⁇ .
- the 'unity' touch interaction is an interaction in a determinate region with a shape and position in accordance with that region.
- Columns of matrix A are representing each reconstruction region.
- Each 'unity' touch interaction can be made by actually touching the panel with a mechanic arm, or by simulating the effect. Those skilled in the art will know how to simulate the effect but a method is proposed anyway.
- the rows of system matrix A represents a determinate pair of emitter-detector, for each emitter-detector pair, the row represents all regions associated with that pair.
- the regions correspond to a pixilated grid and each region has an associated coefficient, so it can be viewed like a 2D gray-scale image canvas.
- To calculate each coefficient an anti-aliased 'white' line is drawn between the emitter and detector on a 'black' background, taking into account emitter and detector coordinates. White corresponds to value 1 and black to 0.
- the 'gray' level of each canvas pixel is then associated with the row coefficient. Then each row coefficient is multiplied with the value of the corresponding pair from base tomogram b ⁇ , to account for actual signal levels involved on the specific device implementation.
- a rectangular system matrix A is used with more columns that rows, so more measurements are taken than the amount of regions. More measurements than regions are useful to have more precision on tomographic reconstruction. It should be noted, that on all steps for tomographic calculations, information associated with some pairs of emitter-detector are not taken into account. Some are discarded according to the following criteria: if the dot product of corresponding row on A matrix with itself is below a threshold. The threshold is selected in a way that the rank of A matrix isn't much below region count.
- a pseudo inverse A' is calculated by the Truncated SVD method, by using MATLAB 's pinv function.
- the tolerance parameter is calculated as follows. A value, say 1, is selected. Then by computer simulation, the reconstruction noise for a set of simulated touch interactions is calculated. Based on the noise properties of simulation results for each region, the value is lowered or incremented, so an optimization algorithm is run. The objective is to have noise lowered and if possible equally distributed for all the regions. A practical value is sometimes around 0.4.
- the pseudo inverse A' is also stored on device for later recall.
- the device enters an interactive loop that runs continuously.
- a tomogram b is acquired as described, with the base tomogram b0 subtracted, so if no touch interactions are occurring, the b vector contains zeros or just normal noise values.
- the reconstructed tomogram, as represented by vector x, is interpreted as an image representing touch interactions, with each pixel corresponding to each reconstruction region.
- the tomogram x is transmitted to the CPU, where it is post processed by a noise gate, thus making zero the values below some threshold.
- the threshold is by default fixed just above normal reconstruction noise.
- the default threshold value is also calculated at factory but a noise control knob is provided to the user that can run a configuration software.
- On the CPU a set of current and previous tomograms are analyzed to find touch events and gestures.
- the software used analyzes the events by executing a Blob Detection and Tracking algorithm.
- the algorithm can be selected by the user with a configuration application, where adapted versions of the software TOUCHLIB, http://www.nuigroup.com/touchlib/ and OPENTOUCH, http://code.google.eom/p/opentouch/ are offered as options.
- events or gestures are recognized, they are sent by TUIO protocol to applications adapted to work with that protocol.
- An example application is a GPS mapping application.
- the application running on the CPU generates visual information as feedback that is sent to the display to be seen through the panel by the user, thus being that visual response feedback of the control operations introduced by touching the touch device of the present invention.
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Apparatus For Radiation Diagnosis (AREA)
- User Interface Of Digital Computer (AREA)
- Geophysics And Detection Of Objects (AREA)
Abstract
A touch screen in the form of a panel (10) is capable of conducting signals (20) and a tomograph (30) including signal flow ports (18) is positioned adjacent the panel with the signal flow ports arrayed around the border (22) of the panel at discrete locations. Signals are introduced into the panel to pass from each discrete border location to a plurality of other discrete border locations for being detected and tomographically processed to determine if any change occurred to signals due to the panel being touched during signal passage through the panel, and therefrom determine any local area (40) on the panel where a change occurred. The tomograph computes and outputs a signal indicative of a panel touch and location, which can be shown on a display (70).
Description
Description
METHOD AND APPARATUS FOR TOMOGRAPHIC TOUCH IMAGING AND INTERACTIVE SYSTEM USING SAME
Disclosure of Invention Cross-reference to related applications
[1] This application claims the priority and benefit of previously filed Argentinean
Patent Application No. 20070105651 filed on Dec. 17, 2007, in the name of Victor Manuel Suarez Rovere, which is here incorporated by reference in its entirety. Technical Field
[2] This invention relates to a method and apparatus for tomographic touch imaging sensor and interactive system using same. More particularly, the invention relates to a method and apparatus for touch sensitive devices for computing systems or for controlling actions of associated devices or equipment, and still more particularly, to devices that detect and/or process simultaneously multiple touch interactions (by fingers or other objects) at distinct locations on a touch-sensitive surface. Background Art
[3] Many types of input devices are presently available for performing operations in a computing system, such as buttons or keys, mice, trackballs, touch panels, joysticks, touch screens and the like. Touch screens, in particular, are becoming increasingly popular because of the ease and versatility of operation, as well as, their declining price. Touch screens can include a touch panel, which can be a clear panel with a touch- sensitive surface. The touch panel can be positioned in front of a display screen so that the touch-sensitive surface covers the viewable area of the display screen. Touch screens can allow a user to make selections and move a cursor by simply touching the display screen via a finger or stylus. In general, the touch screen can recognize the touch and position of the touch on the display screen, and the computing system can interpret the touch and thereafter perform an action based on the touch event.
[4] One limitation of many conventional touch panel technologies is that they are only capable of reporting a single point or touch event, even when multiple objects simultaneously come into contact with the sensing surface. That is, they lack the ability to track multiple points of contact at the same time. Thus, even when two points are touched, these conventional devices can only identify a single location, which is typically the average between the two contacts (e.g. a conventional touchpad on a notebook computer provides such functionality). This single-point identification is a function of the way these devices provide a value representative of the touch point,
which is generally by providing an average resistance or capacitance value.
[5] Another limitation of most touch panels is that, besides being incapable of reporting a plurality of touch points that occurs simultaneously, is that they provide information about just touch coordinates. Known touch panels cannot provide a complete representation of the details of all shapes contacting the panel, because the methods of detection often include just triangulation of touch points. Thus, a need exists for providing a better and fuller representation of touch interactions. Disclosure of Invention Summary
[6] The present invention proposes to provide a method and apparatus for achieving, for example, by providing a pixilated image representative of all the touch areas and their shapes. This ability of providing a full representation of all touch interactions is sometimes called 'true-touch'. Accordingly, it is a principal object of my invention to provide a method and apparatus for tomographic touch imaging and interactive system using same, especially for computing systems but also usable to control actions of other devices, and still more particularly, to devices that detect and/or process simultaneously multiple touch interactions (by fingers or other objects) at distinct locations on a touch-sensitive surface. The foregoing is accomplished by combining a touch panel with a tomograph in a unique way, and by using this combination as a touch imaging sensor and/or as an interactive system in a computer system.
[7] The present invention applies techniques from tomography field of science, but instead of using it for medical purposes, the tomographic techniques are used for making a computer input device. Specifically, the input device is a touch device capable of multi-touch operations, oriented to control the forthcoming new generation of software applications that propose a new paradigm for Human Computer Interaction.
[8] Since tomography is used to determine inner information from the borders of a panel, and because invisible signals are used, it is possible to make a transparent touch panel with almost perfect light transmission properties. The transparent properties make the touch sensor useful to work with a display, as a touch screen. Other such input devices need to put complex elements on a panel that obstruct vision. As these other such input devices need to have at least some transparency, but not 100%, this requirement makes it difficult to manufacture, affecting costs.
[9] The present invention can be implemented with a very clear and homogeneous material, like a simple sheet of acrylic. Also, because the panel can be very thin, a slim and small system including a flat display can be made, thus enabling the system to be portable.
Advantageous Effects
[10] Several objects and advantages of my invention are listed as follows.
• To provide a device that is sensitive to touch and able to detect objects when they are making a subtle pressure. If the object touching is a finger of an user the invention can provide a tactile feedback. This is in contrast to proximity sensors which can be activated when the user isn't touching anything.
• To provide a touch sensitive device that can provide a digital representation of nearly all areas where there is a touch interaction available. This is in contrast to devices that provide location information of just a few touch interactions simultaneously.
• To provide a touch sensitive device that can be thin, thus having a large relative surface area with respect to volume.
• To provide a touch sensitive device that can have a transparent touch panel so that it can include a display visible through the panel.
• To provide a touch sensitive device that can be interactive by providing output information related to touch interactions.
• To provide a thin touch sensitive device that can be flat, or non-planar (e.g. domed) in 3-dimensional space.
• To provide, in some embodiments, a touch sensitive device that can detect a zero-pressure contact.
• To provide, in some embodiments, a touch sensitive device that can discriminate regarding more than one level of pressure on possible different touch interactions that occurs simultaneously.
• To provide, in some embodiments, a touch sensitive device that can be robust to some external unwanted interfering signals.
• To provide, in some embodiments, a touch sensitive device that can have a transparent touch area of quite low absorption with respect to visible light, useful to make the power requirements of a possibly attached display less demanding.
• To provide, in some embodiments, a touch sensitive device that can be constructed with elements relatively easy to manufacture.
[11] Further object and advantages of my invention will become apparent from the following detailed description of preferred embodiments of the invention when taken in conjunction with the appended drawings.
Description of Drawings [12] Figs. IA and IB show, respectively, a top view of the basic tomographic touch device of the present invention, and a cross sectional view taken along line A— A of
Fig. IA. [13] Fig. 1C shows selected imaginary or notional regions or areas of the touch panel shown in Figs. IA and IB illustrating where touch interactions may occur or be estimated. [14] Fig. 2 shows an annular shape representing a particular 'test' touch interaction made on a rectangular touch panel.
[15] Fig. 3 shows components of a typical tomograph as used in the present invention.
[16] Fig. 4A shows one simple way of defining geometries for the tomography apparatus according to the invention.
[17] Fig. 4B shows a very useful way of defining geometries for the tomograph.
[18] Figs. 5A and 5B show, respectively, a top plan view and a cross sectional view of a touch device taken along line B-B of Fig. 5 A that includes a display and a computer as part of an interactive system according to the present invention. [19] Figs. 6 A to 6B show, respectively, a top view of a similar touch device combined with an electrical impedance tomograph and a cross sectional view taken along line C-
C of Fig. 6A. [20] Fig. 7 shows an embodiment of the present invention that uses just an air gap to optically couple emitters and detectors to a panel, and the effect of signal reflection on a panel's edge. Fig. 8 shows two typical emission patterns and its associated detectors, and the signal alteration level measured on each detector after passing through an annular phantom. [21] Fig. 9 illustrates a flowchart of the operation of an embodiment of the touch device as part of an interactive system.
[22] Fig. 1OA illustrates a flow chart of the tomographic touch sensor operation.
[23] Fig. 1OB illustrates a flow chart of the tomographic touch screen operation.
[24] Fig. 11 illustrates a flow chart of the interactive system using a tomographic touch sensor, a data processing element, and an output device. [25] Fig. 12 illustrates a flow chart showing an example embodiment for the acquisition of a tomogram representative of touch interactions. [26] Fig. 13 illustrates a flow chart showing an example embodiment for the acquisition of an unreconstructed tomogram representative of touch interactions. [27] Fig. 14 illustrates a flow chart showing an example embodiment for the reconstructed tomogram calculation. [28] Fig. 15 illustrates a flow chart showing an example embodiment for the definition of parameters for tomogram acquisition.
Touch sensor embodiment [29] Referring now to the drawings, and particularly to the embodiment of the present invention, a touch device is illustrated in Fig. IA (top view) and Fig. IB (cross
sectional view). The touch device, as shown, consists of a thin, flat, circular panel 10 of light conductive material, such as a suitable plastic material known for this purpose. The panel 10, due to its properties is adapted to conduct signals, introduced into the borders of the panel, through the panel. The panel can be opaque or transparent and can be rigid or flexible. The panel can also be resilient and/or compressible. The signals may be any radiation or wave energy that can pass through the panel from border to border or edge to edge including without limitation, light waves in the visible or infrared or ultraviolet regions of the spectrum, electrical energy, electromagnetic or magnetic energy, or sonic and ultrasonic energy or vibration energy. In the following description, light passing through the panel will be described as a preferred embodiment. The light signals 20 are introduced through the edge or border 22 of the panel 10, as shown in Fig. IB. The shape of the panel 10 is not limited to any particular shape and may be irregular or regular, such as, circular, elliptical, square, rectangular or polygonal. The panel has a refraction index greater than air so it can act as a waveguide by the effect of TIR (Total Internal Reflection). Touch interactions at discrete locations on the panel surface 12 can interfere with, disrupt or frustrate the TIR effect so that a portion of the signal (light) is lost by escaping the waveguide of the panel. This is indicated in Fig. IB by the reference numeral 14 showing the light exiting form the panel, i.e. light loss. Thus, the light conductive properties of the panel are altered in those places where the touch interactions occurs, as indicated in Fig. IB.
[30] Surrounding the conductive panel 10, there is an optical tomograph 30. In the context of the present invention, tomograph means, as usually defined, a device that can image from the borders. The tomograph is controlled to emit light signals 20 via light emitters 34, through signal flow ports 18, which also detects light signals.
[31] The term signal flow port refers to both emitters and receivers (detectors), and the signal flow port physically acts either as emitter or receiver. For example, an electrode in the electrical part of the embodiment can be a unique device that could serve both purposes emitter and/or receiver. Also an optoelectronics devices like a LED can be used as a detector, although with less performance. So a signal flow port here isn't anything material and thus, isn't either an element that can house an emitter or detector. In the context of this invention, a signal flow port is a term used to refer interchangeably to an emitter, or to a detector, or to a device that may accomplish both roles. Thus, the port houses an emitter and/or detector and may be virtual or real.
[32] The signal flow ports 18 are positioned at the edge 22 of the panel 10. The light signals 20 penetrate or pass through the panel 10, in the manner described, and are detected by detectors of the signal flow ports 18 after circulating or passing through the panel 10. The tomograph 30 generates a tomogram representative of the panel conductivity, which is a two dimensional representation of the calculated or estimated con-
ductivity on many imaginary or notional regions 40 defined on the panel surface 12, as shown by the array on Fig. 1C (top view). The resulting tomogram is thus a calculation or an estimative representation of the touch interactions, including its shapes and details. The tomogram may be calculated to represent the conductivity change produced by touch interactions, by comparing it with the conductive properties of the panel when no touch interactions occur. The resulting tomogram can be provided as digital information aimed to control or operate a digital device. The digital device may include an output device, such as a display, and a processing element to transform the tomogram and provide output to a user as feedback of touch interactions, thus forming an interactive device.
[33] An exemplary shape 50 of an interesting and quite challenging touch interaction is shown on Fig. 2 (top view). Shown is an annular shape that could be an approximation of the contact area produced by the palm of a hand touching a rectangular touch panel. As will be appreciated, the resulting tomograph can represent any complex figure or shape depending on the touching interactions. As a parallelism with medical X-ray tomography, the annular shape shown could be an approximation of the cross section of a hollow bone. It will be appreciated that CT or computerized tomography is a well known technology and is understood by those of ordinary skill in the art.
[34] The tomograph 30 of the touch device as illustrated on Fig. 3 may typically comprise a set of signal flow ports 18. Each signal port may be an emitter and/or a detector for the signals, where the signal flows through the port. An emitter 34 is typically an electronic device, for example an IR LED (Infrared Light Emitting Diode) or other light emitting device. A detector 32 is typically an electronic device, such as a photo transistor, photo diode, Charge Coupled Device (CCD) and other like devices. The signal flow ports 18 are typically located around the periphery or perimeter (edges 22) of the conductive panel 10 and spaced apart, usually at regular intervals. The emitters 34 are optically coupled to the panel, in a way that at least a portion of the emitted signal penetrates the conductive panel 10 and circulates or passes through by effect of TIR. The detectors 32 are optically coupled to the panel 10 in a way that each detector 32 receives the emission signals 20 emanating from one or more of the emitters 34 or ports 18. The ports 18 may be coupled to the conductive panel by via a coupling object that may be optical glue, or just direct contact, or a waveguide like an optical fiber, a diffuser or just an air gap, or by any other known means provided the light from the emitters 34 is introduced properly into the panel 10 and the light is detected by the detectors 32. Although by using an air gap 92, see Fig. 7, total reflections could occur on the border of a polished panel, before reaching a detector close to the point of reflection, as shown in Fig. 7, anyway the reflected signal may approach another detector, thus tomographic reconstruction still can be accomplished.
[35] The tomograph 30 also includes an emission pattern controller 60 operatively coupled to each emitter 34, for example, using any conventional set of communication channels 62, shown in Fig. 3 as wires. The tomograph 30 further includes a signal sampler 64 operatively coupled to each detector 34. Also the tomograph 30 includes a data processor 66 operatively coupled to the pattern controller 60, and to the signal sampler 64.
[36] The emission pattern controller 60 can energize, upon requirement or according to any predetermined program, any desired combination or set of said emitters 34. Each emitter 34 can be energized at any desired emission level of at least two possible values. The configuration of a set of emitters and its levels defines an emission pattern. A representative emission pattern, according to the invention, can be to have one determinate or preselected emitter 34 active while leaving the rest inactive, and then serially process through the emitters 34, one at a time.
[37] The signal sampler 64 can measure, upon requirement or according to any predetermined program, the signal level received on each detector 32. A representative signal sampler 64, according to the invention, can have one or more Analog to Digital Converters (ADC) and also may have one or more analog and/or digital multiplexers.
[38] The data processor 66 can be incorporated in any known computer system, and is adapted to command the emission pattern controller 60 to energize the emitters 34 using a desired sequence of patterns. Also the emission pattern controller 60 and/or signal sampler 64 may function based on a determinate or preselected program, without being commanded by the data processor, for example a digital logic circuit.
[39] By operating the emission pattern controller 60 and the signal sampler 64 , an unreconstructed tomograph can be acquired.
[40] To operate emission controller 60, a set of emission patterns has to be defined, for example, a sequence of patterns may be to activate, in turn or serially, different emitters 34. Each emission pattern is active at different times, forming a sequence. While each emission pattern is in effect, an associated set of detectors 32 have to be defined to measure the effects of the active emissions. Then, the data processor 66 acquires from the signal sampler 64, the measured values from the associated set of detectors 32 via signal flow ports 18, the set being typically all or a substantial subset of the detector 32 signal flow ports 18. The measured values form part of the unreconstructed tomogram. Thus after applying all emission patterns and obtaining measurements from the detectors, the unreconstructed tomogram is obtained. An example of two emission patterns, its associated detectors, and a representation of the signal alterations, shown in graph of Fig. 8, received for the annular phantom, is shown also in Fig. 4B.
[41] The data processor 66 may be a micro controller, microprocessor, FPGA, ASIC or
special purpose electronic circuit or even part of a complete computer system. Although it may be desirable for the data processor 66 to include a data memory containing a computer program, it is possible to practice the invention performing most computations with just a digital logic circuit. For example, a tomographic reconstruction calculation, as known in the art, can be executed just with an integrated circuit. That integrated circuit can lack a specific 'memory', because computation can be hard wired. The invention contemplates a model using a programmable logic circuit (FPGA) that has a memory, but without any 'computer program' as such stored in the memory.
[42] In a typical embodiment, the data processor 66 commands the emission pattern controller 60 to activate one determinate emitter 34 via its associated signal flow port 18, then, acquires measurements from all detectors 32, then, repeating these steps serially for the rest of the emitters 34, one by one. Thus, if the number of emitters 34 is N6 and the number of detector ports is Nd, a vector of N6 by Nd elements is acquired. One can call this vector the projection vector, or one can call it a measurements vector, or just b. This acquired vector, not to be confused with a matrix, is sometimes called in the context of Computer Tomography (CT) the unreconstructed tomogram or sometimes sinogram.
[43] Having the unreconstructed tomogram with N6 by Nd potentially independent values, the amount of available information to make reconstruction can be thus in quadratic relation with respect to number of signal flow ports 18. So the number of potentially independently recognized touch interactions regions can be in accordance to that quadratic relation, and not in linear relation with respect to number of emitters 34 or detectors 32. Thus, that number of potentially discriminated regions can be quite large or big. Hence, the foregoing is a sample of the potential of the present invention of being able to discriminate not just a few simultaneous touch coordinates, but many.
[44] Having the unreconstructed tomogram, a reconstructed tomogram can be calculated by the use of a Tomographic Reconstruction algorithm. The reconstructed tomogram, or just tomogram, is the representation of the features of the tomographed phantoms, a phantom being a particular 'test' or 'active' touch interaction.
[45] In a tomographic reconstruction algorithm, a linear or nonlinear model of the tomographic system may be used to transform the unreconstructed tomogram. If a linear model is chosen, the system may be modeled by matrix equations and solved by linear matrix operations in a known manner.
[46] A typical equation used to model tomographic systems is Ax=b, where A corresponds to the system matrix, as is sometimes called, x corresponds to the reconstructed tomogram and is a vector that represents the regions conductivity values, not to be confused with a matrix just because regions are usually a grid. The variable b is the un-
reconstructed tomogram, a vector containing the measured projection values reaching the detectors 32 for each emission pattern. The system matrix, in the context of CT, represents the transformation that the typical linear tomograph system accomplishes to generate the projection data. A typical linear tomograph thus works by transforming b into x taking into account matrix A as a transformation operator.
[47] To be able to calculate a two-dimensional tomogram adequately, one needs to have defined appropriate locations to each emitter 34 and detector 32 of the signal flow ports 18, which are typically surrounding the panel 10 on its edge(s). Also, one needs to define imaginary or notional locations and shapes of a set or array of regions 40 on the surface 22 of the panel 10, where an estimation of conductivity is going to be calculated. The regions 40 are sometimes called reconstruction regions. In a typical embodiment, the regions 40 are rectangular and are disposed next to each other in an appropriate imaginary or notional grid of pixels.
[48] Also, to be able to calculate an adequate tomogram, the regions have to be geometrically defined in a way that each region is affected by more than two non-parallel line integrals; each line integral being defined as the path of an actually measured signal between a determined pair of emitter 34 and detector 32 of the signal flow ports 18. This is to accomplish, for each region, adequate angle density as is usually called. In a diffusion tomograph the concept of line integrals may be geometrically not similar to a line, but to a sort of 'field' as shown on Fig. 6A.
[49] A determinate signal corresponding to a line integral may be altered in more than one point of its path without being substantially obstructed by a single touch, as depicted in Fig IB. That physical property of the panel with respect to conduction of signals is an important factor determining ability of permitting adequate tomographic reconstruction. For example if the touch interaction substantially would obstruct the signal, and another touch in other part of the signal path wouldn't further alter the signal, the annular phantom as shown on Fig. 2 would be easily confused, by that hypothetic system, with a filled circle, because that system would be unable to adequately estimate the interior properties of the phantom. That property of possible successive alterations of the signal along a line integral, configures an integral function as sometimes is called. Also, to be able to adequately measure the possible effects of touches over an emitted signal, measurements by the use of analog to digital conversion techniques are important.
[50] Those skilled in the art will understand that the requirements of high angle density for each region and existence of an integral function to do tomographic reconstruction has support in the Projection Slice Theorem, which is related to the Radon Transform. Basically, the projection- slice theorem tells us that if one had an infinite number of one-dimensional projections of an object taken at an infinite number of angles, one
could perfectly reconstruct the original object, f(x,y). Note the need of multiple angles (as much as possible in a non-theoretical device) and the need of projection (i.e. integration data) along the line. However, the present invention is not limited by this theorem.
[51] It's well known that a finger interacting with a waveguide like in the presented embodiments makes better and broader optical contact as more pressure is applied. This effect is because, together with other causes, the skin on the fingerprints gets more deformed, so the average contact area per unit area and thus, average conductivity gets altered in a higher level. This higher change of conductivity could be interpreted as more pressure and used to take distinctive actions. For example an icon of a user interface can be selected with a light pressure but activated with more pressure.
[52] Also, as in any measurement instrument, repeated measurements over time can be done to have a time-varying measurement, so taking a series of tomographs over time is useful to determine how touch interactions changes over time, and that information may be used to detect touch events or gestures, with algorithms well known in the art. Touchscreen embodiment
[53] Another embodiment of the touch device, as illustrated in Figs. 5A and 5B (top plan and cross sectional view, respectively) may include a display 70 beneath the touch panel 72 preferably transparent, to form a touch screen. The touch screen is bent at its edges and the signal flow ports 18 for the emitters 34 and the detectors 32 are arrayed around the bent down perimeter of the device. Light rays 20 pass through the touch screen as described. The touch screen may make available the information related to touch interactions generated by the tomograph, to be interpreted by a digital system, and then the digital system may control the display to show information related to the touch interactions, to be seen through the clear panel. The panel 10 could be convexly curved, and that curved panel could serve to house the display 70 and also to permit the ports 18 to be easily put or mounted on a substrate like a printed circuit board (PCB). Interactive touch device embodiment
[54] Another embodiment of the touch device, as illustrated on Fig. 5B, may include the output device 70 operatively coupled to the data processor 66 or to an alternative data processor to form an interactive system. The interactive system transforms the tomogram information representative of touch interactions produced by a user, into information to be represented in the output device 70 as feedback to touch interactions. Touch user interactions in the interactive system may be used to modify the state of the software running in the processor, and when the software reflects its modified state in an output device, such as a display, the user receives feedback related to the touch in-
teractions. The typical embodiment of the interactive system may be a cell phone including the touch sensitive area and displaying the images on its user interface.
[55] The transformation of the information representative of touch interactions into information for the output device may be of the type commonly used in Graphical User Interfaces (GUI). For example, the transformation could be to show an icon on the display with an alternative color when a touch interaction event is detected on an area inside the icon, so a user can have the sensation of touching the icon.
[56] The output device 70 may typically be a display such as a Liquid Crystal Display
(LCD). A device with a display behind a transparent panel to form an interactive system is one of the main objectives of the invention. However, the touch sensor could be used without a display, for example, just to generate sounds.
[57] The tomograph representative of regions where touch interactions occur may be like a pixilated image representing the conductivity on each region, but an event based GUI may be designed to just accept coordinates of touch interactions or more complex gestures instead of level of activity on determinate regions. So an algorithm to translate activity on each region to coordinates of touch interactions or gestures is useful and contemplated by the invention. The algorithm for determination of touch events often called Blob Detection and Tracking and is well known in the art. It is used by some systems that calculate and track touch locations by first using computer vision over an image of the touch regions, and using the image captured with a video camera. Electrical impedance touch sensor embodiment
[58] In another embodiment of the touch device according to Fig. 6A (top view) a conductive panel 80 may be rendered electrically conductive using the signal flow ports 88, which can be simple electrodes 82, and the signals 84 may be electric currents. The electrically conductive panel 80 may be of a material as shown in US patent 4448837. which is conductive and transparent. Another way to implement the electrically conductive panel 80 is by depositing a thin film of transparent conductive material over a non-conductive transparent substrate, so the touch of a finger makes some part of the currents normally circulating on the panel, to circulate through the skin thus lowering the resistance on the contact points. A display is mounted beneath the touch screen as previously described. Capacitive touch sensor embodiment
[59] In another embodiment of the touch device the panel may be of a material acting as a dielectric, and the tomograph may be a Capacitance Tomograph, where the touch of a finger or other object can alter the dielectric properties of the panel. Acoustic touch sensor embodiment
[60] In another embodiment of the touch device the panel may be of a material acting as a
vibration conducting medium, and the tomograph may have acoustic or piezoelectric transducers as ports, in a similar way as an ultrasonic tomograph for pregnancy is implemented. The acoustic impedance of the panel can be altered by the touch of a finger or other object. Operation of interactive touch device embodiment
[61]
[62] The operation of the touch screen device included in an embodiment of an interactive device (the method of the invention) is shown in the flow chart of Fig. 9 and is described as follows.
[63] Steps Sl to S7 carried out as described in the blocks of Fig. 9 define and calculate parameters for posterior tomograph acquisition and reconstruction. These parameters can be calculated during the interactive operation of the system or device, or optionally the information can be calculated and stored in memory for later recall.
[64] Steps S8 to S22 carried out as described in the blocks of Fig. 9 are a group of steps that repeats indefinitely while a user is interacting with the interactive device. Steps S9 to S14 obtain an unreconstructed tomogram; steps S15 to S17 calculate a reconstructed tomogram representative of detected or sensed touch interactions. This is carried out by a suitable algorithm as detailed in the blocks of Fig. 9 for achieving tomographic reconstruction, but other well known algorithms for tomographic reconstruction can be used. Some of these algorithms are well explained in the book Principles of Computerized Tomographic Imaging by Kak and Slaney, available on the web, see http://www.slaney.org/pct/pct-toc.html.
[65] Step S18 to S22 are carried out as described in the blocks in Fig. 9 for processing touch information to control an application and provide feedback to the user.
[66] Step Sl is for defining positions and shapes of the imaginary or notional reconstruction regions over the sensitive surface. The regions forms typically a pixilated grid but each region can have its own shape, that can be polygonal, or inclusive defining a surface of more than one disjoint shape.
[67] Step S2 is for defining the set of emission and detector ports to be involved in unreconstructed tomogram acquisition, and also for selecting its positions. The port positions may be typically the surroundings of the sensitive panel, for example alternating an emitter with a detector at regular intervals. Also, the positions may be near just some portions of the panel's surroundings, for example on just a subset of edges of a polygonal panel. Where there are not ports, the panel may have reflective materials so signals can bounce there (be reflected) and reach other ports.
[68] Fig. 4A (top view) is a quite unuseful way of define regions 40 and port 18 locations.
If all emitter ports 34 are disposed just on the left of a square panel 10, all detector ports 32 disposed just on the right, measurements done along just horizontal paths, and
reconstruction regions 40 are all vertical rectangles that span from top to bottom, the two dimensional reconstruction that could be done would not produce very interesting results. This example shows that some awkward ways of defining geometries lead to difficulties on tomographic reconstruction. Anyway, the most intuitive and easily defined geometries lead to good results. For example, as shown in Fig. 4B simply disposing alternating emitter and detector ports around the panel, measuring from a substantial subset of the possible emitter-detector pairs, and selecting a pixilated grid for the reconstruction regions in an amount in quadratic relation with number of emitters and detectors, good results can be achieved.
[69] Step S3 is for defining a sequence of emission patterns, where the sequence typically contains a linear independent set of emission patterns, for example, the activation in sequence of just one emitter. Another set of patterns may include patterns interpreted as a matrix that has a substantially high rank so different emission patterns can be discriminated. Matrix rank is well known to those of ordinary skill in the art. Another example of patterns can be Code Sequences as normally used for CDMA, a well known technique for multiplexing. The objective is to permit multiplexing and adequate discrimination of the different patterns.
[70] Step S4 is for defining for each emission pattern a set of associated detector ports.
Once an emission pattern is active, the signal reaches a set of detector ports and some of them are selected for measurement. The set of ports associated with an emission pattern may be selected taking into account the level of the signal normally received.
[71] Step S5 is for taking a reference unreconstructed tomogram b0 that will be used as reference to calculate the differences on conductivity when any touch is occurring. The panel has default conductivity when no touch is affecting its conductivity, and the measurements are for determining the changes produced by touch interactions.
[72] Step S6 is for calculating the system matrix A, assuming a linear model of the tomographic system. A possible way of calculating the system matrix to be further used to calculate the tomogram may be by making X an identity matrix, on equation AX = B , being B a matrix of measurements, a set of unreconstructed tomogram acquisitions. So if X is the identity, thus A=B, thus the way of calculating A may be to touch in sequence each reconstruction region with corresponding shape and 'unit pressure', and measuring in each touch step the projection, so to construct B, and thus this set of measurement results in the system matrix A. This procedure, as explained above, will be readily evident to a person of ordinary skill in the art.
[73] Step S7 is for calculating its pseudo inverse A' by using a Truncated Singular Value
Decomposition (TSVD) algorithm. A typical algorithm for pseudo inverse calculation is provided by the function pinv of the MATLAB® software from MATHWORKS. The parameter that controls the truncation may be used to adjust the results provided
by the reconstruction algorithm, for example to make a better estimation of the conductivity values by reducing resulting reconstruction noise by the use of that truncation parameter. A way of estimating the truncation parameter may be to simulate the system with respect to its noise performance, and optimizing the parameter trying to minimize the noise figure.
[74] The definition of parameters for tomographic reconstruction, that includes the definition of regions, position of ports, and sequence of emission patterns with its associated detectors, needs to be selected in a way to permit adequate reconstruction. A way of evaluating suitability of parameters, is by calculating the associated system matrix A, assuming a linear model of the tomographic problem, and then evaluating characteristics of the matrix A. For example, a criterion could be to evaluate if the matrix rank is close to number of defined regions, and with a condition number that characterizes a well-conditioned problem. If after evaluating suitability of tomographic parameters, they don't meet the defined criteria, the parameters can be redefined until the criteria are met. For example, less and bigger regions may be defined. Conditions number, rank, and matrix conditioning are concepts well understood to those skilled in the art.
[75] Step S8 represents the actions of a user that perform control operations through one or more touch interactions that can occur simultaneously over the sensitive surface.
[76] Step S9 is for initializing a vector b for holding the posterior acquired unreconstructed tomograph.
[77] Step SlO is for energizing emitter ports by configuring the pattern controller with an emission pattern. In each step of the loop, each emission pattern is selected, in sequence, from the set of defined emission patterns and applied.
[78] Step SIl is for measuring the signal levels reaching the set of detector ports associated with the active emission pattern. To measure signal levels having some rejection to possible interfering signals, for example external light on the optical embodiment, a double reading can be done. For example, the signal levels reaching detectors with all emitters off can be subtracted from the value measured when emission pattern is active.
[79] An embodiment can discard some of the acquired values with a criterion, or just not measure the value. For example, emitting from a determinate emitter and measuring from a detector in close proximity, may be of little use. Other criteria to discard measurements, may include discarding emitter-detector combinations that involves a weak signal between them, because, for example, their relative angle of incidence. Most emitters and receivers respond weakly to signals at high angles with respect to the perpendicular. Also, the measured values can be weighted by associating weighting factors with each measured value, being that operation useful to reduce overall recon-
struction noise.
[80] Step S12 is for subtracting to this value, the corresponding reference values taken from vector bθ. So when no touch is active, the resulting value will be zero plus the possible measurement noise. And in Step S13, the obtained values are appended to the vector b. As shown on Step S14, the steps from SlO to S13 are repeated until unreconstructed tomograph is completely acquired.
[81] Step S15 is for preprocessing the unreconstructed tomogram, for example to linearize it by applying for example a nonlinear operation, in a similar way that most X-ray tomography devices first apply a logarithm to linearize the cumulative effect of successive attenuations that tissue imposes to X-rays. Step S16 is for calculating the reconstructed tomograph x by solving the equation x = A' b, where A' is the pseudo inverse of system matrix and b is the unreconstructed tomograph. Step S17 is for post processing the reconstructed tomogram, for example with a noise gate that assign zero to values that are below some threshold. The threshold can be set just above normal peak noise values.
[82] Step S18 is for recognizing touch interactions taking into account the reconstructed tomogram and generate touch events. To recognize events, a set of previously acquired tomograms may be also used. Algorithms of recognition of touch events and gestures are well known in the art and are sometimes called Blob Detection and Tracking, and may be based on algorithms such as the Watershed algorithm. The kind of events recognized can be a new touch appearing, a touch disappearing, an existing touch moving, or other gestures. A gesture can be a pinch gesture that is produced by touching the panel in two locations, and separating apart the fingers. That pinch gesture can be used for example to control the zoom factor in a map application. A touch event can be associated with a coordinate that can have more resolution than the provided by the reconstructed tomogram. A set of values around an area where the touch is occurring, can be processed to obtain a sub-pixel coordinate. For example, having a 3x3 pixels area, the coordinate of each pixel center on that area can be averaged by weighting its coordinate with the value associated with each pixel, obtaining a coordinate similar to each pixel but with sub-pixel resolution.
[83] Alternatively, another embodiment of the touch device operations may skip touch the events detection and generation, transforming the reconstructed tomogram by DSP algorithms or other methods to control the output device. For example the output device may be a sound generation device. The application to be controller may be a synthesizer, using touch interactions to alter synthesizer parameters. As an example, interactions on the horizontal axis may be mapped to pitch and interactions on the vertical axis, to harmonics levels, thus forming a multi spectral instrument.
[84] Step S19 is for modifying the state of a user interface program in response to the
touch events. For example if the user interface has a model of a virtual button, that button state becomes active if a touch interaction is detected on the area within the virtual button perimeter.
[85] Step S20 is for generating the output information representative of the altered state of the UI program. For example, in the virtual button case, the output information is one that maps to different colors according to the button activation state.
[86] Step S21 is for showing output information on the display adapted to that information through a transparent implementation of the touch sensitive panel.
[87] Finally, the output of Step S21 is represented by the action of the user receiving feedback for its touch control operations by seeing results on the display. Further interactions from the user can be processed by repeating steps from S8 to S22. Additional examples of touch device operation
[88] Fig. 1OA illustrates a flowchart of tomographic touch sensor operation. In step IOOA a tomogram representative of touch interactions over the sensitive surface of the panel is acquired from a tomograph, using as an example the embodiment in Fig.12. In step IOOB the tomogram output of step IOOA is made accessible as digital information used to control a digital device as a result of the touch interactions.
[89] Fig. 1OB illustrates a flow chart of tomographic touch screen operation in which step
105A acquires, when needed, a tomogram representative of touch interactions over the sensitive surface from a tomograph and provide it to a computer using possible embodiment in Fig. 12, and in step 105B access, when needed, image information generated by the computer, and display result, to be seen through the sensitive surface as feedback of touch interactions.
[90] Fig. 11 is a flow chart of an Interactive system using a tomographic touch sensor, a data processing element, and an output device. In step HOA a data processor accesses the tomogram representative of touch interactions produced by a user over the sensitive surface. In step HOB the tomogram is processed with DSP algorithms. In step HOC a decision is taken whether it is an events-based interactive system? If YES, the program advances to step HOD where touch interactions from the tomogram are recognized and touch events are generated. In step HOF, the touch events in step IOOD are received and a user-interface software program running in the data processor modifies is state in response to the touch events. If NO, the program advances to step HOE where a user- interface software program running in the data processor transforms the tomogram and modifies its state. The output of steps HOE and IOOF feed to step HOG where the user interface software provides information representative of its modified state to the output device as feedback of the touch interactions.
[91] Fig. 12 is a flow chart showing an example embodiment for the acquisition of a tomogram representative of touch interactions. In step 120A, parameters are defined
for tomogram acquisition, including emission pattern sequence, set of detector ports associated with each emission pattern, and amount, position and shape of each reconstruction region, using possible embodiment in Fig.15. In step 120B an unreconstructed tomogram representative of the touch interactions over the sensitive surface is acquired by estimating and measuring the ports around the sensitive surface, using possible embodiment in Fig.13. In step 120C the reconstructed tomogram is calculated by transforming the unreconstructed tomogram with a tomographic reconstruction algorithm, using possible embodiment in Fig.14.
[92] Fig. 13 is a flow chart illustrating an example embodiment for the acquisition of an unreconstructed tomogram representative of touch interactions. In step 130A configure a pattern controller with an emission pattern selected in turns from the set of defined emission patterns to energize emitter ports. In step 130B access through a signal sampler, to the measurement values from the set of detector ports associated with the selected emission pattern. In step 130C append the measurement values to the partially acquired unreconstructed tomogram. The output is put to a decision in step 130D regarding is any pattern of the set of defined emission patterns not yet applied to have complete the unreconstructed tomogram acquisition? If YES, then the program jumps or loops back to step 130A; if NO, the program outputs.
[93] Fig. 14 illustrates a flowchart for an example embodiment for the reconstructed tomogram calculation. In step 140A the system provides matrix A, obtains its pseudo inverse A' by truncated SVD In step 140B the unreconstructed tomogram is optionally preprocessed. In step 140Cx: = Kb is calculated, where x is the reconstructed tomogram and b is the unreconstructed tomogram. In step 140D the unreconstructed tomogram is optionally post-processed.
[94] Fig. 15 illustrates a flowchart of an example embodiment for the definition of parameters for tomogram acquisition. In step 150A the amount, positions, and shapes of imaginary or notional reconstruction regions on the sensitive surface are defined. In step 150B amount of emission ports to be involved in unreconstructed tomogram acquisition is defined, and positions selected. In step 150C a linearly independent set of emission patterns is defined. In step 150D amount of detector ports to be involved in unreconstructed tomogram acquisition is defined, and positions selected. In step 150E for each emission pattern a set of associated detector ports is defined. In step 150F the system matrix A is calculated assuming a linear model of the tomographic system. The output of step 150F is passed to a decision in step 150G, namely, is the rank and condition number of the system matrix A enough to allow an adequate posterior reconstruction? If the answer is YES, the signal is output; if the answer is NO, the signal is looped back to step 150A. Detailed description of FTIR based interactive touch device embodiment
[95] A preferred embodiment contemplated for practicing the present invention, is an interactive system with the optical FTIR based embodiment. This embodiment permits the use of a panel with almost perfect light transmission properties, the detection of touch interactions with zero-pressure and also with above-zero pressure, and the use of a very low-cost panel, and other manufacturing advantages, like ease of mounting. This embodiment is detailed as follows.
[96] A conventional LCD display is used as an output device. On top of the display, there's a clear panel made of a sheet of acrylic of less than a few millimeters thick, where infrared light passes from the borders. The objective is to make the screen visible through the panel. The panel is in mechanic contact with the screen by using two-sided tape put on the borders of the screen top, out of its active visible area. To prevent the display making unwanted TIR effect frustration on the panel's bottom surface, the tape is reflective or mirror-like. The tape is thick so if mechanic deformation occurs when pressure is applied with the fingers, the panel doesn't touch the screen. The panel shape is similar to a rectangle, but octagonal with short diagonal corners similar to the panel shown on Fig. 5A. The panel borders are unpolished so to prevent total reflections on the borders and also to provide some diffusion when light enters or exits the panel's border.
[97] Surrounding all panel perimeter, there are infrared LEDs used as emitters and photo transistors used as detectors, touching the panel' borders, so infrared light can enter and exit from the panel. The LED and photo transistors are alternating, each emitter between two detectors and vice versa. The emitters and detectors are spaced apart at regular intervals on each panel edge, separated by about one millimeter. The emitters and detectors are soldered on different Printed Circuit Boards, one for each panel edge.
[98] The emitters are connected to a circuit that works similar to a LED Matrix controller that permits to activate any LED. The possible activation levels for each LED are two: no current or the maximum current allowed for the emitters. Taking into account each LED is active just a short period of time, more than the nominal continuum rate of emission is possible, like in conventional LED Matrices.
[99] The photo transistor detectors have infrared filters matching the emission spectrum of emitters. Each photo transistor makes its current flow through a resistance to convert the current in voltage. The detectors are grouped in groups of eight elements, and each group connected to an 8-to-l analog multiplexer.
[100] Each analog multiplexer is associated with a 14-bit Analog to Digital Converter, and each digital output of the ADCs are connected to a digital multiplexer, so each ADC conversion value can be addressed. The digital multiplexer is a bus-based one, so each non-selected ADC is in high impedance state while the addressed one ADC is in low impedance with respect to the bus.
[101] The LED matrix controller and the digital multiplexer are connected to a FPGA that controls them and is able to execute a Tomographic Reconstruction Algorithm.
[102] The FPGA is connected to a CPU adapted to be able to run an application with an User Interface. The CPU is also connected to the display.
[103] On the panel's top surface a pixilated matrix is defined. The total area matches the screen's visible area. The pixels are square with a side size matching emitter and detectors separations. A vector representing an unreconstructed tomogram is acquired by energizing in turn each emitter, and while each emitter is active, taking a measurement from all detectors. To optimize acquisition time, all ADCs are commanded in parallel to take the measurements selecting a multiplexer channel , repeating this for the eight acquisition channels. The measurements are sent to the FPGA through the bus.
[104] Each time an emitter is energized and measurements are taken, another set of measurements is taken but with all emitters deactivated. This measurement is to read ambient light interference level to be subtracted from the measurement with the emission pattern active. That measurement is taken many times because ambient light can be varying, for example most light bulbs oscillates at 120Hz. To estimate the interference level at the time of measurement with the emitter active, an average with previous and past interference measurement is calculated. This takes into account the fact that photo transistors are not as fast as other optoelectronic devices like PIN photo diodes, so before measurement a delay is inserted to account for phototransistor time response.
[105] With the tomogram acquisition as described, a base measurement representative of the tomogram acquired when no touch interactions occurs, is taken at factory and stored with the device on a provided memory. That base measurement is called bθ. This measurement is taken in every touch device produced, to account possible variations on each specific device properties. For example, not all LED in a batch produces the same level of light emission for same conditions. So this measurement accounts for that possible variation.
[106] A linear model of the tomographic system is assumed. The system matrix A is also calculated at factory by methods used to achieve Algebraic Reconstruction Methods. Each column of matrix A is the unreconstructed tomogram taken when a determinate 'unity' touch interaction is in effect. This unreconstructed tomogram is the change with respect to the base tomogram bθ. The 'unity' touch interaction is an interaction in a determinate region with a shape and position in accordance with that region. Columns of matrix A are representing each reconstruction region. Each 'unity' touch interaction can be made by actually touching the panel with a mechanic arm, or by simulating the effect. Those skilled in the art will know how to simulate the effect but a method is
proposed anyway. Since the rows of system matrix A represents a determinate pair of emitter-detector, for each emitter-detector pair, the row represents all regions associated with that pair. The regions correspond to a pixilated grid and each region has an associated coefficient, so it can be viewed like a 2D gray-scale image canvas. To calculate each coefficient, an anti-aliased 'white' line is drawn between the emitter and detector on a 'black' background, taking into account emitter and detector coordinates. White corresponds to value 1 and black to 0. The 'gray' level of each canvas pixel is then associated with the row coefficient. Then each row coefficient is multiplied with the value of the corresponding pair from base tomogram bθ, to account for actual signal levels involved on the specific device implementation. A rectangular system matrix A is used with more columns that rows, so more measurements are taken than the amount of regions. More measurements than regions are useful to have more precision on tomographic reconstruction. It should be noted, that on all steps for tomographic calculations, information associated with some pairs of emitter-detector are not taken into account. Some are discarded according to the following criteria: if the dot product of corresponding row on A matrix with itself is below a threshold. The threshold is selected in a way that the rank of A matrix isn't much below region count.
[107] Having the system matrix A, a pseudo inverse A' is calculated by the Truncated SVD method, by using MATLAB 's pinv function. The tolerance parameter is calculated as follows. A value, say 1, is selected. Then by computer simulation, the reconstruction noise for a set of simulated touch interactions is calculated. Based on the noise properties of simulation results for each region, the value is lowered or incremented, so an optimization algorithm is run. The objective is to have noise lowered and if possible equally distributed for all the regions. A practical value is sometimes around 0.4. The pseudo inverse A' is also stored on device for later recall.
[108] Once all parameters for tomographic reconstruction are calculated, the device enters an interactive loop that runs continuously.
[109] A tomogram b is acquired as described, with the base tomogram b0 subtracted, so if no touch interactions are occurring, the b vector contains zeros or just normal noise values. A reconstructed tomogram x is calculated by solving the equation x = A' b on the FPGA, by techniques well know to those skilled in the art. The reconstructed tomogram, as represented by vector x, is interpreted as an image representing touch interactions, with each pixel corresponding to each reconstruction region.
[110] The tomogram x is transmitted to the CPU, where it is post processed by a noise gate, thus making zero the values below some threshold. The threshold is by default fixed just above normal reconstruction noise. The default threshold value is also calculated at factory but a noise control knob is provided to the user that can run a configuration software.
[111] On the CPU, a set of current and previous tomograms are analyzed to find touch events and gestures. The software used analyzes the events by executing a Blob Detection and Tracking algorithm. The algorithm can be selected by the user with a configuration application, where adapted versions of the software TOUCHLIB, http://www.nuigroup.com/touchlib/ and OPENTOUCH, http://code.google.eom/p/opentouch/ are offered as options.
[112] After events or gestures are recognized, they are sent by TUIO protocol to applications adapted to work with that protocol. An example application is a GPS mapping application.
[113] The application running on the CPU generates visual information as feedback that is sent to the display to be seen through the panel by the user, thus being that visual response feedback of the control operations introduced by touching the touch device of the present invention.
[114] Although the present invention has been described regarding several illustrative embodiments, numerous variations and modifications will be apparent to those skilled in the art that do not depart from the teachings herein. Such variations and modifications are deemed to fall within the purview of the appended claims.
Claims
[1] A touch screen comprising a panel capable of conducting signals and a tomograph adjacent the panel including signal flow ports arrayed around the border of the panel at discrete locations whereby signals can be introduced into the panel to pass from each discrete border location to a plurality of other discrete border locations for being detected and tomographically processed to determine if any change occurred to signals due to the panel being touched during signal passage through the panel, and therefrom determine any local area on the panel where a change occurred, said tomograph computing and outputting a signal indicative of a panel touch and location.
[2] The invention of claim 1 wherein the signal flow ports are arrayed around the edges of the panel.
[3] The invention of claim 1 wherein the panel is transparent.
[4] The invention of claim 3 wherein a display is operatively coupled to the tomograph for displaying a response to the outputted signal indicative of panel touch and location.
[5] The invention of claim 1 wherein the emitters emit a signal comprised of energy of one of electrical, electromagnetic, magnetic, light, sonic and vibration.
[6] The invention of claim 1 wherein each signal flow port includes one of an emitter and a detector.
[7] The invention of claim 6 wherein the tomograph further includes a controller that establishes patterns for each emitter with a set of detectors, and a multiplexer for serially energizing the emitters.
[8] The invention of claim 1 wherein the tomograph can determine simultaneous touching of the panel at a plurality of local areas.
[9] The invention of claim 1 wherein said tomograph further includes each signal flow port having one of an emitter and a detector, a controller for controlling the emitters to activate an emitter to emit signals into the panel, a signal sampler for sampling signals detected by detectors and a data processor operatively coupled to said controller and said signal sampler whereby signals introduced by an emitter into the panel pass from one discrete border location to a plurality of other discrete border locations whereat the signals are detected by detectors at said other discrete border locations and tomographically processed by the data processor to determine if any change occurred to the signal due to the panel being touched during passage from said one discrete border location to said plurality of other discrete border locations and thereby determine a local area on the panel where a change occurred.
[10] An interactive system comprising a transparent panel, a display located behind the panel visible through the panel, a tomograph adjacent the panel including signal flow ports arrayed around the border of the panel at discrete locations whereby signals can be introduced into the panel to pass from each discrete border location to a plurality of other discrete border locations for being detected and tomographically processed to determine if any change occurred to signals due to the panel being touched during signal passage through the panel, and therefrom determine any local area on the panel where a change occurred, said tomograph computing an output indicative of panel touches and location, and controlling the display responsive to said output.
[11] The invention of claim 10 wherein the signals are comprised of energy of one of electrical, electromagnetic, magnetic, light, sonic and vibration.
[12] The invention of claim 10 wherein each signal flow port includes one of an emitter and a detector, said tomograph further includes a controller for controlling the emitters to emit signals into the panel, a signal sampler for sampling signals detected by detectors and a data processor operatively coupled to said controller and said signal sampler, said tomograph correlated with the panel to define reconstruction regions over the panel surface, the tomograph defining sets of emission patterns and for each emission pattern an associated set of detectors, the tomograph receiving from associated sets of detectors measured values from which to compute a tomogram indicative of a panel touch is computed, and in response thereto the display indicates a panel touch.
[13] The invention of claim 12 wherein the controller that establishes patterns for each emitter with an associated set of detectors, and a multiplexer serially energizes the emitters.
[14] A method for determining touch on a screen comprising the steps of establishing a panel capable of conducting signals, associating a tomograph having signal flow ports adjacent the panel, arraying the signal flow ports around the border of the panel at discrete locations, introducing signals into the panel via the signal flow ports to pass from each discrete border location to a plurality of other discrete border locations, detecting and tomographically processing signals passed through said panel to determine if any change occurred in the panel by being touched during signal passage through the panel, and therefrom determining any local area on the panel where a change occurred, and outputting a signal indicative of panel touch and location.
[15] A method according to claim 14 including the further step of displaying in response to said outputted signal an indication of panel touch and location.
[16] A method according to claim 14 wherein the signal is comprised of energy of
one of electrical, electromagnetic, magnetic, light, sonic and vibration.
[17] A method for operating an interactive system comprising the steps of providing a transparent panel, locating a display behind the panel visible through the panel, arranging a tomograph including signal flow ports adjacent the panel, arraying the signal flow ports around the border of the panel at discrete locations, introducing signals into the panel to pass from each discrete border location to a plurality of other discrete border locations, detecting and tomographically processing signals passed through the panel to determine if any change occurred to signals due to the panel being touched during signal passage through the panel, and therefrom determining any local area on the panel where a change occurred, generating an output indicative of panel touch and location, and controlling the display responsive to said output.
[18] The invention of claim 17 wherein the signals are comprised of energy of one of electrical, electromagnetic, magnetic, light, sonic and vibration.
[19] The invention of claim 17 comprising the further steps of including in each signal flow one of an emitter and a detector, tomographically processing said signals further including controlling the emitters to emit signals into the panel, sampling signals detected by detectors, defining reconstruction regions over the panel surface, defining sets of emission patterns and for each emission pattern an associated set of detectors, receiving measured values from associated sets of detectors, computing a tomogram indicative of a panel touch, and in response thereto displaying an indication thereof.
[20] The invention of claim 19 including the further steps of establishing patterns for each emitter with an associated set of detectors, and multiplexing by serially energizing the emitters.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP08863275A EP2245522A4 (en) | 2007-12-17 | 2008-12-15 | Method and apparatus for tomographic touch imaging and interactive system using same |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
ARP070105651A AR064377A1 (en) | 2007-12-17 | 2007-12-17 | DEVICE FOR SENSING MULTIPLE CONTACT AREAS AGAINST OBJECTS SIMULTANEOUSLY |
ARP20070105651 | 2007-12-17 |
Publications (2)
Publication Number | Publication Date |
---|---|
WO2009077962A2 true WO2009077962A2 (en) | 2009-06-25 |
WO2009077962A3 WO2009077962A3 (en) | 2009-10-22 |
Family
ID=40639611
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/IB2008/055281 WO2009077962A2 (en) | 2007-12-17 | 2008-12-15 | Method and apparatus for tomographic touch imaging and interactive system using same |
Country Status (4)
Country | Link |
---|---|
US (2) | US8803848B2 (en) |
EP (1) | EP2245522A4 (en) |
AR (1) | AR064377A1 (en) |
WO (1) | WO2009077962A2 (en) |
Cited By (55)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2011028170A1 (en) | 2009-09-02 | 2011-03-10 | Flatfrog Laboratories Ab | Touch-sensitive system and method for controlling the operation thereof |
EP2370884A4 (en) * | 2008-12-05 | 2012-05-23 | Flatfrog Lab Ab | A touch sensing apparatus and method of operating the same |
EP2466429A1 (en) | 2010-12-16 | 2012-06-20 | FlatFrog Laboratories AB | Scanning ftir systems for touch detection |
WO2012105893A1 (en) | 2011-02-02 | 2012-08-09 | Flatfrog Laboratories Ab | Optical incoupling for touch-sensitive systems |
WO2013055282A2 (en) | 2011-10-11 | 2013-04-18 | Flatfrog Laboratories Ab | Improved multi-touch detection in a touch system |
WO2013089622A2 (en) | 2011-12-16 | 2013-06-20 | Flatfrog Laboratories Ab | Tracking objects on a touch surface |
JP2013527528A (en) * | 2010-05-03 | 2013-06-27 | フラットフロッグ ラボラトリーズ アーベー | Contact determination by tomographic reconstruction |
WO2013165306A2 (en) | 2012-05-02 | 2013-11-07 | Flatfrog Laboratories Ab | Object detection in touch systems |
WO2013165305A2 (en) | 2012-05-02 | 2013-11-07 | Flatfrog Laboratories Ab | Object detection in touch systems |
EP2706443A1 (en) | 2012-09-11 | 2014-03-12 | FlatFrog Laboratories AB | Touch force estimation in a projection-type touch-sensing apparatus based on frustrated total internal reflection |
WO2014112904A1 (en) | 2013-01-16 | 2014-07-24 | Flatfrog Laboratories Ab | Touch-sensing display panel |
US8803848B2 (en) | 2007-12-17 | 2014-08-12 | Victor Manuel SUAREZ ROVERE | Method and apparatus for tomographic touch imaging and interactive system using same |
US8872801B2 (en) | 2010-12-16 | 2014-10-28 | Flatfrog Laboratories Ab | Touch apparatus with separated compartments |
US8884900B2 (en) | 2011-07-13 | 2014-11-11 | Flatfrog Laboratories Ab | Touch-sensing display apparatus and electronic device therewith |
US8890849B2 (en) | 2011-09-27 | 2014-11-18 | Flatfrog Laboratories Ab | Image reconstruction for touch determination |
US8963886B2 (en) | 2011-07-13 | 2015-02-24 | Flatfrog Laboratories Ab | Touch-sensing display panel |
US8982084B2 (en) | 2011-12-16 | 2015-03-17 | Flatfrog Laboratories Ab | Tracking objects on a touch surface |
WO2015108478A2 (en) | 2014-01-16 | 2015-07-23 | Flatfrog Laboratories Ab | Oled display panel |
WO2015108477A1 (en) | 2014-01-16 | 2015-07-23 | Flatfrog Laboratories Ab | Touch-sensing quantum dot lcd panel |
US9274645B2 (en) | 2010-12-15 | 2016-03-01 | Flatfrog Laboratories Ab | Touch determination with signal enhancement |
US9588619B2 (en) | 2012-01-31 | 2017-03-07 | Flatfrog Laboratories Ab | Performance monitoring and correction in a touch-sensitive apparatus |
US9626040B2 (en) | 2012-05-23 | 2017-04-18 | Flatfrog Laboratories Ab | Touch-sensitive apparatus with improved spatial resolution |
US9639210B2 (en) | 2011-12-22 | 2017-05-02 | Flatfrog Laboratories Ab | Touch determination with interaction compensation |
US9678602B2 (en) | 2012-05-23 | 2017-06-13 | Flatfrog Laboratories Ab | Touch-sensitive apparatus with improved spatial resolution |
US9785287B2 (en) | 2012-12-17 | 2017-10-10 | Flatfrog Laboratories Ab | Optical coupling in touch-sensing systems |
CN107402736A (en) * | 2017-07-21 | 2017-11-28 | 驭势(上海)汽车科技有限公司 | A kind of method for information display and equipment |
US9830019B2 (en) | 2013-01-16 | 2017-11-28 | Flatfrog Laboratories Ab | Touch-sensing LCD panel |
US9857916B2 (en) | 2012-07-24 | 2018-01-02 | Flatfrog Laboratories Ab | Optical coupling in touch-sensing systems using diffusively transmitting element |
US9857917B2 (en) | 2012-12-17 | 2018-01-02 | Flatfrog Laboratories Ab | Optical coupling of light into touch-sensing systems |
US9864470B2 (en) | 2014-05-30 | 2018-01-09 | Flatfrog Laboratories Ab | Enhanced interaction touch system |
US9874978B2 (en) | 2013-07-12 | 2018-01-23 | Flatfrog Laboratories Ab | Partial detect mode |
US9927920B2 (en) | 2011-12-16 | 2018-03-27 | Flatfrog Laboratories Ab | Tracking objects on a touch surface |
US10019113B2 (en) | 2013-04-11 | 2018-07-10 | Flatfrog Laboratories Ab | Tomographic processing for touch detection |
US10126882B2 (en) | 2014-01-16 | 2018-11-13 | Flatfrog Laboratories Ab | TIR-based optical touch systems of projection-type |
US10146376B2 (en) | 2014-01-16 | 2018-12-04 | Flatfrog Laboratories Ab | Light coupling in TIR-based optical touch systems |
US10152176B2 (en) | 2013-11-22 | 2018-12-11 | Flatfrog Laboratories Ab | Touch sensitive apparatus with improved spatial resolution |
US10161886B2 (en) | 2014-06-27 | 2018-12-25 | Flatfrog Laboratories Ab | Detection of surface contamination |
US10168835B2 (en) | 2012-05-23 | 2019-01-01 | Flatfrog Laboratories Ab | Spatial resolution in touch displays |
US10268319B2 (en) | 2012-12-17 | 2019-04-23 | Flatfrog Laboratories Ab | Edge-coupled touch-sensitive apparatus |
US10282035B2 (en) | 2016-12-07 | 2019-05-07 | Flatfrog Laboratories Ab | Touch device |
US10318074B2 (en) | 2015-01-30 | 2019-06-11 | Flatfrog Laboratories Ab | Touch-sensing OLED display with tilted emitters |
US10365768B2 (en) | 2012-12-20 | 2019-07-30 | Flatfrog Laboratories Ab | TIR-based optical touch systems of projection-type |
US10401546B2 (en) | 2015-03-02 | 2019-09-03 | Flatfrog Laboratories Ab | Optical component for light coupling |
EP3537269A1 (en) | 2015-02-09 | 2019-09-11 | FlatFrog Laboratories AB | Optical touch system |
US10437389B2 (en) | 2017-03-28 | 2019-10-08 | Flatfrog Laboratories Ab | Touch sensing apparatus and method for assembly |
US10481737B2 (en) | 2017-03-22 | 2019-11-19 | Flatfrog Laboratories Ab | Pen differentiation for touch display |
US10761657B2 (en) | 2016-11-24 | 2020-09-01 | Flatfrog Laboratories Ab | Automatic optimisation of touch signal |
US11182023B2 (en) | 2015-01-28 | 2021-11-23 | Flatfrog Laboratories Ab | Dynamic touch quarantine frames |
US11256371B2 (en) | 2017-09-01 | 2022-02-22 | Flatfrog Laboratories Ab | Optical component |
US11301089B2 (en) | 2015-12-09 | 2022-04-12 | Flatfrog Laboratories Ab | Stylus identification |
US11474644B2 (en) | 2017-02-06 | 2022-10-18 | Flatfrog Laboratories Ab | Optical coupling in touch-sensing systems |
US11567610B2 (en) | 2018-03-05 | 2023-01-31 | Flatfrog Laboratories Ab | Detection line broadening |
US11709568B2 (en) | 2020-02-25 | 2023-07-25 | Promethean Limited | Convex interactive touch displays and related systems and methods |
US11893189B2 (en) | 2020-02-10 | 2024-02-06 | Flatfrog Laboratories Ab | Touch-sensing apparatus |
US11943563B2 (en) | 2019-01-25 | 2024-03-26 | FlatFrog Laboratories, AB | Videoconferencing terminal and method of operating the same |
Families Citing this family (84)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9471170B2 (en) | 2002-11-04 | 2016-10-18 | Neonode Inc. | Light-based touch screen with shift-aligned emitter and receiver lenses |
US8094137B2 (en) | 2007-07-23 | 2012-01-10 | Smart Technologies Ulc | System and method of detecting contact on a display |
FR2930340B1 (en) * | 2008-04-22 | 2013-03-22 | Total Sa | METHOD FOR SAMPLING HYDROGEN SULFIDE |
US8553014B2 (en) * | 2008-06-19 | 2013-10-08 | Neonode Inc. | Optical touch screen systems using total internal reflection |
US20100079409A1 (en) * | 2008-09-29 | 2010-04-01 | Smart Technologies Ulc | Touch panel for an interactive input system, and interactive input system incorporating the touch panel |
US20100083109A1 (en) * | 2008-09-29 | 2010-04-01 | Smart Technologies Ulc | Method for handling interactions with multiple users of an interactive input system, and interactive input system executing the method |
US8810522B2 (en) * | 2008-09-29 | 2014-08-19 | Smart Technologies Ulc | Method for selecting and manipulating a graphical object in an interactive input system, and interactive input system executing the method |
US9063614B2 (en) | 2009-02-15 | 2015-06-23 | Neonode Inc. | Optical touch screens |
US9158416B2 (en) * | 2009-02-15 | 2015-10-13 | Neonode Inc. | Resilient light-based touch surface |
US20100309169A1 (en) * | 2009-06-03 | 2010-12-09 | Lumio Inc. | Optical Touch Screen with Reflectors |
US8416206B2 (en) * | 2009-07-08 | 2013-04-09 | Smart Technologies Ulc | Method for manipulating a graphic widget in a three-dimensional environment displayed on a touch panel of an interactive input system |
CA2772424A1 (en) * | 2009-09-01 | 2011-03-10 | Smart Technologies Ulc | Interactive input system with improved signal-to-noise ratio (snr) and image capture method |
EP2491479B1 (en) * | 2009-10-19 | 2018-10-03 | FlatFrog Laboratories AB | Extracting touch data that represents one or more objects on a touch surface |
CA2778154A1 (en) * | 2009-10-19 | 2011-04-28 | Flatfrog Laboratories Ab | Determining touch data for one or more objects on a touch surface |
EP2491480A4 (en) * | 2009-10-19 | 2014-07-30 | Flatfrog Lab Ab | Touch surface with two-dimensional compensation |
CN102053757B (en) * | 2009-11-05 | 2012-12-19 | 上海精研电子科技有限公司 | Infrared touch screen device and multipoint positioning method thereof |
US8436833B2 (en) | 2009-11-25 | 2013-05-07 | Corning Incorporated | Methods and apparatus for sensing touch events on a display |
KR101657963B1 (en) * | 2009-12-08 | 2016-10-04 | 삼성전자 주식회사 | Operation Method of Device based on a alteration ratio of touch area And Apparatus using the same |
KR101579091B1 (en) * | 2010-01-07 | 2015-12-22 | 삼성디스플레이 주식회사 | Method for detecting touch position, detecting apparatus of touch position for performing the method and display apparatus having the detecting apparatus of touch position |
US8502789B2 (en) * | 2010-01-11 | 2013-08-06 | Smart Technologies Ulc | Method for handling user input in an interactive input system, and interactive input system executing the method |
US8279190B2 (en) * | 2010-07-02 | 2012-10-02 | Himax Technologies Limited | Filter for removing DC signal and high frequency noise and method thereof for touch sensor |
GB201014053D0 (en) * | 2010-08-23 | 2010-10-06 | St Microelectronics Res & Dev | Optical navigation device |
US9411444B2 (en) | 2010-10-11 | 2016-08-09 | Flatfrog Laboratories Ab | Touch determination by tomographic reconstruction |
CA2826390A1 (en) * | 2011-02-08 | 2012-08-16 | Haworth, Inc. | Multimodal touchscreen interaction apparatuses, methods and systems |
EP2684113A4 (en) * | 2011-03-09 | 2015-01-21 | Flatfrog Lab Ab | Touch determination with signal compensation |
US20140055400A1 (en) | 2011-05-23 | 2014-02-27 | Haworth, Inc. | Digital workspace ergonomics apparatuses, methods and systems |
US9471192B2 (en) | 2011-05-23 | 2016-10-18 | Haworth, Inc. | Region dynamics for digital whiteboard |
CA2836813C (en) | 2011-05-23 | 2020-10-27 | Jeffrey Jon REUSCHEL | Digital whiteboard collaboration apparatuses, methods and systems |
US9465434B2 (en) | 2011-05-23 | 2016-10-11 | Haworth, Inc. | Toolbar dynamics for digital whiteboard |
US9170683B2 (en) * | 2011-07-22 | 2015-10-27 | Rapt Ip Limited | Optical coupler for use in an optical touch sensitive device |
WO2013062471A2 (en) * | 2011-10-27 | 2013-05-02 | Flatfrog Laboratories Ab | Touch determination by tomographic reconstruction |
WO2013038283A1 (en) * | 2011-09-12 | 2013-03-21 | Koninklijke Philips Electronics N.V. | Method of image reconstruction for a filtered back projection in limited angle tomography |
US8941619B2 (en) * | 2011-11-18 | 2015-01-27 | Au Optronics Corporation | Apparatus and method for controlling information display |
CN104160366A (en) | 2011-11-28 | 2014-11-19 | 康宁股份有限公司 | Robust optical touch-screen systems and methods using a planar transparent sheet |
WO2013081894A1 (en) | 2011-11-28 | 2013-06-06 | Corning Incorporated | Optical touch-screen systems and methods using a planar transparent sheet |
US9250794B2 (en) * | 2012-01-23 | 2016-02-02 | Victor Manuel SUAREZ ROVERE | Method and apparatus for time-varying tomographic touch imaging and interactive system using same |
EP2817696A4 (en) | 2012-02-21 | 2015-09-30 | Flatfrog Lab Ab | Touch determination with improved detection of weak interactions |
TW201351263A (en) | 2012-03-09 | 2013-12-16 | Flatfrog Lab Ab | Efficient tomographic processing for touch determination |
WO2013133756A1 (en) | 2012-03-09 | 2013-09-12 | Flatfrog Laboratories Ab | Efficient tomographic processing for touch determination |
US9880653B2 (en) | 2012-04-30 | 2018-01-30 | Corning Incorporated | Pressure-sensing touch system utilizing total-internal reflection |
US9479549B2 (en) | 2012-05-23 | 2016-10-25 | Haworth, Inc. | Collaboration system with whiteboard with federated display |
US9479548B2 (en) | 2012-05-23 | 2016-10-25 | Haworth, Inc. | Collaboration system with whiteboard access to global collaboration data |
US9377900B1 (en) * | 2012-06-18 | 2016-06-28 | Amazon Technologies, Inc. | Optical touch sensor |
WO2014018121A1 (en) | 2012-07-26 | 2014-01-30 | Changello Enterprise Llc | Fingerprint-assisted force estimation |
WO2014018116A1 (en) | 2012-07-26 | 2014-01-30 | Changello Enterprise Llc | Ultrasound-based force sensing and touch sensing |
WO2014018115A1 (en) | 2012-07-26 | 2014-01-30 | Changello Enterprise Llc | Ultrasound-based force sensing of inputs |
WO2014035479A2 (en) | 2012-08-30 | 2014-03-06 | Changello Enterprise Llc | Auto-baseline determination for force sensing |
US9557846B2 (en) | 2012-10-04 | 2017-01-31 | Corning Incorporated | Pressure-sensing touch system utilizing optical and capacitive systems |
US10324565B2 (en) | 2013-05-30 | 2019-06-18 | Neonode Inc. | Optical proximity sensor |
US10282034B2 (en) | 2012-10-14 | 2019-05-07 | Neonode Inc. | Touch sensitive curved and flexible displays |
US9164625B2 (en) | 2012-10-14 | 2015-10-20 | Neonode Inc. | Proximity sensor for determining two-dimensional coordinates of a proximal object |
US9207800B1 (en) | 2014-09-23 | 2015-12-08 | Neonode Inc. | Integrated light guide and touch screen frame and multi-touch determination method |
US9921661B2 (en) | 2012-10-14 | 2018-03-20 | Neonode Inc. | Optical proximity sensor and associated user interface |
US9753559B2 (en) * | 2012-10-19 | 2017-09-05 | Texas Instruments Incorporated | Feedback integrator current source, transistor, and resistor coupled to input |
US9092091B2 (en) * | 2012-11-30 | 2015-07-28 | Rapt Ip Limited | Optical touch tomography |
US11861561B2 (en) | 2013-02-04 | 2024-01-02 | Haworth, Inc. | Collaboration system including a spatial event map |
US10304037B2 (en) | 2013-02-04 | 2019-05-28 | Haworth, Inc. | Collaboration system including a spatial event map |
US9910527B2 (en) | 2013-02-15 | 2018-03-06 | Flatfrog Laboratories Ab | Interpretation of pressure based gesture |
US20160070416A1 (en) * | 2013-04-11 | 2016-03-10 | Flatfrog Laboratories Ab | A Coupling Arrangement, A Panel and a Touch Sensitive System |
WO2014168568A1 (en) * | 2013-04-11 | 2014-10-16 | Flatfrog Laboratories Ab | A printed circuit assembly and a touch sensitive system comprising the assembly |
US9250744B2 (en) * | 2013-04-18 | 2016-02-02 | Dell Products, Lp | Edge to edge touch screen |
CN105308548A (en) * | 2013-06-04 | 2016-02-03 | 内奥诺德公司 | Optical touch screens |
WO2014194944A1 (en) * | 2013-06-05 | 2014-12-11 | Ev Group E. Thallner Gmbh | Measuring device and method for ascertaining a pressure map |
TW201500968A (en) * | 2013-06-21 | 2015-01-01 | Utechzone Co Ltd | Three-dimensional interactive system and interactive sensing method thereof |
WO2015005846A1 (en) * | 2013-07-12 | 2015-01-15 | Flatfrog Laboratories Ab | Touch-sensing apparatus suitable for mass production using unitary optical modules |
US9430097B2 (en) | 2013-09-30 | 2016-08-30 | Synaptics Incorporated | Non-orthogonal coding techniques for optical sensing |
KR102277902B1 (en) * | 2014-09-05 | 2021-07-15 | 삼성전자주식회사 | Object contact pressure meter and methods of manufacturing and measuring the same |
CN104375719A (en) * | 2014-11-21 | 2015-02-25 | 合肥鑫晟光电科技有限公司 | Infrared touch screen, touch detection method and display equipment |
KR102264748B1 (en) * | 2015-03-03 | 2021-06-14 | 삼성디스플레이 주식회사 | Touch device and display including the same |
US10802783B2 (en) | 2015-05-06 | 2020-10-13 | Haworth, Inc. | Virtual workspace viewport following in collaboration systems |
CN104793814A (en) * | 2015-05-12 | 2015-07-22 | 京东方科技集团股份有限公司 | Optical touch control device and touch control display device |
US20170140233A1 (en) * | 2015-11-13 | 2017-05-18 | Fingerprint Cards Ab | Method and system for calibration of a fingerprint sensing device |
US10255023B2 (en) | 2016-02-12 | 2019-04-09 | Haworth, Inc. | Collaborative electronic whiteboard publication process |
WO2018067613A1 (en) | 2016-10-03 | 2018-04-12 | Christopher Harrison | Touch-sensing system |
US11934637B2 (en) | 2017-10-23 | 2024-03-19 | Haworth, Inc. | Collaboration system including markers identifying multiple canvases in multiple shared virtual workspaces |
US11126325B2 (en) | 2017-10-23 | 2021-09-21 | Haworth, Inc. | Virtual workspace including shared viewport markers in a collaboration system |
US12019850B2 (en) | 2017-10-23 | 2024-06-25 | Haworth, Inc. | Collaboration system including markers identifying multiple canvases in multiple shared virtual workspaces |
US10747404B2 (en) * | 2017-10-24 | 2020-08-18 | Microchip Technology Incorporated | Touchscreen including tactile feedback structures and corresponding virtual user interface elements |
WO2019191126A1 (en) * | 2018-03-26 | 2019-10-03 | Christopher Harrison | Touch-sensing system including a touch-sensitive paper |
US11573694B2 (en) | 2019-02-25 | 2023-02-07 | Haworth, Inc. | Gesture based workflows in a collaboration system |
CN115039060A (en) | 2019-12-31 | 2022-09-09 | 内奥诺德公司 | Non-contact touch input system |
US11212127B2 (en) | 2020-05-07 | 2021-12-28 | Haworth, Inc. | Digital workspace sharing over one or more display clients and authorization protocols for collaboration systems |
US11750672B2 (en) | 2020-05-07 | 2023-09-05 | Haworth, Inc. | Digital workspace sharing over one or more display clients in proximity of a main client |
EP4222586A1 (en) | 2020-09-30 | 2023-08-09 | Neonode Inc. | Optical touch sensor |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20040252091A1 (en) | 2003-06-14 | 2004-12-16 | Massachusetts Institute Of Technology | Input device based on frustrated total internal reflection |
WO2007112742A1 (en) | 2006-03-30 | 2007-10-11 | Flatfrog Laboratories Ab | A system and a method of determining a position of a scattering/reflecting element on the surface of a radiation transmissive element |
Family Cites Families (82)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
GB1283915A (en) | 1968-08-23 | 1972-08-02 | Emi Ltd | A method of and apparatus for examination of a body by radiation such as x or gamma radiation |
GB1430088A (en) * | 1972-05-17 | 1976-03-31 | Emi Ltd | Radiography |
US3681110A (en) | 1970-05-05 | 1972-08-01 | Rca Corp | Method of producing a luminescent-screen structure including light-emitting and light-absorbing areas |
US4484179A (en) * | 1980-04-16 | 1984-11-20 | At&T Bell Laboratories | Touch position sensitive surface |
US4448837A (en) * | 1982-07-19 | 1984-05-15 | Oki Densen Kabushiki Kaisha | Pressure-sensitive conductive elastic sheet |
US4746914A (en) * | 1985-02-05 | 1988-05-24 | Zenith Electronics Corporation | Cathode ray tube for use in a touch panel display system |
US4644100A (en) * | 1985-03-22 | 1987-02-17 | Zenith Electronics Corporation | Surface acoustic wave touch panel system |
AR241973A1 (en) | 1985-02-05 | 1993-01-29 | Zenith Electronics Corp | An acoustic wave touch panel device. |
US4700176A (en) * | 1985-02-05 | 1987-10-13 | Zenith Electronis Corporation | Tough control arrangement for graphics display apparatus |
US4791416A (en) * | 1985-02-05 | 1988-12-13 | Zenith Electronics Corporation | Touch control system for controllable apparatus |
DE3601516A1 (en) * | 1986-01-20 | 1987-07-23 | Agie Ag Ind Elektronik | PHOTOELECTRIC BARRIER |
US4746770A (en) * | 1987-02-17 | 1988-05-24 | Sensor Frame Incorporated | Method and apparatus for isolating and manipulating graphic objects on computer video monitor |
US4980646A (en) * | 1988-03-30 | 1990-12-25 | The Trustees Of The University Of Pennsylvania | Impedance tomographic tactile sensor |
US6141000A (en) * | 1991-10-21 | 2000-10-31 | Smart Technologies Inc. | Projection display system with touch sensing on screen, computer assisted alignment correction and network conferencing |
US5646066A (en) * | 1995-03-01 | 1997-07-08 | Texas Instruments Incorporated | Method for forming electrical contact to the optical coating of an infrared detector from the backside of the detector |
US5577309A (en) * | 1995-03-01 | 1996-11-26 | Texas Instruments Incorporated | Method for forming electrical contact to the optical coating of an infrared detector |
US6164135A (en) * | 1995-09-07 | 2000-12-26 | Sonident Anstalt | Method and device for the structure analysis and/or for detecting the position of layered objects |
US5768386A (en) | 1996-05-31 | 1998-06-16 | Transaction Technology, Inc. | Method and system for encrypting input from a touch screen |
US5847405A (en) * | 1997-09-04 | 1998-12-08 | Xerox Corporation | Size or position sensing of intermixed sheets in a sheet stacking tray with sheet edge shadow detection |
US7663607B2 (en) | 2004-05-06 | 2010-02-16 | Apple Inc. | Multipoint touchscreen |
KR100595924B1 (en) * | 1998-01-26 | 2006-07-05 | 웨인 웨스터만 | Method and apparatus for integrating manual input |
DE19961524A1 (en) | 1999-12-20 | 2001-07-05 | Siemens Ag | Method for operating a CT device |
US7157649B2 (en) * | 1999-12-23 | 2007-01-02 | New Transducers Limited | Contact sensitive device |
JP5124071B2 (en) | 1999-12-23 | 2013-01-23 | ニュー トランスデューサーズ リミテッド | Contact detection device |
US7859519B2 (en) * | 2000-05-01 | 2010-12-28 | Tulbert David J | Human-machine interface |
US6864882B2 (en) * | 2000-05-24 | 2005-03-08 | Next Holdings Limited | Protected touch panel display system |
US6690363B2 (en) * | 2000-06-19 | 2004-02-10 | Next Holdings Limited | Touch panel display system |
DE60138118D1 (en) | 2000-09-29 | 2009-05-07 | Ge Med Sys Global Tech Co Llc | SPIRAL ANTIBODY RECONSTRUCTION OF THE HEART OF TOMOGRAPH WITH TWO-DIMENSIONAL DETECTOR ASSEMBLY |
US6570557B1 (en) * | 2001-02-10 | 2003-05-27 | Finger Works, Inc. | Multi-touch system and method for emulating modifier keys via fingertip chords |
KR100347439B1 (en) * | 2001-02-28 | 2002-08-03 | 주식회사 에이터치 | Method of Sensing the Pressure in Touch Screen and Structure of Sensing the Pressure in Touch Screen |
SE0103835L (en) * | 2001-11-02 | 2003-05-03 | Neonode Ab | Touch screen realized by display unit with light transmitting and light receiving units |
US7334485B2 (en) * | 2002-02-11 | 2008-02-26 | Battelle Energy Alliance, Llc | System, method and computer-readable medium for locating physical phenomena |
US6871149B2 (en) * | 2002-12-06 | 2005-03-22 | New Transducers Limited | Contact sensitive device |
GB0228512D0 (en) | 2002-12-06 | 2003-01-15 | New Transducers Ltd | Contact sensitive device |
US6961051B2 (en) * | 2003-01-29 | 2005-11-01 | Visteon Global Technologies, Inc. | Cross-point matrix for infrared touchscreen |
US7629967B2 (en) * | 2003-02-14 | 2009-12-08 | Next Holdings Limited | Touch screen signal processing |
CN1777859B (en) * | 2003-03-12 | 2010-04-28 | 平蛙实验室股份公司 | System and method for determining ray emmitting unit |
WO2004100068A2 (en) * | 2003-05-05 | 2004-11-18 | D3D, L.P. | Optical coherence tomography imaging |
EP1665024B1 (en) | 2003-09-12 | 2011-06-29 | FlatFrog Laboratories AB | A system and method of determining a position of a radiation scattering/reflecting element |
US7442914B2 (en) * | 2003-09-12 | 2008-10-28 | Flatfrog Laboratories Ab | System and method of determining a position of a radiation emitting element |
US7265748B2 (en) * | 2003-12-11 | 2007-09-04 | Nokia Corporation | Method and device for detecting touch pad input |
FR2866726B1 (en) * | 2004-02-23 | 2006-05-26 | Jazzmutant | CONTROLLER BY HANDLING VIRTUAL OBJECTS ON A MULTI-CONTACT TOUCH SCREEN |
US6965836B2 (en) * | 2004-04-19 | 2005-11-15 | Battelle Energy Alliance, Llc | Method and apparatus for two dimensional surface property analysis based on boundary measurement |
US7538759B2 (en) * | 2004-05-07 | 2009-05-26 | Next Holdings Limited | Touch panel display system with illumination and detection provided from a single edge |
WO2006003245A1 (en) * | 2004-07-06 | 2006-01-12 | Upm-Kymmene Corporation | Sensor product for electric field sensing |
US8381135B2 (en) * | 2004-07-30 | 2013-02-19 | Apple Inc. | Proximity detector in handheld device |
US7653883B2 (en) * | 2004-07-30 | 2010-01-26 | Apple Inc. | Proximity detector in handheld device |
US7598949B2 (en) * | 2004-10-22 | 2009-10-06 | New York University | Multi-touch sensing light emitting diode display and method for using the same |
US8599140B2 (en) * | 2004-11-17 | 2013-12-03 | International Business Machines Corporation | Providing a frustrated total internal reflection touch interface |
US8130210B2 (en) * | 2004-11-30 | 2012-03-06 | Avago Technologies Ecbu Ip (Singapore) Pte. Ltd. | Touch input system using light guides |
US7750304B2 (en) | 2004-12-15 | 2010-07-06 | Koninklijke Philips Electronics N.V. | Concurrent reconstruction using multiple bed frames or continuous bed motion |
US20060132454A1 (en) * | 2004-12-16 | 2006-06-22 | Deng-Peng Chen | Systems and methods for high resolution optical touch position systems |
US7595790B2 (en) * | 2005-01-31 | 2009-09-29 | Panasonic Corporation | Pressure sensitive conductive sheet, method of manufacturing the same, and touch panel using the same |
US20090135162A1 (en) * | 2005-03-10 | 2009-05-28 | Koninklijke Philips Electronics, N.V. | System and Method For Detecting the Location, Size and Shape of Multiple Objects That Interact With a Touch Screen Display |
US7705835B2 (en) * | 2005-03-28 | 2010-04-27 | Adam Eikman | Photonic touch screen apparatus and method of use |
WO2006108443A1 (en) | 2005-04-13 | 2006-10-19 | Sensitive Object | Method for determining the location of impacts by acoustic imaging |
US8013845B2 (en) | 2005-12-30 | 2011-09-06 | Flatfrog Laboratories Ab | Optical touch pad with multilayer waveguide |
GB0611032D0 (en) * | 2006-06-05 | 2006-07-12 | Plastic Logic Ltd | Multi-touch active display keyboard |
FR2903207B1 (en) | 2006-06-28 | 2008-11-07 | Jazzmutant Soc Par Actions Sim | MULTIPOINT TOUCH SENSOR WITH ACTIVE MATRIX |
US8086971B2 (en) * | 2006-06-28 | 2011-12-27 | Nokia Corporation | Apparatus, methods and computer program products providing finger-based and hand-based gesture commands for portable electronic device applications |
US8441467B2 (en) * | 2006-08-03 | 2013-05-14 | Perceptive Pixel Inc. | Multi-touch sensing display through frustrated total internal reflection |
EP2047308A4 (en) | 2006-08-03 | 2010-11-24 | Perceptive Pixel Inc | Multi-touch sensing display through frustrated total internal reflection |
JP4567028B2 (en) * | 2006-09-26 | 2010-10-20 | エルジー ディスプレイ カンパニー リミテッド | Liquid crystal display device having multi-touch sensing function and driving method thereof |
US8022941B2 (en) * | 2006-10-12 | 2011-09-20 | Disney Enterprises, Inc. | Multi-user touch screen |
WO2008066004A1 (en) | 2006-11-30 | 2008-06-05 | Sega Corporation | Position inputting apparatus |
US7948477B2 (en) * | 2006-12-15 | 2011-05-24 | Apple Inc. | PET-based touchpad |
US8970501B2 (en) * | 2007-01-03 | 2015-03-03 | Apple Inc. | Proximity and multi-touch sensor detection and demodulation |
US8125455B2 (en) * | 2007-01-03 | 2012-02-28 | Apple Inc. | Full scale calibration measurement for multi-touch surfaces |
DE102007008361B3 (en) * | 2007-02-16 | 2008-04-03 | Isis Sentronics Gmbh | Optical scanning sensor system for contactless optical scanning of object surface, has optical unit fastened to head by fastening device having bearings with bearing parts that are formed such that they cause lateral movement of parts |
WO2008146098A1 (en) | 2007-05-28 | 2008-12-04 | Sensitive Object | Method for determining the position of an excitation on a surface and device for implementing such a method |
WO2008148307A1 (en) | 2007-06-04 | 2008-12-11 | Beijing Irtouch Systems Co., Ltd. | Method for identifying multiple touch points on an infrared touch screen |
WO2008154792A1 (en) | 2007-06-15 | 2008-12-24 | Vtron Technologies Ltd. | Infrared touch screen and multi-point touch positioning method |
CN101689080B (en) * | 2007-06-25 | 2012-08-08 | 诺基亚公司 | Improvements in or relating to user interfaces and associated apparatus and methods |
US8094137B2 (en) * | 2007-07-23 | 2012-01-10 | Smart Technologies Ulc | System and method of detecting contact on a display |
JP5052279B2 (en) * | 2007-09-28 | 2012-10-17 | 富士フイルム株式会社 | Optical tomographic imaging system |
US8232977B2 (en) * | 2007-11-14 | 2012-07-31 | N-Trig Ltd. | System and method for detection with a digitizer sensor |
AR064377A1 (en) | 2007-12-17 | 2009-04-01 | Rovere Victor Manuel Suarez | DEVICE FOR SENSING MULTIPLE CONTACT AREAS AGAINST OBJECTS SIMULTANEOUSLY |
US9489089B2 (en) | 2008-01-25 | 2016-11-08 | Elo Touch Solutions, Inc. | Touch-sensitive panel |
TW200945123A (en) * | 2008-04-25 | 2009-11-01 | Ind Tech Res Inst | A multi-touch position tracking apparatus and interactive system and image processing method there of |
EP2338105B1 (en) * | 2008-08-07 | 2012-12-19 | Owen Drumm | Optical control systems with modulated emitters |
SE533704C2 (en) | 2008-12-05 | 2010-12-07 | Flatfrog Lab Ab | Touch sensitive apparatus and method for operating the same |
US8605046B2 (en) | 2010-10-22 | 2013-12-10 | Pq Labs, Inc. | System and method for providing multi-dimensional touch input vector |
-
2007
- 2007-12-17 AR ARP070105651A patent/AR064377A1/en active IP Right Grant
-
2008
- 2008-12-15 EP EP08863275A patent/EP2245522A4/en not_active Withdrawn
- 2008-12-15 US US12/334,670 patent/US8803848B2/en active Active
- 2008-12-15 WO PCT/IB2008/055281 patent/WO2009077962A2/en active Application Filing
-
2013
- 2013-03-15 US US13/836,136 patent/US9836149B2/en active Active
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20040252091A1 (en) | 2003-06-14 | 2004-12-16 | Massachusetts Institute Of Technology | Input device based on frustrated total internal reflection |
WO2007112742A1 (en) | 2006-03-30 | 2007-10-11 | Flatfrog Laboratories Ab | A system and a method of determining a position of a scattering/reflecting element on the surface of a radiation transmissive element |
Non-Patent Citations (1)
Title |
---|
See also references of EP2245522A4 |
Cited By (97)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9836149B2 (en) | 2007-12-17 | 2017-12-05 | Victor Manuel SUAREZ ROVERE | Method and apparatus for tomographic tough imaging and interactive system using same |
US8803848B2 (en) | 2007-12-17 | 2014-08-12 | Victor Manuel SUAREZ ROVERE | Method and apparatus for tomographic touch imaging and interactive system using same |
US8581884B2 (en) | 2008-12-05 | 2013-11-12 | Flatfrog Laboratories Ab | Touch sensing apparatus and method of operating the same |
EP2370884A4 (en) * | 2008-12-05 | 2012-05-23 | Flatfrog Lab Ab | A touch sensing apparatus and method of operating the same |
EP2983070A1 (en) * | 2008-12-05 | 2016-02-10 | FlatFrog Laboratories AB | A touch sensing apparatus and method of operating the same |
US9442574B2 (en) | 2008-12-05 | 2016-09-13 | Flatfrog Laboratories Ab | Touch sensing apparatus and method of operating the same |
US10048773B2 (en) | 2008-12-05 | 2018-08-14 | Flatfrog Laboratories Ab | Touch sensing apparatus and method of operating the same |
US10474249B2 (en) | 2008-12-05 | 2019-11-12 | Flatfrog Laboratories Ab | Touch sensing apparatus and method of operating the same |
WO2011028170A1 (en) | 2009-09-02 | 2011-03-10 | Flatfrog Laboratories Ab | Touch-sensitive system and method for controlling the operation thereof |
US8686974B2 (en) | 2009-09-02 | 2014-04-01 | Flatfrog Laboratories Ab | Touch-sensitive system and method for controlling the operation thereof |
EP3196739A1 (en) | 2009-09-02 | 2017-07-26 | FlatFrog Laboratories AB | Touch-sensitive system and method for controlling the operation thereof |
KR101840991B1 (en) | 2010-05-03 | 2018-05-04 | 플라트프로그 라보라토리즈 에이비 | Touch determination by tomographic reconstruction |
JP2013527528A (en) * | 2010-05-03 | 2013-06-27 | フラットフロッグ ラボラトリーズ アーベー | Contact determination by tomographic reconstruction |
KR101760539B1 (en) | 2010-05-03 | 2017-07-21 | 플라트프로그 라보라토리즈 에이비 | Touch determination by tomographic reconstruction |
US9274645B2 (en) | 2010-12-15 | 2016-03-01 | Flatfrog Laboratories Ab | Touch determination with signal enhancement |
US9594467B2 (en) | 2010-12-15 | 2017-03-14 | Flatfrog Laboratories Ab | Touch determination with signal enhancement |
US8872098B2 (en) | 2010-12-16 | 2014-10-28 | Flatfrog Laboratories Ab | Scanning FTIR systems for touch detection |
US8872801B2 (en) | 2010-12-16 | 2014-10-28 | Flatfrog Laboratories Ab | Touch apparatus with separated compartments |
EP2466429A1 (en) | 2010-12-16 | 2012-06-20 | FlatFrog Laboratories AB | Scanning ftir systems for touch detection |
EP3173914A1 (en) | 2011-02-02 | 2017-05-31 | FlatFrog Laboratories AB | Optical incoupling for touch-sensitive systems |
US10151866B2 (en) | 2011-02-02 | 2018-12-11 | Flatfrog Laboratories Ab | Optical incoupling for touch-sensitive systems |
US9552103B2 (en) | 2011-02-02 | 2017-01-24 | Flatfrog Laboratories Ab | Optical incoupling for touch-sensitive systems |
WO2012105893A1 (en) | 2011-02-02 | 2012-08-09 | Flatfrog Laboratories Ab | Optical incoupling for touch-sensitive systems |
US8963886B2 (en) | 2011-07-13 | 2015-02-24 | Flatfrog Laboratories Ab | Touch-sensing display panel |
US8884900B2 (en) | 2011-07-13 | 2014-11-11 | Flatfrog Laboratories Ab | Touch-sensing display apparatus and electronic device therewith |
US8890849B2 (en) | 2011-09-27 | 2014-11-18 | Flatfrog Laboratories Ab | Image reconstruction for touch determination |
US9377884B2 (en) | 2011-10-11 | 2016-06-28 | Flatfrog Laboratories Ab | Multi-touch detection in a touch system |
WO2013055282A2 (en) | 2011-10-11 | 2013-04-18 | Flatfrog Laboratories Ab | Improved multi-touch detection in a touch system |
WO2013089622A2 (en) | 2011-12-16 | 2013-06-20 | Flatfrog Laboratories Ab | Tracking objects on a touch surface |
US9927920B2 (en) | 2011-12-16 | 2018-03-27 | Flatfrog Laboratories Ab | Tracking objects on a touch surface |
EP3506069A1 (en) | 2011-12-16 | 2019-07-03 | FlatFrog Laboratories AB | Tracking objects on a touch surface |
US8982084B2 (en) | 2011-12-16 | 2015-03-17 | Flatfrog Laboratories Ab | Tracking objects on a touch surface |
US9639210B2 (en) | 2011-12-22 | 2017-05-02 | Flatfrog Laboratories Ab | Touch determination with interaction compensation |
US9588619B2 (en) | 2012-01-31 | 2017-03-07 | Flatfrog Laboratories Ab | Performance monitoring and correction in a touch-sensitive apparatus |
US10372265B2 (en) | 2012-01-31 | 2019-08-06 | Flatfrog Laboratories Ab | Performance monitoring and correction in a touch-sensitive apparatus |
WO2013165305A2 (en) | 2012-05-02 | 2013-11-07 | Flatfrog Laboratories Ab | Object detection in touch systems |
WO2013165306A2 (en) | 2012-05-02 | 2013-11-07 | Flatfrog Laboratories Ab | Object detection in touch systems |
US9626018B2 (en) | 2012-05-02 | 2017-04-18 | Flatfrog Laboratories Ab | Object detection in touch systems |
US10318041B2 (en) | 2012-05-02 | 2019-06-11 | Flatfrog Laboratories Ab | Object detection in touch systems |
US10001881B2 (en) | 2012-05-23 | 2018-06-19 | Flatfrog Laboratories Ab | Touch-sensitive apparatus with improved spatial resolution |
US10168835B2 (en) | 2012-05-23 | 2019-01-01 | Flatfrog Laboratories Ab | Spatial resolution in touch displays |
US9626040B2 (en) | 2012-05-23 | 2017-04-18 | Flatfrog Laboratories Ab | Touch-sensitive apparatus with improved spatial resolution |
US9678602B2 (en) | 2012-05-23 | 2017-06-13 | Flatfrog Laboratories Ab | Touch-sensitive apparatus with improved spatial resolution |
US9857916B2 (en) | 2012-07-24 | 2018-01-02 | Flatfrog Laboratories Ab | Optical coupling in touch-sensing systems using diffusively transmitting element |
EP2706443A1 (en) | 2012-09-11 | 2014-03-12 | FlatFrog Laboratories AB | Touch force estimation in a projection-type touch-sensing apparatus based on frustrated total internal reflection |
EP3327557A1 (en) | 2012-09-11 | 2018-05-30 | FlatFrog Laboratories AB | Touch force estimation in a projection-type touch-sensing apparatus based on frustrated total internal reflection |
US10088957B2 (en) | 2012-09-11 | 2018-10-02 | Flatfrog Laboratories Ab | Touch force estimation in touch-sensing apparatus |
US9857917B2 (en) | 2012-12-17 | 2018-01-02 | Flatfrog Laboratories Ab | Optical coupling of light into touch-sensing systems |
US9785287B2 (en) | 2012-12-17 | 2017-10-10 | Flatfrog Laboratories Ab | Optical coupling in touch-sensing systems |
US10268319B2 (en) | 2012-12-17 | 2019-04-23 | Flatfrog Laboratories Ab | Edge-coupled touch-sensitive apparatus |
US10365768B2 (en) | 2012-12-20 | 2019-07-30 | Flatfrog Laboratories Ab | TIR-based optical touch systems of projection-type |
US9830019B2 (en) | 2013-01-16 | 2017-11-28 | Flatfrog Laboratories Ab | Touch-sensing LCD panel |
WO2014112904A1 (en) | 2013-01-16 | 2014-07-24 | Flatfrog Laboratories Ab | Touch-sensing display panel |
US10019113B2 (en) | 2013-04-11 | 2018-07-10 | Flatfrog Laboratories Ab | Tomographic processing for touch detection |
US9874978B2 (en) | 2013-07-12 | 2018-01-23 | Flatfrog Laboratories Ab | Partial detect mode |
US10152176B2 (en) | 2013-11-22 | 2018-12-11 | Flatfrog Laboratories Ab | Touch sensitive apparatus with improved spatial resolution |
US10146376B2 (en) | 2014-01-16 | 2018-12-04 | Flatfrog Laboratories Ab | Light coupling in TIR-based optical touch systems |
WO2015108477A1 (en) | 2014-01-16 | 2015-07-23 | Flatfrog Laboratories Ab | Touch-sensing quantum dot lcd panel |
WO2015108478A2 (en) | 2014-01-16 | 2015-07-23 | Flatfrog Laboratories Ab | Oled display panel |
US10126882B2 (en) | 2014-01-16 | 2018-11-13 | Flatfrog Laboratories Ab | TIR-based optical touch systems of projection-type |
US9864470B2 (en) | 2014-05-30 | 2018-01-09 | Flatfrog Laboratories Ab | Enhanced interaction touch system |
US10324566B2 (en) | 2014-05-30 | 2019-06-18 | Flatfrog Laboratories Ab | Enhanced interaction touch system |
US10161886B2 (en) | 2014-06-27 | 2018-12-25 | Flatfrog Laboratories Ab | Detection of surface contamination |
US11182023B2 (en) | 2015-01-28 | 2021-11-23 | Flatfrog Laboratories Ab | Dynamic touch quarantine frames |
US10318074B2 (en) | 2015-01-30 | 2019-06-11 | Flatfrog Laboratories Ab | Touch-sensing OLED display with tilted emitters |
EP3537269A1 (en) | 2015-02-09 | 2019-09-11 | FlatFrog Laboratories AB | Optical touch system |
US11029783B2 (en) | 2015-02-09 | 2021-06-08 | Flatfrog Laboratories Ab | Optical touch system comprising means for projecting and detecting light beams above and inside a transmissive panel |
US10496227B2 (en) | 2015-02-09 | 2019-12-03 | Flatfrog Laboratories Ab | Optical touch system comprising means for projecting and detecting light beams above and inside a transmissive panel |
US10401546B2 (en) | 2015-03-02 | 2019-09-03 | Flatfrog Laboratories Ab | Optical component for light coupling |
EP4075246A1 (en) | 2015-12-09 | 2022-10-19 | FlatFrog Laboratories AB | Stylus for optical touch system |
US11301089B2 (en) | 2015-12-09 | 2022-04-12 | Flatfrog Laboratories Ab | Stylus identification |
US10761657B2 (en) | 2016-11-24 | 2020-09-01 | Flatfrog Laboratories Ab | Automatic optimisation of touch signal |
US11579731B2 (en) | 2016-12-07 | 2023-02-14 | Flatfrog Laboratories Ab | Touch device |
EP3667475A1 (en) | 2016-12-07 | 2020-06-17 | FlatFrog Laboratories AB | A curved touch device |
US10775935B2 (en) | 2016-12-07 | 2020-09-15 | Flatfrog Laboratories Ab | Touch device |
US10282035B2 (en) | 2016-12-07 | 2019-05-07 | Flatfrog Laboratories Ab | Touch device |
US11281335B2 (en) | 2016-12-07 | 2022-03-22 | Flatfrog Laboratories Ab | Touch device |
EP4152132A1 (en) | 2016-12-07 | 2023-03-22 | FlatFrog Laboratories AB | An improved touch device |
US11474644B2 (en) | 2017-02-06 | 2022-10-18 | Flatfrog Laboratories Ab | Optical coupling in touch-sensing systems |
US11740741B2 (en) | 2017-02-06 | 2023-08-29 | Flatfrog Laboratories Ab | Optical coupling in touch-sensing systems |
US10481737B2 (en) | 2017-03-22 | 2019-11-19 | Flatfrog Laboratories Ab | Pen differentiation for touch display |
US11016605B2 (en) | 2017-03-22 | 2021-05-25 | Flatfrog Laboratories Ab | Pen differentiation for touch displays |
US10606414B2 (en) | 2017-03-22 | 2020-03-31 | Flatfrog Laboratories Ab | Eraser for touch displays |
US11099688B2 (en) | 2017-03-22 | 2021-08-24 | Flatfrog Laboratories Ab | Eraser for touch displays |
US11269460B2 (en) | 2017-03-28 | 2022-03-08 | Flatfrog Laboratories Ab | Touch sensing apparatus and method for assembly |
US11281338B2 (en) | 2017-03-28 | 2022-03-22 | Flatfrog Laboratories Ab | Touch sensing apparatus and method for assembly |
US10845923B2 (en) | 2017-03-28 | 2020-11-24 | Flatfrog Laboratories Ab | Touch sensing apparatus and method for assembly |
US10739916B2 (en) | 2017-03-28 | 2020-08-11 | Flatfrog Laboratories Ab | Touch sensing apparatus and method for assembly |
US10606416B2 (en) | 2017-03-28 | 2020-03-31 | Flatfrog Laboratories Ab | Touch sensing apparatus and method for assembly |
US10437389B2 (en) | 2017-03-28 | 2019-10-08 | Flatfrog Laboratories Ab | Touch sensing apparatus and method for assembly |
CN107402736A (en) * | 2017-07-21 | 2017-11-28 | 驭势(上海)汽车科技有限公司 | A kind of method for information display and equipment |
US11256371B2 (en) | 2017-09-01 | 2022-02-22 | Flatfrog Laboratories Ab | Optical component |
US11650699B2 (en) | 2017-09-01 | 2023-05-16 | Flatfrog Laboratories Ab | Optical component |
US11567610B2 (en) | 2018-03-05 | 2023-01-31 | Flatfrog Laboratories Ab | Detection line broadening |
US11943563B2 (en) | 2019-01-25 | 2024-03-26 | FlatFrog Laboratories, AB | Videoconferencing terminal and method of operating the same |
US11893189B2 (en) | 2020-02-10 | 2024-02-06 | Flatfrog Laboratories Ab | Touch-sensing apparatus |
US11709568B2 (en) | 2020-02-25 | 2023-07-25 | Promethean Limited | Convex interactive touch displays and related systems and methods |
Also Published As
Publication number | Publication date |
---|---|
US20130201142A1 (en) | 2013-08-08 |
WO2009077962A3 (en) | 2009-10-22 |
EP2245522A2 (en) | 2010-11-03 |
US8803848B2 (en) | 2014-08-12 |
AR064377A1 (en) | 2009-04-01 |
EP2245522A4 (en) | 2013-03-20 |
US20090153519A1 (en) | 2009-06-18 |
US9836149B2 (en) | 2017-12-05 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9836149B2 (en) | Method and apparatus for tomographic tough imaging and interactive system using same | |
US9250794B2 (en) | Method and apparatus for time-varying tomographic touch imaging and interactive system using same | |
JP6315354B2 (en) | Frequency conversion in touch sensor | |
US10691279B2 (en) | Dynamic assignment of possible channels in a touch sensor | |
JP6487935B2 (en) | System and method for identifying user, hand, and object by orthogonal signaling touch | |
CN104620207B (en) | For the low power run for the optical touch-sensitive device for detecting multi-touch event | |
US20180024674A1 (en) | Fast multi-touch stylus and sensor | |
US20180032179A1 (en) | Fast multi-touch noise reduction | |
US9626018B2 (en) | Object detection in touch systems | |
JP2017510010A (en) | Decimation strategy for input event processing | |
US8754862B2 (en) | Sequential classification recognition of gesture primitives and window-based parameter smoothing for high dimensional touchpad (HDTP) user interfaces | |
US8125458B2 (en) | Detecting finger orientation on a touch-sensitive device | |
US20120056846A1 (en) | Touch-based user interfaces employing artificial neural networks for hdtp parameter and symbol derivation | |
JP2017507406A (en) | Apparatus and method for operating with reduced sensitivity in a touch sensing device | |
US20110210943A1 (en) | Curve-fitting approach to hdtp parameter extraction | |
US20150130769A1 (en) | Object detection in touch systems | |
JP2018502382A (en) | Differential transmission for reducing crosstalk in projected capacitive touch sensors | |
KR20160019401A (en) | Fast multi-touch sensor with user identification techniques | |
US20170285809A1 (en) | Fast multi-touch sensor wtih user identification techniques | |
AU2014232460A1 (en) | Active optical stylus and sensor |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 08863275 Country of ref document: EP Kind code of ref document: A2 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
REEP | Request for entry into the european phase |
Ref document number: 2008863275 Country of ref document: EP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2008863275 Country of ref document: EP |