US9804710B2 - Touchless user interfaces - Google Patents
Touchless user interfaces Download PDFInfo
- Publication number
- US9804710B2 US9804710B2 US14/823,095 US201514823095A US9804710B2 US 9804710 B2 US9804710 B2 US 9804710B2 US 201514823095 A US201514823095 A US 201514823095A US 9804710 B2 US9804710 B2 US 9804710B2
- Authority
- US
- United States
- Prior art keywords
- electronic device
- cavity
- surface portion
- transmitter
- cavities
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
- G06F3/043—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means using propagating acoustic waves
- G06F3/0433—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means using propagating acoustic waves in which the acoustic waves are either generated by a movable member and propagated within a surface layer or propagated within a surface layer and captured by a movable member
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S15/00—Systems using the reflection or reradiation of acoustic waves, e.g. sonar systems
- G01S15/02—Systems using the reflection or reradiation of acoustic waves, e.g. sonar systems using reflection of acoustic waves
- G01S15/06—Systems determining the position data of a target
- G01S15/42—Simultaneous measurement of distance and other co-ordinates
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/52—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S15/00
- G01S7/521—Constructional features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
- G06F2203/041—Indexing scheme relating to G06F3/041 - G06F3/045
- G06F2203/04108—Touchless 2D- digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface without distance measurement in the Z direction
Definitions
- This invention relates to the control of electronic devices through the use of signals, particularly ultrasonic signals, reflected from an input object such as a human hand.
- Ultrasonic gesture recognition uses ultrasound enabled transmitters to transmit signals towards a scene, then receives the echoes from an input object in front of the device, typically the user's hand, using a set of sensing receivers.
- the sensors are thus generally placed on the front of the device.
- MEMS microelectromechanical systems
- speakers and microphones may potentially be small enough to be integrated into media devices such as smart-phones, they conventionally require holes in the front surface of the device in order for their membrane or diaphragm to be in direct contact with the atmosphere and therefore the incoming signals.
- this does not fit with the objective of the physical design, as the holes needed can be seen by the user and so cannot provide a largely unbroken smooth surface as is desired.
- the use of a number of holes allows for access to the inner layers of the device, leaving it vulnerable to damage, and the ingress of moisture, dust etc.
- the present invention aims to address the problems set out above.
- it provides an electronic device having a touchless user interface for providing at least one input to the device, said touchless user interface comprising at least one ultrasound transmitter arrangement arranged to transmit ultrasonic signals and at least one ultrasound receiver arrangement arranged to receive reflections of said ultrasonic signals from an input object, wherein the device further comprises a substantially continuous outer surface portion, wherein said outer surface portion comprises at least one localised zone having a greater compliance for moving in response to impingement by said ultrasonic signals or reflections such that said localised zone forms part of said transmitter arrangement and/or said receiver arrangement.
- the outer surface is altered at certain points to allow a distinguishable response to the ultrasonic signals which can be transmitted or detected from the rear of the surface without requiring holes in the surface.
- the greater compliance could be achieved with a different material to the rest of the surface which is disposed such that it gives the outer surface portion a substantially continuous appearance.
- a hole in the surface e.g. a glass surface
- a plug of more acoustically compliant material e.g. a plastics material
- an outer layer of said outer surface portion is continuous, such that there is no break or boundary in the surface, and the surface is thinner in said localised zones of greater compliance.
- the outer surface portion could be monolithic and the localised greater compliance could simply result from a variation in the thickness of the surface. Additionally or alternatively the outer surface portion could comprise a number of layers with the outer layer being monolithic and thinning of the surface portion as a whole deriving from a thinning in or break in one or more inner layers.
- the Applicant has appreciated that a locally different vibration compliance will give rise to a local change in resonant frequency which allows the effect of ultrasonic vibrations passing through it and/or around it to be determined
- the localised zone has a greater compliance by the surface of the device having reduced thickness.
- the invention provides an electronic device comprising a touchless user interface and having a continuous front face, locally thinner in one or more places to allow greater movement by ultrasonic signals impinging thereon.
- the localised zones could have a raised or indented profile relative to the rest of the surface—e.g. being formed as bumps, dimples, ridges etc. in a set of preferred embodiments the surface is flat.
- a vibrating element is physically coupled to the outer surface in the localised zone.
- the more compliant zone of the surface effectively acts as the diaphragm of the transmitter.
- a self-contained transmitter is provided spaced from the outer surface. In these embodiments the outer surface acts as a secondary ‘transmitter’ driven by impingement of the signal generated by the real transmitter.
- a vibration sensitive element e.g. a piezo-electric film or a contact microphone
- the more compliant zone of the surface effectively acts as the diaphragm of the receiver.
- a self-contained receiver is provided spaced from the outer surface. In these embodiments the outer surface acts as a secondary ‘transmitter’ driven by impingement of the incoming reflected signals.
- the greater compliance of the localised zone may, as set out above, be achieved by the surface simply being thinner in that zone.
- a similar effect can be achieved by measuring the vibration as a point within the depth of the surface layer such that there is less material (e.g. glass) between the measurement point and the exterior which in turn means such that in the localised zone there is less mass of material to be moved by the incoming sound waves.
- a pressure sensitive transducer such as a piezo-electric film or other contact microphone could be inserted into the surface. This could be achieved by forming a hole or slit in the edge of the surface member or layer into which the transducer is inserted.
- the density, the area and the distance of the transducer from the outer surface may be chosen to match to the frequency of the acoustic signal of interest.
- the device comprises an arrangement for sensing movement of said localised zone comprising means for emitting a signal towards the rear of said outer surface in the localised zone and means for detecting said signal after reflection from said surface.
- the signal could be optical, e.g. a laser, or acoustic, e.g. ultrasound. Movement of the outer surface could be measured for example by measuring phase changes in the reflected signal. In the example of a laser this could be done by using an interferometer arrangement such as a Michelson or Fabry-Perot interferometer.
- optical arrangements can be highly sensitive such that a lesser increase in compliance—e.g. less localised thinning—is necessary.
- the back of the outer surface portion, at least in the localised zone is treated to enhance its reflectivity. This could be for example through a reflective coating.
- vibration in the localised zone is measured by directing a measurement signal such as a laser beam through the material of the outer surface portion in order that the beam passes through said material in the localised zone.
- a measurement signal such as a laser beam
- the measurement beam would pass through the thinned section in accordance with such a feature.
- the beam could be introduced at an edge of the surface for example.
- the beam may be arranged to pass substantially parallel to the outer surface.
- the localised zone is tuned to have a resonant frequency within the band of the ultrasonic signals. This may be achieved through one or more of the size (e.g. diameter), thickness and material thereof.
- the localised zone is between 1 and 10 mm wide. It may be circular. Preferably a plurality of localised zones is provided. For example there may be between 2 and 20, e.g. between 4 and 16.
- a plurality of localised zones is provided, each having a different resonant frequency. This allows a broader bandwidth signal to be used whilst retaining the benefit of signal strength optimisation through resonance.
- an acoustic cavity is provided behind the localised zone of greater compliance.
- a cavity would typically be defined on one side by the localised zone of the outer surface portion and have in addition one or more side walls extending away from the localised zone.
- the adjacent, thicker regions of the outer surface portion could provide said side wall(s), at least partially.
- the acoustic cavity may help to enhance the vibration being transmitted or received by the relevant transducer, particularly in arrangements in which a transducer (transmitter and/or receiver) is provided in the cavity remote from the outer surface portion.
- this feature has been found to allow an ultrasound transmitter to be mounted facing away from the front surface from which the ultrasonic signals are required to emanate. This is advantageous as the current design of transmitters is relatively large and the ability to alter the orientation in this way facilitates the compact design of the device.
- the dimensions of the cavity are such that it has a resonant frequency within the band of the ultrasonic signals. Having a resonant cavity can allow optimum transfer of a small vibration through the outer surface portion and thus facilitate reception and/or transmission. Indeed the Applicant has appreciated that it may not be necessary to have a localised zone of greater compliance—e.g.
- the invention provides an electronic device having a touchless user interface for providing at least one input to the device, said touchless user interface comprising at least one ultrasound transmitter arranged to transmit ultrasonic signals having a frequency band and at least one ultrasound receiver arranged to receive reflections of said ultrasonic signals from an input object, wherein the device further comprises a substantially continuous outer surface portion, wherein at least one of the transmitter and the receiver is provided behind said outer surface portion in a resonant cavity having a resonant frequency within said frequency band.
- At least one acoustic cavity is provided behind the outer surface portion as part of said transmitter arrangement or said receiver arrangement, said cavity having an echoic signature based on its shape and the device being arranged to process reflections received by the receiving arrangement to determine a direction from which the reflection was received by taking into account said echoic signature.
- the invention provides an electronic device having a touchless user interface for providing at least one input to the device, said touchless user interface comprising at least one ultrasound transmitter arrangement arranged to transmit ultrasonic signals and at least one ultrasound receiver arrangement arranged to receive reflections of said ultrasonic signals from an input object, wherein the device further comprises a substantially continuous outer surface portion, wherein at least one of the transmitter and the receiver arrangement is provided behind said outer surface portion and comprises an acoustic cavity having an echoic signature based on its shape, the device being arranged to process reflections received by the receiving arrangement to determine a direction from which the reflection was received by taking into account said echoic signature.
- the cavity may be in an area of the outer surface portion which has a localized zone of greater compliance, but this is not necessary; the outer surface portion may be of constant thickness/compliance.
- the arrangements described above are predicated on the realisation that for certain cavity shapes, the transmitted ultrasound signals or reflections thereof from the input object, that traverse the cavity are given a distinctive echoic signature which is dependent on the angle at which they enter or leave the cavity.
- the effect of the cavity can be measured or pre-calculated it can be used to help determine the angle of propagation of the signals or reflections. This may be carried out using a mathematical function, for example a linear inverse, but preferably it is carried out using a convolution function.
- an effect akin to beamforming may be achieved but without employing ⁇ /2 spaced array of transducers—that is an array in which adjacent transducers are separated by a maximum of ⁇ /2 , where ⁇ is the wavelength of the transmitted signals, in order to collect sufficient information to resolve the direction of the input object.
- ⁇ is the wavelength of the transmitted signals
- only a single transducer may be arranged to transmit signals into or receive reflections from the cavity although in other embodiments a plurality is provided, allowing a degree of spatial resolution of vibrations in the cavity.
- the acoustic cavity may have any shape, ideally one which gives a distinctive signature.
- the cavity is elongate—i.e. longer in one dimension than in the perpendicular dimension.
- the longer dimension may be more than 2, 5 or 10 times the shorter dimension,
- Such an elongate cavity may have a constant depth, but in a set of embodiments the depth varies along the length (longest dimension) of the cavity. This prevents the cavity having too strong a directional reaction to incoming signals.
- the cavity shape may be more complex and may comprise complex variation in at least one of its width and its depth, producing a significant signature echo pattern.
- a single acoustic cavity of the kind described above may be provided but in a set of embodiments a plurality of cavities is provided. In a set of embodiments, these cavities are arranged on at least one side of the screen. Preferably, at least two cavities are provided perpendicular to each other on different respective sides of a screen.
- the cavity conforms to the shape and size of the ultrasonic transducer. This may be the case when the ultrasonic transducer is a contact microphone, e.g. based on a resonating film.
- a cavity such as a resonant cavity, containing an ultrasound transducer
- it may be sealed.
- an opening is provided in the cavity providing gaseous communication with the exterior of the device via an aperture on a face of the device away from the outer surface portion specified hereinabove.
- the cavity may be open to the side of the device whereas the outer surface portion is at the front of the device.
- the invention provides an electronic device having a touchless user interface for providing at least one input to the device, said touchless user interface comprising at least one ultrasound transmitter arranged to transmit ultrasonic signals and at least one ultrasound receiver arranged to receive reflections of said ultrasonic signals from an input object, wherein the receiver is provided in a cavity behind a substantially continuous outer surface portion and wherein an opening is provided in the cavity providing gaseous communication with the exterior of the device via an aperture on a face of the device away from the outer surface portion.
- the idea of providing localised zones of different resonant frequencies may be implemented using cavities having different resonant frequencies through having different characteristics—e.g. different sizes.
- the cavities may range in depth, for example between 10% and 90% of the thickness of the outer surface portion. They may also range in width, or radius for circular cavities.
- the multiple cavities could be grouped together and/or arranged around the device. Where they are each provided with a transducer this may of course be provided in the cavity. However this is not essential.
- a composite cavity may be provided having discrete sections with different resonant frequencies but with each section being coupled for communication with a common transducer—e.g. through being open on one side.
- cavities in the bulk material of the outer surface portion may just be filled with air. Alternatively however a different material may be used to fill them fully or partially. The material choice will in general influence the resonant frequency and thus this gives another characteristic which may be varied to achieve the desired resonant frequency. Filling the cavities also helps to preserve structural strength in the surface layer of the device—which is e.g. made of glass.
- cavities could contain a transducer, either individually or within a composite cavity complex, this is not essential.
- a resonant cavity is provided which does not contain a transducer.
- a pocket could be formed in the glass or other material. Vibration of the air or other material inside the cavity may be measured from externally of the pocket—e.g. optically using a laser as is described elsewhere herein. Alternatively, the vibration may be picked up using a pressure sensitive transducer such as a piezoelectric film or contact microphone, as described elsewhere herein outside the cavity (e.g. in a further cavity).
- the apparatus is arranged to generate an estimated location of the input object and to use said estimated location to process said reflections to characterise the location and/or movement of the input object. Assuming that the relative spatial positions of a number of cavities is known, this can allow compensation to be made for the slightly different path length from each to the input object.
- the invention provides an electronic device having a touchless user interface for providing at least one input to the device, said touchless user interface comprising at least one ultrasound transmitter arranged to transmit ultrasonic signals and at least one ultrasound receiver arranged to receive reflections of said ultrasonic signals from an input object, the device further comprising a substantially continuous outer surface portion, wherein at least one of said transmitter and said receiver is disposed beneath said outer surface portion so as to transmit or receive signals through said outer surface portion, the apparatus being arranged to generate an estimated location of said input object and to use said estimated location to process said transmissions and/or reflections to characterise the location and/or movement of the input object.
- a different delay factor is applied to components of different frequencies. This allows compensation for the differing spatial positions of resonant cavities of different frequencies thereby mitigating the smearing effect referred to above.
- the delay factors might be approximated by applying phase shifts when the signals are sufficiently narrowband.
- a plurality of resonant cavities are used for transmission with a single resonant cavity for receiving reflections.
- a delay factor is applied to individual signal streams, i.e. either transmissions or reflections associated with different respective cavities. This may be used to offset the difference in distance between transmitters/receivers and the input object.
- each cavity is arranged to have a specific resonant frequency and therefore transmits and/or receive signals at a specific frequency in order to exploit the resonance. This arrangement may produce a double gain, as in addition to increased overall signal strength the combined signal may have a wider bandwidth than would be possible from a single cavity.
- each resonant cavity transmits a part of a chirp signal, such that a combined signal from a plurality of resonant cavities forms a wideband chirp.
- transmit signals which are substantially non-overlapping in the frequency domain allows the individual signals to be separated after being received and combined.
- a delay function is applied to a combined signal of reflections.
- they may be substantially non-overlapping in the frequency domain, meaning that they can be separated after they are combined, and any delay compensation can be carried out on the combined signal by applying different delays to different frequency bands.
- only one transmitter cavity is used with a plurality of receiver cavities.
- the receiver cavities may be identical, or they may receive different components of a transmitted signal. Compensation may therefore be applied both for the different cavity positions and for the different signal components at each cavity. It can therefore be seen that delay factors may be applied after reflections from an input object have been received at the receiver cavities. This allows a plurality of input object locations to be tested for without retransmitting a signal, as the different candidate adjustments can be applied at the receiving end.
- a set of delay factors is applied based on said estimated location of the input object. This allows a delay to be processed based on a decision of the location of the reflector. It may be assumed that the estimated position is correct, and therefore the relevant delay factors are applied to the transmissions and reflections.
- the apparatus is arranged to test for a number of different locations and delay factors and select a location with the best match.
- the best match may be the strongest signal.
- the number of different locations to be tested may be limited using prior knowledge, for example using a previously measured location or a required region for a particular input. For example, if a particular input is required for a function, only locations within a desired input area may be tested. This limiting of the testing positions may resolve ambiguities in processing of input signals and allow power consumption to be reduced by reducing the number of locations which must be tested.
- an embedded pressure sensitive transducer may be thought of as a narrow cavity which is filled with the transducer.
- the transducer will be made from a material different from the surface (e.g. glass) and may contain an internal cavity, causing a change in density similar to when the cavities are filled with resonating material. This allows the transducer to pick up vibrations due to ultrasonic pulses without the need for a further cavity, although, as mentioned above, this arrangement may also be used with a further cavity.
- the band of the ultrasonic signals is intended to refer to the frequency range in which at least 50% of the transmitted energy is contained.
- the resonant frequency of a cavity is intended to refer either to the centre frequency of the resonance band or to the frequency which has greatest response.
- the outer surface portion could comprise an entire face of the device. This is not essential however. For example there may be a portion of the face which includes one or more apertures for transmitters, receivers buttons etc. The invention might still be advantageously be employed to reduce the number of apparent apertures for the number of transducers.
- the recited outer surface portion may be of a different material to the rest of the front of the device.
- the recited outer surface portion could be made of Plexiglass, which is easier to machine and may have preferable resonance characteristics, whilst the rest of the front of the device is made from Gorilla Glass(TM) available from Corning, Inc. as is conventional. Where two different materials are provided they could be suitably bonded together to make a continuous surface. Additionally or alternatively a thin laminate layer may be provided over both to provide the continuous surface.
- the outer surface portion is disposed on a face of the device intended to face the user in use—i.e. the front face—and the at least one transmitter is arranged to transmit towards an input object in front of the device.
- the outer surface portion includes a touch screen.
- a touch screen it may be possible to integrate the ultrasonic transmission and detection arrangements disclosed herein with the touch screen. For example, if a sufficiently transparent touch screen is used, this can be placed below the outer surface portion but above the ultrasound transducer(s) so that both touch and touchless gesture recognition can be supported from the same part of the screen.
- a non-transparent touch screen may be used, with the transducer(s) being placed above the touch screen. While this would occlude these sections of the touch screen, the affected area may be sufficiently small—e.g. as little as a single pixel—that functionality would not be affected significantly.
- transducers it is not essential for transducers to be intimately integrated with a touch screen as outlined above.
- one or more transducers may be placed around the edge of the touch screen but still under the same outer surface portion as the touch screen such that they do not affect the functionality at all and simplifying the integration of the microphones with the touch screen.
- Such an arrangement can still deliver the benefit of hiding transducers by avoiding the need to provide holes in an otherwise continuous surface.
- this difficulty is addressed by arranging for a plurality of transducers to operate coherently with one another to implement beamforming
- the device may be arranged to process signals received by a plurality of receivers such that linear or other combinations of measured received signals are generated to represent deflections at arbitrary points, to create ‘virtual receivers’ partially to overcome these design limitations.
- the beamforming methods involve using the combined inputs to infer motion of the surface in at least one position which does not actually have a receiver.
- the outer surface portion could comprise any suitable material—e.g. a suitably robust plastics material.
- the outer surface portion comprises glass. Indeed it is envisaged that any glass which is currently used for smart-phone touch screens could be used.
- FIG. 1 shows schematically a touchless gesture system for reference purposes only
- FIG. 2 shows a touchless enabled smart-phone embodying the invention
- FIG. 3 shows a partial cross section of a transmitter portion of the embodiment in FIG. 2 ;
- FIG. 4 shows a cross section of a receiver portion of the embodiment in FIG. 2 ;
- FIG. 5 shows another embodiment of the invention using an optical measuring system
- FIG. 6 shows an alternative optical system of measurement, which uses an interferometer for measuring surface deflections
- FIG. 7 shows a covering glass for a device in accordance with the invention
- FIG. 8 shows a partial cross-section of glass embodying the invention
- FIG. 9 shows a partial cross-section of an alternative embodiment
- FIG. 10 shows a partial cross-section of a third embodiment
- FIG. 11 shows a device in accordance with the embodiments of FIGS. 8 to 10 ;
- FIG. 12 shows a partial cross-section of an embodiment using two different materials
- FIG. 13 shows a partial cross-section of a further embodiment
- FIG. 14 shows a device according to the embodiment of FIG. 13 ;
- FIG. 15 shows an embodiment of the invention using transducers external to the cavities
- FIG. 16 shows an embodiment using a combination of resonating bars and cavities
- FIG. 17 shows an embodiment which does not use cavities
- FIG. 18 shows an alternative embodiment of the device that does not use cavities
- FIG. 19 shows a device according to the embodiments of FIGS. 16 to 18 ;
- FIG. 20 shows an embodiment of the invention which uses time compensation to account for spatially offset cavities
- FIG. 21 shows an alternative embodiment of the invention which uses time compensation to account for spatially offset cavities.
- FIG. 1 shows a prior art illustration of the basic principles of a touchless detection system.
- An electronic device such as a smart phone 2 has a touch screen 3 taking up most of the front surface.
- an ultrasonic transmitter 4 and four ultrasonic receivers 6 , 8 , 10 , and 12 .
- the respective transducers are provided within the body of the phone 2 but corresponding apertures are provided in the front glass layer to allow the air-borne ultrasound to reach them.
- FIGS. 2 to 4 show an embodiment of the invention which uses touchless gesture detection in accordance with the invention.
- FIG. 2 shows an arrangement in which a number of transmitters 16 and receivers 18 are provided at the front of a smart phone 20 , around the edge of the front surface 22 .
- the transmitters 16 and receivers 18 are shown as dotted lines since they are not associated with apertures in the front surface 22 but rather they are disposed under the glass, such that the front surface 22 is flat and continuous as will be explained hereinbelow.
- there are a number of openings around the sides of the device 20 which can be used for buttons 24 or access to acoustic cavities 26 , as well as other things such as sockets.
- FIG. 3 shows a partial cross section of the transmitter portion 16 of the embodiment shown in FIG. 2 .
- the touchless system is combined with a touch display 28 , which is attached between the front piece of glass 30 and the PCB layer 32 .
- the touch display 28 stops short of the side bezel 29 of the phone whereas the front glass 30 extends up to meet the bezel. This means that the although most of the front surface is double-layered, there is a localised zone 30 ′ which is thinner and therefore more compliant to being caused to vibrate by when ultrasound impinges on it.
- the transmitter transducer 34 is received in a cavity 36 formed inside the device and bounded by the localised zone 30 ′.
- the cavity is tuned in size and shape to the centre frequency of the ultrasound with which the transmitter is driven in use in order to encourage resonance. It will be noted that the transducer 34 need not be mounted facing the glass 30 and this facilitates compact construction since the transmitter is a relatively large component.
- FIG. 4 shows a cross section of a receiver portion 18 of the embodiment.
- a MEMS microphone 38 is received in a further cavity 40 . Again the cavity is tuned in size and shape to the centre frequency of the ultrasound with which the transmitter is driven in use in order to encourage resonance.
- a further localised zone 30 ′′ in which the glass 30 is thinner is provided by a break in the touch screen 28 . Again this provides an area of greater compliance to movement above the microphone 38 .
- the cavity 40 in this Figure is not sealed as there is an opening 46 in the side wall 48 of the device.
- the transmitter 16 transmits an ultrasonic signal towards the user. This is achieved by the transmitter transducer 34 emitting an ultrasonic signal into the cavity 36 , which is then reflected from both the base and the side in resonance, before being transmitted through the localised zone 30 ′ of the front protective glass through vibrations thereof.
- the transmitted signal is reflected from an input object, i.e. a user's hand as described with reference to FIG. 1 , before being received by the receiving portion 18 of the device 20 .
- the microphone 38 is able to pick up the signal due to the vibration of the glass 30 ′′ replicating the signal within the cavity 40 , as it acts as a diaphragm for the receiver 18 .
- the opening 46 in the cavity allows for the microphone 38 also to detect audible signals for other purposes such as adaptive noise cancellation.
- the ultrasonic signals received will then be processed in order to calculate the motion of the input object as previously described.
- the receivers 18 Due to the positioning of the receivers 18 , i.e. around the edges of the touch screen portion 22 , they may not be in ideal positions for complete functionality of the touchless technology. However, through the use of beamforming methods, the inputs of the receivers 18 that are already available on the device may be combined to infer motion of the surface in positions where there is not actually a receiver 18 .
- FIG. 5 shows an alternative embodiment of the system which uses light signals to measure the vibrations of the acoustic membrane 50 .
- the cover glass 52 of the device is thinned such that there is a cavity 54 over an optical read out unit 56 .
- the entire device is covered by a hard surface covering 58 , which provides protection for the device and ensures a smooth front covering.
- an acoustic pressure wave 60 causes the membrane 50 to vibrate.
- the light signals 62 from the optical unit 56 are reflected by the membrane 50 , and these reflections are registered and measured by the optical read out unit 56 in order to calculate the motion of the input object.
- FIG. 6 shows an embodiment of the system which uses a Michelson interferometer 64 for measurement of the vibrations caused by incoming ultrasonic waves.
- the acoustic membrane 66 has a cavity 68 etched into it, which is aligned with a gap in the touch and display layer 70 directly below.
- a Michelson interferometer 64 In line with the cavity 68 created by the two layers is a Michelson interferometer 64 .
- This is formed from a light source 72 , a beam splitter 74 , a reference mirror 76 and a photo detector 78 .
- the light beam from the source 72 is split at the beam splitter 74 , with part travelling to the acoustic membrane 66 , and part to the reference mirror 76 .
- This interference pattern can be used to determine the path difference between the two beams, from which the distance travelled to the acoustic membrane and therefore the degree to which it was displaced by vibrations can be measured. This allows for very accurate measurement of the movement of the glass.
- FIG. 7 shows a cross-section of a front layer 80 of a further device in accordance with the invention.
- three cavities 82 of different sizes are formed in a monolithic piece of glass, but the number of cavities could vary.
- the cavities range in depth from 40% to 90% of the glass depth, and are each tuned to a different resonant frequency.
- Each cavity 82 has its own ultrasonic receiver (not shown), tuned to that particular wavelength. As there are a number of receivers operating at different wavelengths, the overall effect is that of a broadband device, but as each cavity is operating at resonance, the signals are all maximised, which would not happen with a single cavity and broadband receiver.
- FIG. 8 shows another embodiment using a combination of different sized cavities 85 , each containing a narrow band receiver 84 .
- the cavities vary in width rather than height as in FIG. 7 . It is also possible to have an embodiment in which both the height and width of the cavities vary. This would require less variation in a particular direction to achieve the same change in resonant frequency.
- the device may instead use a composite cavity 86 , as shown in FIG. 9 .
- the cavity 86 has several discrete sections 87 of different dimensions which each have their own resonant frequency. However, these communicate with a common broadband receiver 81 .
- An intermediate layer 88 is provided which channels the vibrations from the discrete sections 87 to the receiver 84 . While this embodiment requires fewer receivers, it does not have the benefit of the receiver being able to be tuned to a particular frequency, so a better broad response is required to achieve similar performance.
- a plastic or glass spacer 83 can be introduced in order to adjust the size of the resonant cavities 89 without changing the thickness of the glass 80 .
- the receivers 84 are disposed below the lower surface of the glass 80 , changing the resonant frequencies of the cavities. This could be used in combination with cavities that vary in height or width, or alternatively in addition to a composite cavity.
- FIG. 11 shows a device demonstrating the possible positions of a series of cavities 82 in accordance with the previously described embodiments. They are shown as dotted lines because the cavities are under the upper surface of the device.
- the cavities 82 are arranged such that they could be used with either the embodiment of FIG. 8 or FIG. 9 . Such a combination of cavities 82 could be repeated around the outer region of the device for example.
- FIG. 12 shows a cross section of an embodiment where the cavities in the covering glass 91 are filled with a different material 90 rather than being filled with air as in the previous figures.
- a different material 90 rather than being filled with air as in the previous figures.
- an array of cavities of the same size is used, but filled cavities could be used for combinations of cavities as in FIGS. 7 to 10 .
- the material may act to enhance resonance.
- FIG. 13 shows a partial cross section of a device which uses a combination of different glass types.
- a number of cavities 92 (here only one is shown) are formed in Plexiglass 94 , with the touch screen portion of the screen being formed from Gorilla Glass(TM) 96 .
- the two types of glass are covered with a thin laminate layer 98 , providing a smooth, uninterrupted finish for the device.
- An example of the division between the two types of glass is shown in FIG. 14 , where the Gorilla Glass(TM) 96 is bordered at the top and bottom by Plexiglass 94 containing cavities and transducers (not shown) at the top and bottom of the device.
- the proportions of Plexiglass 98 and Gorilla Glass (TM) 96 may be altered, preferably such that the area of Plexiglass 98 is minimised, but it can also be changed in order to maximise the coverage by ultrasonic signals for the device, to improve the quality of the touchless technology.
- FIG. 15 shows an alternative embodiment in which the transducers are not contained within the cavities.
- a laser beam 108 is sent directly between the transmitter 100 and receiver 102 across the top of the cavity.
- the amplitude of the received signal changes as the beam passes nearer to or further from the surface of the cavity.
- the vibrations of the cavity 106 cause the amplitude of the laser beam 108 to change over time. The variation in this curve can be plotted and mapped to find the vibration at a particular time.
- FIG. 16 shows an alternative embodiment of a cover glass of a device.
- the cavity 113 ′ in the glass 114 ′ conforms to the shape of the ultrasonic transducer.
- the transducer is a pressure-sensitive transducer, in the form of a film-type contact microphone 110 ′ which is placed within a cavity 113 ′ in the glass 114 ′.
- An AD converter 116 ′ is attached to the resonating film, which is connected via circuitry 118 ′ to the rest of the device.
- the microphone 110 ′ vibrates according to the vibrations of the incoming ultrasonic pulses, and the AD converter 116 ′ then converts the vibrations into digital data which can be read out.
- a pressure-sensitive transducer as used above may be combined with a second cavity 112 , placed above the conforming cavity 113 .
- This may be a resonant cavity, acting to enhance the vibrations caused by the incoming ultrasonic waves.
- the transducer can therefore be placed further from the surface of the glass.
- FIG. 18 also shows an embodiment of the device using a microphone 110 ′′ embedded in the cover glass 114 ′′.
- the microphone 110 ′′ is placed at the edge of the device, such that only the microphone itself is enclosed in the cover glass 114 ′′, rather than the circuitry. This reduces the size of the slit needed to insert the device into the glass 114 ′′, as all circuitry is connected from the side of the glass, rather than being held with the microphone under the glass.
- FIG. 19 shows a device using microphones as shown in FIGS. 16 to 18 .
- the microphones are placed at the top and bottom of the device, such that they do not interfere with the touch screen region 122 of the device. Alternatively, they could be placed down the sides of the device, or around the entire perimeter.
- FIG. 20 shows an embodiment of the invention in which a plurality of transmitter cavities 124 and receiver cavities 126 are used.
- Each of the transmitter cavities 124 and receiver cavities 126 is connected to delay logic 128 , 130 .
- the receiver cavities 126 are then connected to a combining element 132 .
- delay factors are applied both to the transmitted signals and received signals.
- a predicted location of an input object is used to apply delay factors to each of the cavities 126 using the delay logic 128 .
- To apply the right compensating delay factor to the transmitted signals the input object position should be known prior to the transmissions. However, perfect knowledge of the position of the input object cannot be expected, especially since the objective of the transmissions is typically to locate the position of the input object.
- the compensating time shifts may be derived from approximate knowledge of the position of the input object. For example, in the event of active tracking the position estimate obtained in the previous time-step can be used as a proxy. Even though this may produce an error or mismatch the net result may be considerably better than if no compensation was applied.
- Signals are then transmitted from each of the cavities 124 towards said input object, before being reflected and received by the receiver cavities 126 .
- a second delay factor is then applied using delay logic 130 to each of the received signals. After the delay factor is added, the received signals can then be combined using the combining element 132 , generating the output signal.
- FIG. 21 shows an alternative embodiment of the invention in which a plurality of transmitter cavities 124 and receiver cavities 126 are used. Equal numbers of transmitter cavities 124 and receiver cavities 126 are used, so that there is both a transmitter cavity 124 and a receiver cavity 126 with each resonant frequency, which are substantially non-overlapping.
- the receiver cavities 126 are connected to a combining element 134 , which is then connected to delay logic 136 .
- the transmitter cavities 124 transmit signals towards an input object, which are then reflected to the receiver cavities 126 .
- the received signals are then combined at the combining element 124 , before delay factors are applied to the combined signal by the delay logic 136 .
- Different cavities may require different delay factors, but as the cavities have substantially non-overlapping frequencies, different frequency delay factors can be applied to the combined signal, impacting on only the necessary portion of the signal. This allows the hardware to be simplified significantly, as only one delay element 136 is needed for n cavities, as opposed to ⁇ delay elements 128 , 130 for the arrangement of FIG. 20 .
- FIG. 21 could be said to show a plurality of transmitter cavities 124 with one overall receiver.
- the received signals y(t) may be represented as
- x i (t) is the signal transmitted from the th transmitter cavity 124
- ⁇ i is the round trip delay from the th transmitter cavity 124 via the input object and back to the receiver.
- the received signal may therefore be represented as a superposition of delayed replicas of the transmitted signals.
- the round trip delay may vary between different cavities.
- a delay factor may be applied to a transmitted signal to account for a variation in spatial location.
- the received signal may in this case be represented by
- the applied delay factors may be dependent on both the positions of the cavities and the position of the input object.
- any of these embodiments for the screen can be used in combination with the methods of measurement mentioned above.
- the combination of cavities of different sizes provides a wider range of resonant frequencies, increasing the bandwidth available for use in the device.
- a touchless gesture detection system can be implemented without needing to breach the continuous glass surface of a smart phone or other device thereby allowing a large number of transducers without compromising the aesthetic design of the device.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Remote Sensing (AREA)
- Radar, Positioning & Navigation (AREA)
- Human Computer Interaction (AREA)
- Computer Networks & Wireless Communication (AREA)
- Acoustics & Sound (AREA)
- Position Input By Displaying (AREA)
- Transducers For Ultrasonic Waves (AREA)
- Measurement Of Velocity Or Position Using Acoustic Or Ultrasonic Waves (AREA)
Applications Claiming Priority (5)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
GB1302701.6 | 2013-02-15 | ||
GB201302701A GB201302701D0 (en) | 2013-02-15 | 2013-02-15 | Touchless user interface |
GB201305124A GB201305124D0 (en) | 2013-03-20 | 2013-03-20 | Touchless user interfaces |
GB1305124.8 | 2013-03-20 | ||
PCT/GB2014/050443 WO2014125294A1 (fr) | 2013-02-15 | 2014-02-14 | Interfaces utilisateurs sans contact |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/GB2014/050443 Continuation WO2014125294A1 (fr) | 2013-02-15 | 2014-02-14 | Interfaces utilisateurs sans contact |
Publications (2)
Publication Number | Publication Date |
---|---|
US20160026340A1 US20160026340A1 (en) | 2016-01-28 |
US9804710B2 true US9804710B2 (en) | 2017-10-31 |
Family
ID=50231451
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/823,095 Active US9804710B2 (en) | 2013-02-15 | 2015-08-11 | Touchless user interfaces |
Country Status (3)
Country | Link |
---|---|
US (1) | US9804710B2 (fr) |
EP (1) | EP2956840B1 (fr) |
WO (1) | WO2014125294A1 (fr) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20220129092A1 (en) * | 2019-03-15 | 2022-04-28 | Elliptic Laboratories As | Touchless interaction using audio components |
US11581864B2 (en) * | 2019-03-15 | 2023-02-14 | Elliptic Laboratories As | Touchless interaction using audio components |
Families Citing this family (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2015022498A1 (fr) * | 2013-08-15 | 2015-02-19 | Elliptic Laboratories As | Interfaces utilisateur sans contact |
US9501810B2 (en) * | 2014-09-12 | 2016-11-22 | General Electric Company | Creating a virtual environment for touchless interaction |
GB201421427D0 (en) | 2014-12-02 | 2015-01-14 | Elliptic Laboratories As | Ultrasonic proximity and movement detection |
US9811212B2 (en) * | 2015-02-25 | 2017-11-07 | Microsoft Technology Licensing, Llc | Ultrasound sensing of proximity and touch |
CN105183245B (zh) * | 2015-08-31 | 2019-07-26 | 联想(北京)有限公司 | 一种信息处理方法及电子设备 |
US9633546B2 (en) * | 2015-09-11 | 2017-04-25 | WashSense, Inc. | Touchless compliance system |
GB201602319D0 (en) | 2016-02-09 | 2016-03-23 | Elliptic Laboratories As | Proximity detection |
WO2018053956A1 (fr) * | 2016-09-26 | 2018-03-29 | 华为技术有限公司 | Procédé d'interaction et dispositif portable |
JP6782357B2 (ja) * | 2016-09-29 | 2020-11-11 | 北京小米移動軟件有限公司Beijing Xiaomi Mobile Software Co.,Ltd. | 端末、タッチ認識方法、装置及び電子機器 |
EP3755007A1 (fr) | 2019-06-19 | 2020-12-23 | Infineon Technologies AG | Dispositif pour détecter un mouvement d'une surface déflectrice |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20030068056A1 (en) | 2001-10-08 | 2003-04-10 | Roland Aubauer | Mobile communications terminal with flat loudspeaker disposed in the terminal housing |
US20060140438A1 (en) | 2004-12-28 | 2006-06-29 | Nec Corporation | Flat panel speaker, electronic device having same, and structure and method for mounting same |
US20120162143A1 (en) | 2009-08-31 | 2012-06-28 | Yoshihiro Kai | Mount structure of touch panel with vibration function |
US20120230517A1 (en) * | 2011-03-10 | 2012-09-13 | Fang-Ching Lee | Electronic device |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7839394B2 (en) * | 2007-01-08 | 2010-11-23 | Pegasus Technologies Ltd. | Electronic pen device |
CA2718680C (fr) | 2008-03-18 | 2016-12-06 | Elliptic Laboratories As | Detection d'objet et de mouvement |
EP2271129A1 (fr) * | 2009-07-02 | 2011-01-05 | Nxp B.V. | Transducteur avec cavité résonante |
US9367178B2 (en) * | 2009-10-23 | 2016-06-14 | Elliptic Laboratories As | Touchless interfaces |
EP2482164B1 (fr) * | 2011-01-27 | 2013-05-22 | Research In Motion Limited | Dispositif électronique portable et son procédé |
-
2014
- 2014-02-14 WO PCT/GB2014/050443 patent/WO2014125294A1/fr active Application Filing
- 2014-02-14 EP EP14708083.2A patent/EP2956840B1/fr active Active
-
2015
- 2015-08-11 US US14/823,095 patent/US9804710B2/en active Active
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20030068056A1 (en) | 2001-10-08 | 2003-04-10 | Roland Aubauer | Mobile communications terminal with flat loudspeaker disposed in the terminal housing |
US20060140438A1 (en) | 2004-12-28 | 2006-06-29 | Nec Corporation | Flat panel speaker, electronic device having same, and structure and method for mounting same |
US20120162143A1 (en) | 2009-08-31 | 2012-06-28 | Yoshihiro Kai | Mount structure of touch panel with vibration function |
US20120230517A1 (en) * | 2011-03-10 | 2012-09-13 | Fang-Ching Lee | Electronic device |
Non-Patent Citations (2)
Title |
---|
International Search Report dated Jun. 4, 2014 from International Application No. PCT/GB2014/050443. |
Written Opinion dated Jun. 4, 2014 from International Application No. PCT/GB2014/050443. |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20220129092A1 (en) * | 2019-03-15 | 2022-04-28 | Elliptic Laboratories As | Touchless interaction using audio components |
US11543921B2 (en) * | 2019-03-15 | 2023-01-03 | Elliptic Laboratories As | Touchless interaction using audio components |
US11581864B2 (en) * | 2019-03-15 | 2023-02-14 | Elliptic Laboratories As | Touchless interaction using audio components |
Also Published As
Publication number | Publication date |
---|---|
US20160026340A1 (en) | 2016-01-28 |
WO2014125294A1 (fr) | 2014-08-21 |
EP2956840B1 (fr) | 2023-05-10 |
EP2956840A1 (fr) | 2015-12-23 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9804710B2 (en) | Touchless user interfaces | |
KR102618684B1 (ko) | Mems 구성요소 및 mems 구성요소를 포함하는 모바일 디바이스 | |
US20170193768A1 (en) | Calibration and Detection Techniques in Haptic Systems | |
US20160291716A1 (en) | In-air ultrasonic rangefinding and angle estimation | |
AU778231B2 (en) | Contact sensitive device | |
CN208722170U (zh) | 触摸和力敏设备,电子设备及可穿戴音频设备 | |
Dahl et al. | Applications of airborne ultrasound in human–computer interaction | |
FR2948787A1 (fr) | Dispositif et procede de localisation d'un contact localement deformant sur une surface tactile deformable d'un objet | |
JP2010521861A (ja) | Memsマイクロホン | |
US20150071030A1 (en) | Ultrasonic measurement apparatus, ultrasonic imaging apparatus, and ultrasonic measurement method | |
CN104781766A (zh) | 对物体的触敏表面上的至少一个接触进行检测和定位的系统和方法 | |
JP2011082624A (ja) | 近接超音波センサ | |
US20130233080A1 (en) | System for detecting and locating a disturbance in a medium, and corresponding method and computer program | |
US20220379346A1 (en) | Ultrasonic transducers | |
Anzinger et al. | Broadband acoustical MEMS transceivers for simultaneous range finding and microphone applications | |
Cho et al. | Impact of array tilt on source-range estimation in shallow water using the array invariant | |
US11698701B1 (en) | Force sensing in touch sensor by directivity control of transmit transducers | |
Anzinger et al. | Low power capacitive ultrasonic transceiver array for airborne object detection | |
WO2002030099A1 (fr) | Procede et dispositif de reduction d'echo | |
EP2710585B1 (fr) | Transducteur pour systèmes acoustiques en réseau à commande de phase | |
US8422715B2 (en) | Microphone unit | |
EP2425317B1 (fr) | Système de transcription numérique utilisant des détecteurs acoustiques ayant des ouvertures à orientation verticale par rapport à la surface de travail | |
KR20200022164A (ko) | 센싱 기능을 갖는 압전 액추에이터 스피커 | |
Durofchalk et al. | Analysis of the ray-based blind deconvolution algorithm for shipping sources | |
US20220237940A1 (en) | Ultrasonic imaging device and method for image acquisition in the ultrasonic device |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: ELLIPTIC LABORATORIES AS, NORWAY Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:DAHL, TOBIAS GULDEN;BANG, HANS JORGEN;KAVLI, TOM;SIGNING DATES FROM 20150825 TO 20150831;REEL/FRAME:036803/0773 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YR, SMALL ENTITY (ORIGINAL EVENT CODE: M2551); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY Year of fee payment: 4 |