US20120299870A1 - Wearable Heads-up Display With Integrated Finger-tracking Input Sensor - Google Patents
Wearable Heads-up Display With Integrated Finger-tracking Input Sensor Download PDFInfo
- Publication number
- US20120299870A1 US20120299870A1 US13/477,547 US201213477547A US2012299870A1 US 20120299870 A1 US20120299870 A1 US 20120299870A1 US 201213477547 A US201213477547 A US 201213477547A US 2012299870 A1 US2012299870 A1 US 2012299870A1
- Authority
- US
- United States
- Prior art keywords
- finger
- display
- input device
- processor
- display element
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
- 230000033001 locomotion Effects 0.000 claims abstract description 36
- 238000000034 method Methods 0.000 claims abstract description 30
- 239000011521 glass Substances 0.000 claims description 36
- 229920003023 plastic Polymers 0.000 claims description 11
- 239000004033 plastic Substances 0.000 claims description 10
- 238000010897 surface acoustic wave method Methods 0.000 claims description 7
- 238000012634 optical imaging Methods 0.000 claims description 5
- 239000000758 substrate Substances 0.000 claims description 3
- 238000004519 manufacturing process Methods 0.000 claims 5
- 230000006870 function Effects 0.000 description 35
- 238000004891 communication Methods 0.000 description 18
- 230000003993 interaction Effects 0.000 description 7
- 230000008569 process Effects 0.000 description 7
- 230000008859 change Effects 0.000 description 6
- 230000004044 response Effects 0.000 description 6
- 238000004590 computer program Methods 0.000 description 5
- 239000000463 material Substances 0.000 description 5
- 230000003190 augmentative effect Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 239000002184 metal Substances 0.000 description 4
- 229910052751 metal Inorganic materials 0.000 description 4
- 230000005043 peripheral vision Effects 0.000 description 4
- 230000001133 acceleration Effects 0.000 description 3
- 230000009471 action Effects 0.000 description 3
- 230000005686 electrostatic field Effects 0.000 description 3
- 239000007787 solid Substances 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- LYCAIKOWRPUZTN-UHFFFAOYSA-N Ethylene glycol Chemical compound OCCO LYCAIKOWRPUZTN-UHFFFAOYSA-N 0.000 description 2
- 239000004698 Polyethylene Substances 0.000 description 2
- 238000013459 approach Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 230000001413 cellular effect Effects 0.000 description 2
- 239000011248 coating agent Substances 0.000 description 2
- 238000000576 coating method Methods 0.000 description 2
- 239000004020 conductor Substances 0.000 description 2
- 230000004438 eyesight Effects 0.000 description 2
- 210000003128 head Anatomy 0.000 description 2
- AMGQUBHHOARCQH-UHFFFAOYSA-N indium;oxotin Chemical compound [In].[Sn]=O AMGQUBHHOARCQH-UHFFFAOYSA-N 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 229910044991 metal oxide Inorganic materials 0.000 description 2
- 150000004706 metal oxides Chemical class 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- -1 polyethylene Polymers 0.000 description 2
- 229920000573 polyethylene Polymers 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 210000001525 retina Anatomy 0.000 description 2
- 238000010079 rubber tapping Methods 0.000 description 2
- 239000005341 toughened glass Substances 0.000 description 2
- 239000012780 transparent material Substances 0.000 description 2
- FERIUCNNQQJTOY-UHFFFAOYSA-M Butyrate Chemical compound CCCC([O-])=O FERIUCNNQQJTOY-UHFFFAOYSA-M 0.000 description 1
- FERIUCNNQQJTOY-UHFFFAOYSA-N Butyric acid Natural products CCCC(O)=O FERIUCNNQQJTOY-UHFFFAOYSA-N 0.000 description 1
- 229920004142 LEXAN™ Polymers 0.000 description 1
- 239000004418 Lexan Substances 0.000 description 1
- 239000004642 Polyimide Substances 0.000 description 1
- NIXOWILDQLNWCW-UHFFFAOYSA-N acrylic acid group Chemical group C(C=C)(=O)O NIXOWILDQLNWCW-UHFFFAOYSA-N 0.000 description 1
- 239000003570 air Substances 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- XMQFTWRPUQYINF-UHFFFAOYSA-N bensulfuron-methyl Chemical compound COC(=O)C1=CC=CC=C1CS(=O)(=O)NC(=O)NC1=NC(OC)=CC(OC)=N1 XMQFTWRPUQYINF-UHFFFAOYSA-N 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 239000003990 capacitor Substances 0.000 description 1
- 229920006217 cellulose acetate butyrate Polymers 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000001816 cooling Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 210000005069 ears Anatomy 0.000 description 1
- 230000005674 electromagnetic induction Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000005057 finger movement Effects 0.000 description 1
- 230000004886 head movement Effects 0.000 description 1
- 238000010438 heat treatment Methods 0.000 description 1
- WGCNASOHLSPBMP-UHFFFAOYSA-N hydroxyacetaldehyde Natural products OCC=O WGCNASOHLSPBMP-UHFFFAOYSA-N 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000012905 input function Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 230000004297 night vision Effects 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 239000002985 plastic film Substances 0.000 description 1
- 229920006255 plastic film Polymers 0.000 description 1
- 229920000515 polycarbonate Polymers 0.000 description 1
- 239000004417 polycarbonate Substances 0.000 description 1
- 229920000728 polyester Polymers 0.000 description 1
- 229920005644 polyethylene terephthalate glycol copolymer Polymers 0.000 description 1
- 229920001721 polyimide Polymers 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000035939 shock Effects 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
- 230000007723 transport mechanism Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
- G06F3/0354—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of 2D relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks
- G06F3/03547—Touch pads, in which fingers can move on a surface
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/0118—Head-up displays characterised by optical features comprising devices for improving the contrast of the display / brillance control visibility
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
- G02B2027/0178—Eyeglass type
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0179—Display position adjusting means not related to the information to be displayed
- G02B2027/0187—Display position adjusting means not related to the information to be displayed slaved to motion of at least a part of the body of the user, e.g. head, eye
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
- G06F2203/033—Indexing scheme relating to G06F3/033
- G06F2203/0339—Touch strips, e.g. orthogonal touch strips to control cursor movement or scrolling; single touch strip to adjust parameter or to implement a row of soft keys
Definitions
- a heads-up display can be incorporated into a pair of goggles, glasses, a headband, a helmet, or other such device that the user can wear.
- a heads-up display is typically positioned near the user's eyes and calibrated and/or aligned to the user's field of view to allow the user to review displayed information with little or no head movement.
- the display may also be transparent or translucent, allowing the user to view and interact with the surrounding environment while wearing the heads-up display. In some cases, the display may not be transparent, but may project a captured image of the environment on the display to simulate transparency or translucency.
- the display may be formed directly on a user's retina via a low-powered laser scanning technique.
- a computer processing system may be used to generate display information such as images and text for display.
- Such heads-up displays have a variety of applications, such as aviation information systems, vehicle navigation systems, and video games, among others.
- display information may include airspeed, altitude, a horizon line, heading, turn/bank, and slip/skid indicators. Display information may also include aviation data and symbols for use in approaches and landings.
- a flight guidance system in the heads-up display can provide visual cues based on navigation aids such as an Instrument Landing System or augmented Global Positioning System.
- navigation aids such as an Instrument Landing System or augmented Global Positioning System.
- display information may include speedometer, tachometer, and navigation system displays. Additionally, night vision information may be displayed to aid in night driving. Other automotive uses are possible as well.
- a pilot may need to interface with the information being displayed in the heads-up display, perhaps to switch between a parameter display interface (e.g., displaying airspeed, altitude, a horizon line, heading, etc.) and an approach/landing display interface.
- a parameter display interface e.g., displaying airspeed, altitude, a horizon line, heading, etc.
- a pilot may wish to interact with other pilots in the sky, and/or send information to one or more selected pilots.
- Current solutions, such as buttons, are limited in their functionality, and generally increase bulk, block light, and/or reduce peripheral vision due to their size, location, and composition requirements.
- the wearable heads-up display may include a processor, a display element configured to receive display information from the processor and to display the display information, a wearable frame structure supporting the display element and having a projection extending away from the display element, the projection securing the heads-up display to a user's body in a mariner that, when secured, places the display element within a user's field of view, and a finger-operable input device secured to the wearable frame structure and configured to sense at least one of a position and movement of a user's finger along a planar direction relative to a surface of the input device, and to provide corresponding input information to the processor.
- an improved method and device for interfacing with, and providing input to, the wearable heads-up display may be provided.
- the processor may transmit new display information to the display element. Further input could cause further updates to the display information or may cause the processor to execute other functions.
- the display information may include at least one possible input operation that may be made via the input device and one of a function and a selection associated with the input operation. For example, a list of four options may be provided on the display element via the display information, and an indication of an associated input operation to select or execute each one of the four options may be simultaneously provided.
- an upwards facing arrow may be displayed next to the first of the four options, indicating that an upwards swipe across the input device may select the first option, while a downwards facing arrow may be displayed next to the second of the four options, indicating that a downwards swipe across the input device may select the second option.
- Forwards and backwards facing arrows may be displayed on the display element with respect to the remaining two of the four options, and respective forwards and backwards swipes across the input device may select between the remaining two options.
- Other symbols could be used for other input actions, such as a single circle for a single tap, and a double circle (adjacent or within one another) for a double tap.
- Other types of swiping patterns e.g., gestures
- other types of functions are possible as well.
- the heads-up display may include two display elements coupled together via the wearable frame structure.
- the wearable frame structure may be a pair of glasses comprising two projections disposed on opposite sides of the two display elements (e.g., lenses), each projection extending away from each respective lens display element.
- the finger-operable input device may then be secured to the wearable frame structure via a first of the two projections.
- a second finger-operable input device may be secured to the wearable frame structure via a second of the two projections and configured to sense at least one of a position and movement of a finger along a planar direction relative to a surface of the second input device, and to provide corresponding input information to the processor.
- the dual-input structure allows for an input system where the first finger-operable input device is configured to provide fine-motor input information to the processor and the second finger-operable input device is configured to provide gross-motor input information to the processor.
- the stems and/or input devices may block or hinder the peripheral vision of the user and/or block light from reaching a user's eyes.
- the input device(s) may be translucent.
- the input device(s) may be substantially transparent.
- the input devices may be built upon a translucent or substantially transparent glass or plastic substrate.
- Metal conductor sensing or driving lines may be formed of a substantially transparent metal oxide, such as indium tin oxide (ITO).
- the input device may include a touch-sensitive pad configured to sense at least one of a position and movement of a finger along a planar direction relative to a surface of the input device via capacitive sensing, resistance sensing, and/or via a surface acoustic wave (SAW) process.
- the input device may be an optical imaging device and the surface of the optical imaging device a lens provided at an outer surface of the optical imaging device. By capturing images many times per second, the optical imaging device can accurately detect a position and movement of a finger along a planar direction relative to the lens surface of the input device.
- FIG. 1 shows an example embodiment of a wearable heads-up display with an integrated finger-tracking input sensor device
- FIG. 2 shows a block diagram of an example embodiment of a wearable heads-up display system
- FIG. 3 shows an example embodiment of a wearable heads-up display device including display elements
- FIG. 4 shows an example embodiment of various input interfaces for a wearable heads-up display device, including an integrated finger-tracking input sensor device;
- FIG. 5 illustrates an example method of interacting with a heads-up display device using the integrated finger-tracking input sensor device of FIG. 4 ;
- FIG. 6 illustrates another example method of interacting with a heads-up display device using the integrated finger-tracking input sensor device of FIG. 4 ;
- FIGS. 7 a and 7 h illustrate additional or alternative integrated finger-tracking input sensor devices
- FIG. 8 is a flow-chart illustrating an example method of providing input to a wearable heads-up display device via an integrated finger-tracking input sensor device.
- FIG. 9 is a functional block diagram of a computing device for supporting the wearable heads-up display device system of FIG. 2 ;
- FIG. 10 is a schematic illustrating a conceptual partial view of an example computer program product.
- the methods and systems disclosed herein generally relate to wireless directional identification and communication between wearable heads-up displays.
- wearable heads-up displays will be discussed, followed subsequently by discussions of their operation and input interaction.
- FIG. 1 shows an example embodiment of a wearable heads-up display device. While FIG. 1 illustrates glasses 102 as an example of a wearable heads-up display device, other types of wearable heads-up displays could additionally or alternatively be used
- glasses 102 comprise frame elements including lens-frames 104 , 106 , respective lens elements 108 and 110 , center frame support 112 , two extending side-arms 114 and 116 , and finger-operable touch pads 122 , 124 .
- the center support 112 and the two extending side-arms 114 and 116 are configured to secure the glasses 102 to a user's face via a user's nose and ears, respectively.
- Each of the frame elements 104 , 106 , 112 and side-arms 114 , 116 may be formed of a solid structure of plastic or metal, or may be formed of a hollow structure of similar material so as to allow wiring and component interconnects to be internally routed through the glasses 102 .
- Each of side-arms 114 and 116 is an example of a projection that extends away from a display element and is used to secure the display element to a user.
- the projection may be formed of a rigid or semi-rigid material such as plastic or metal, but may also be formed of other materials, including, for example, elastic or rubber.
- the projection may additionally or alternatively secure the display element to the user by extending around a rear portion of the user's head, or perhaps by connecting to a head-mounted helmet structure. Other possibilities exist as well.
- FIG. 2 shows an example embodiment of a wearable heads-up display system.
- a wearable heads-up display system 200 may include glasses 102 coupled to a computing device 202 via a connection 206 .
- the structure of computing device 202 will be described in more detail with respect to FIG. 9 .
- the computing device 202 may be incorporated into the glasses 102 themselves.
- the computing device 202 may be a head-mounted computing device incorporated into, for example, a hat or helmet, or may be a body-mounted computing device incorporated into, for example, a waist-mounted cell phone or personal digital assistant.
- the connection 206 may be a wired and/or wireless link.
- a wired link may include, for example, a parallel bus or a serial bus such as a Universal Serial Bus (USB).
- a wireless link may include, for example, Bluetooth, IEEE 802.11, Cellular (such as (SM, CDMA, UMTS, EV-DO, WiMAX, or LTE), or Zigbee, among other possibilities.
- the connection 206 may function to transmit data and/or commands to and/or from the glasses 102 , to transmit input received from finger-operable touch pads 122 , 124 , and/or to transmit display data for display on respective lenses 108 and/or 110 .
- FIG. 3 shows an example embodiment of a wearable heads-up display device including display elements.
- the lenses 108 and 110 may act as display elements.
- Glasses 102 may include a miniature projector 302 coupled to an inside-surface of side-arm 116 and configured to project a display onto an inside-surface of lens 108 .
- the projected display may include a current time 304 .
- a second projector 306 may be coupled to an inside-surface of side-arm 114 and configured to project a display onto an inside-surface of lens 110 .
- the projected display may include a current battery-level 308 associated with the system 200 .
- a signal strength indicator 310 may provide a user with an indication of a signal strength associated with a wireless connection (such as connection 208 with remote device 210 ). While indicator 310 is illustrated as providing a plurality of signal strength bars, other types of signal strength displays could be used, such as a numeric text, a line-graph, etc.
- the lenses 108 and 110 act as a combiner in a light projection system, and may include a coating that reflects the light projected onto them from the projectors 302 , 306 . In some embodiments, a special coating may not be required (e.g., when the projectors 302 , 306 are scanning laser devices).
- the lens 108 , 110 themselves may include a transparent or semi-transparent matrix display such as an electroluminescent (EL) display or liquid crystal display (LCD).
- EL electroluminescent
- LCD liquid crystal display
- a corresponding display driver may be disposed within the lens frames 104 , 106 for driving such a matrix display.
- a laser or LED source and scanning system could be used to draw a raster display directly onto the retina of one or more of the user's eyes. Other possibilities exist as well.
- connection 208 may be a wired and/or wireless link having one or more characteristics described above in relation to connection 206 .
- the remote device 210 may be a device associated with the wearable heads-up display system 200 (or a user thereof, not shown), and may be a cell phone, a personal data assistant (PDA), or some other device.
- the remote device 210 may be a cell phone having Bluetooth capabilities, and may provide information for display on respective lenses 108 and/or 110 , or may provide a target for transmission of data or instructions responsive to input operations received via finger-operable touch pads 122 , 124 .
- connection 208 may comprise one or more base stations, routers, switches, LANs, WLANs, WANs, access points, or other network infrastructure.
- remote device 210 may be accessible via the Internet, and may comprise a computing cluster associated with a particular web service (e.g., social-networking, photo sharing, address book, etc.).
- FIG. 4 shows an example embodiment of various input interfaces for glasses 102 that allows a user to interact with the glasses 102 and computing device 202 .
- the input interfaces may comprise one or more of finger-operable touch pad 122 , a movement sensor 402 , and a microphone 404 , among other possible input elements. While FIG. 4 illustrates a side-view of side-arm 116 , additional and similar input interfaces may be provided on side-arm 114 . For example, and as illustrated in FIG. 3 , an additional finger-operable touch pad 124 may be provided on side-arm 114 .
- the finger-operable touch pad 122 may sense at least one of a position and movement of a finger along a planar direction relative to a surface of the touch pad 122 (e.g., parallel to the surface of FIG. 4 ) via capacitive sensing, resistance sensing, and/or via a surface acoustic wave (SAW) process, among other possibilities.
- the finger-operable touch pad 122 may be capable of sensing movement of a finger in a direction normal to the surface of the touch pad 122 (e.g., into the surface of FIG. 4 ), including perhaps sensing a level of pressure applied to the touch pad 122 .
- a capacitive touch pad In a capacitive touch pad, one or more insulating layers are coated with one or more conducting layers, and a driving signal applied to at least one of the one or more conducting layers.
- a user's body acts as a conductor
- touching the pad with one's finger causes a distortion in at least one of the conducting layer's electrostatic field, measurable as a change in capacitance.
- Different capacitive technologies may be used to determine the location of the touch. For example, in a surface capacitance method, only oneside of an insulating layer is coated with a conductive layer. A small voltage is then applied to the conductive layer, resulting in an electrostatic field.
- a capacitor When a user's finger touches the touch pad surface, a capacitor is dynamically formed, and a controller can determine the location of the touch indirectly from the change in capacitance.
- vertically and horizontally-arranged driving lines e.g., two conductive layers
- Bringing a finger close to the surface of the array changes the local electrostatic field around an intersection of the separated driving lines, changing the mutual capacitance between driving lines at corresponding intersecting areas.
- mutual capacitance can be used to determine touch locations at a plurality of locations (e.g., multi-touch).
- a resistive touch pad In a resistive touch pad, two electrically conductive layers having horizontal and vertical lines are formed separated by an insulating gap (e.g., glass, plastic, air, etc.), and a voltage gradient is applied to the first conductive layer.
- an insulating gap e.g., glass, plastic, air, etc.
- the two conductive layers When contact is made with the surface of the touch pad, the two conductive layers are pressed together, and the second sheet measures the voltage as distance along the first sheet, providing an X coordinate. After the X contact coordinate has been acquired, a second voltage gradient is applied to the second sheet to ascertain the Y coordinate.
- conductive layers are not disposed throughout the pad itself. Rather, transmitting and receiving transducers and reflectors are disposed at edges of the track pad. Waves emitted by the transmitting transducers are reflected across the touch pad in the X and Y directions and to receiving transducers via the reflectors. When a finger touches the screen, portions of the waves are absorbed, causing a touch event and its corresponding location to be detected by control circuitry.
- a width of the side-arm 116 may be formed thicker in a region in which the touch pad 122 is formed, and thinner in a region in which the touch pad 122 is not formed, so as to accommodate sufficient space to detect finger movements in all planar directions (e.g., 360 c ), or at the very least, two pairs of diametrically opposed directions such as up, down, forward, and back.
- the side-arm 116 and/or the touch pad 122 may be formed of a translucent or substantially transparent material.
- the side-arm 116 may be formed of a translucent or substantially transparent plastic material such as Acrylic (polymethlmethacrylate), Butyrate (cellulose acetate butyrate), Lexan polycarbonate), and PETG (glycol modified polyethylene terphthalate). Other types of plastics could also be used. Translucent or substantially transparent materials other than plastic could also be used.
- the touch pad 122 may be formed of one or more translucent or transparent insulating (e.g., glass or plastic) layers and one or more translucent or transparent conducting (e.g., metal) layers.
- the glass may be tempered or toughened glass manufactured through a process of extreme heating and rapid cooling.
- the plastic may be a polyimide, polyethylene, or polyester based plastic film. Other types of translucent and/or substantially transparent glasses and plastics could also be used.
- the conducting layer may be formed of a metal oxide, such as Indium Tin Oxide (ITO). Other types of insulating and conducting layers could also be used.
- ITO Indium Tin Oxide
- Edges of the touch pad 122 may be formed to have a raised, indented, or roughened surface, so as to provide tactile feedback to a user when the user's finger reaches the edge of the touch pad 122 .
- Such a structure may also allow a user (who has limited or no visual cues as to the location of the touch pad 122 ) to locate the touch pad 122 on the side-arm 116 quickly, similar to the way in which physical indentions normally provided on the “F” and “J” keys of a keyboard allow a typist to quickly position their fingers correctly on the keyboard.
- the raised indented, and/or roughened surface could alternatively or additionally be formed in the side-arm 116 just past the edge of the touch pad 122 .
- a similar roughened, raised, or indented element may be provided at substantially a center of the touch pad 122 to provide additional tactile cues to a user.
- the movement sensor 402 may be provided on or in a frame element of the glasses 102 , and may act as an input device configured to track a user's movements.
- the movement sensor 402 may include one or more of an accelerometer, a magnetometer, or a gyroscope, among other options.
- An accelerometer is a device that measures acceleration. Single- and multi-axis models can detect magnitude and direction of the acceleration as a vector quantity, and can be used to sense orientation, acceleration, vibration shock, and falling.
- a gyroscope is a device for measuring or maintaining orientation, based on the principles of conservation of angular momentum.
- gyroscope a microelectromechanical system (MEMS) based gyroscope
- MEMS microelectromechanical system
- Other types of gyroscopes could be used as well.
- a magnetometer is a device used to measure the strength and/or direction of the magnetic field in the vicinity of the device, and can be used to determine a direction in which a person or device is facing.
- Other types of movement sensors could additionally, or alternatively, be used.
- the movement sensor 402 may be used, for example, to determine when, how much, and perhaps how quickly, a user wearing the glasses 102 turns or moves his or her head or body to the right, left, tilted up, or tilted down.
- the sensor 402 may also be able to determine a cardinal direction in which the user is facing.
- Microphone 404 may be any acoustic-to-electric transducer or sensor that converts sound into an electrical signal.
- microphone 404 may use electromagnetic induction, capacitance change, piezoelectric generation, or light modulation, among other techniques, to produce an electrical voltage signal from mechanical vibration.
- the microphone 404 may communicate with a speech recognition program at computing device 202 to allow a user to speak voice commands that cause the computing device 202 to take particular action(s).
- the microphone 404 may also be used for other purposes.
- touch-sensitive pad 122 While one touch-sensitive pad 122 , one movement sensor 402 , and one microphone 404 is illustrated in FIG. 4 , in some embodiments a subset of these devices may be provided. In at least one embodiment, a plurality of touch-sensitive pads may be disposed on the side-arm 116 and/or the side-arm 114 . In another embodiment, an array of (same or different) microphones or array of (same or different) movement sensors may be provided on the side-arm 116 and/or the side-arm 114 . Additionally, the touch pad 122 may be provided having a different shape or dimensions than that shown in FIG. 4 .
- the input interface may be wiredly or wirelessly coupled to the computing device 202 (perhaps via connection 206 ) to allow a user to control settings and features of the wearable heads-up display system 200 , to initiate communications with other wearable heads-up displays, to provide positioning and/or movement information from sensor 402 , and/or to control and interact with displays on the lens 108 , 110 .
- FIGS. 5 and 6 illustrate example input interactions in which one or more of the touch pads 122 , 124 may be used to interact with the glasses 102 .
- FIG. 5 illustrates a display of input commands and their associated functions relative to a display of text (such as an e-mail), and
- FIG. 6 illustrates a display of input commands and their associated functions relative to interacting with real-world objects.
- these figures are exemplary in nature only, and many other applications and combinations of input commands and associated functions are possible in light of this disclosure.
- FIG. 5 illustrates one embodiment in which interactions with displayed text (such as e-mails in an e-mail inbox) may be accomplished via one or more of the touch pads 122 , 124 .
- either one of touch pad 122 or touch pad 124 may be operated, and the same input function performed by the glasses 102 in response to the input.
- One advantage of providing touch pads 122 , 124 on respective sides of glasses 102 is that the same glasses 102 may be used naturally by both left-handed and right-handed persons.
- FIG. 5 illustrates a simultaneous display of different content on both lenses 108 and 110
- simultaneous displays of content in overlapping regions of the bi-focal human vision field could confuse and/or disorient a user as a result of the brain's attempts to combine and make sense of the two differing displays. Accordingly, steps may be taken to prevent such confusion and/or disorientation.
- both lenses 108 , 110 may be populated with overlapping content, and a user may be expected or instructed to close one eye to focus on one lens 108 , 110 at a time.
- only one of the lenses 108 , 110 is configured to contain the information illustrated in FIG. 5 at any one time.
- both lenses 108 , 110 may be populated, but with corresponding overlapping portions of one of the displays 108 , 110 eliminated, faded, blurred, or otherwise augmented to prevent visual confusion and/or disorientation at intersecting regions of the displayed content. Other possibilities exist as well.
- a content display 500 projected on lens 110 by projecting device 306 may include text of an e-mail, perhaps retrieved from an e-mail inbox associated with a user of the glasses 102 and stored at remote device 210 .
- the e-mail text may represent just one e-mail out of a plurality of available e-mails.
- a scroll bar 502 may provide an indication of a relative position of the displayed text relative to a remainder of the e-mail from which the text is retrieved.
- a help display 504 intended to inform the user of available input commands that may be executed on one or more of touch pads 122 and 124 , and of their associated functions, may be displayed on lens 108 via projecting device 302 .
- the content display 500 may be removed from lens 110 , corresponding overlapping portions of the content display 500 augmented, or a user expected or instructed to close a corresponding eye looking through lens 110 .
- the help display 504 includes a Scroll Up command 506 , a Scroll Down command 508 , a Next E-mail command 510 , and a Previous E-mail command 512 .
- Each of the commands 506 - 512 is displayed with a corresponding symbol illustrating an input operation that may be executed on one of the touch pads 122 and/or 124 to execute the associated function.
- the Scroll Down command 508 may be executed by a user swiping their finger across touch pad 124 in a downwards direction (as indicated by the downward facing arrow symbol).
- the computing device 202 may cause the content display 500 to be updated with additional text not previously displayed, and may cause the scroll bar 502 to advance downwards in accordance with a length of the e-mail from which the text is pulled.
- the Scroll Up command 506 (as indicated by the upward facing arrow symbol) may cause the content display 500 to be updated with previously displayed text, and may cause the scroll bar 502 to retract upwards in accordance with a length of the e-mail from which the text is pulled.
- the Next E-mail command 510 may cause the content display 500 to be updated with text corresponding to a next e-mail (e.g., amore recently received or less recently received e-mail) from the e-mail inbox associated with the user.
- the Previous E-mail command 512 may cause the content display 500 to be updated with text corresponding to a previous e-mail that was, perhaps, previously shown in the content display 500 .
- Other commands and other types of text, symbols, or images could be used as well.
- the commands and their associated functions may be modified by a user.
- a forward-swipe across touch pad 124 is associated with loading a next e-mail, it may be more intuitive for some users that a backward-swipe across touch pad 124 loads the next e-mail.
- Associations between commands and associated functions may, for example, be stored in a list or database in computing device 202 and/or at remote device 210 .
- a locally or remotely accessible interface may allow the user to access the stored list or database and modify the associations.
- the user may be able to access the computing device 202 and/or the remote device 210 via a wired or wireless connection, and modify the Next E-mail command to be associated with a backward-swipe across touch pad 124 instead of a forward-swipe.
- the computing device 202 (or remote device 210 ) may, in response to recognizing that the newly assigned backward-swipe was previously assigned to another function (the previous e-mail function), request the user to associate a new command with the previous e-mail function.
- the user may record a forward-swipe across touch pad 124 as being associated with the previous e-mail function.
- the selection of a new command may be executed by a user selecting the new command from a list, or may be executed by the user entering the new command via the touch pad 124 when requested (e.g., swiping forward across the touch pad 124 when requested).
- unique patterns of input commands can be created by a user, and personalized per user, by recording a finger-motion pattern detected at the touch pad 124 when requested.
- the list or database stored at computing device 202 may be customized per application, such that, for example, the change to the Next E-mail command is applied to the e-mail function alone, and not to other functions, such as web browsing (e.g., a Next Link or Next Page command).
- the list or database may store global commands that, once changed, are applied to other related, or to all, functions executed at computing device 202 . Other possibilities exist as well.
- the help display 504 may be displayed in lens 108 and/or 110 when the e-mail application is first started, so as to remind the user of the available input commands executable via, touch pad 122 and/or 124 to navigate the e-mail application. After displaying the help display 504 for some period of time (e.g., 1-5 seconds), the help display 504 may be replaced with the content display 500 (in lens 108 and/or 110 ).
- the help display 504 may be displayed only upon demand (perhaps via, a particular motion across touch pad 122 or 124 associated with displaying help display 504 , a particular area of touch pad 122 or 124 associated with displaying help display 504 , or an algorithm executing at computing device 202 that detects that a user is having difficulty navigating via touch pad 122 or 124 ).
- Computing device 202 may detect that a user is having difficulty navigating based on a number of unrecognized movements across touch pad 122 or 124 , a number of times that a user “undoes” a previous command by subsequently doing the opposite (e.g., inputting the Next E-mail command once, and then the Previous E-mail command twice), or by some other combination of input and logic.
- FIG. 6 illustrates an embodiment in which interactions with real world objects may be accomplished via one or more of the touch pads 122 , 124 .
- each touch pad 122 , 124 may be operated independently, and may provide different corresponding functions.
- one of the touch pads 122 , 124 may be associated with a user's dominant hand, and the other may be associated with a user's non-dominant hand, For example, assuming a right-handed user is wearing the glasses of FIG. 6 , touch pad 124 would be associated with the user's dominant hand, while touch pad 122 would be associated with the user's non-dominant hand.
- Different functions may be assigned to similar input operations executed at a respective touch pad 122 , 124 based on this distinction.
- real-world objects 600 and 602 are viewable through translucent and/or transparent lenses 108 , 110 .
- real-world object 600 is illustrated as a soccer ball, while real-world object 602 is illustrated as a chair. While objects 600 and 602 are shown twice (in lens 108 and 110 ), it should be understood that there is really only one of each object 600 and 602 .
- the doubling of objects in FIG. 6 is meant to reflect the real-world binaural vision characteristics of the human-user (e.g., viewing the objects 600 , 602 from two slightly different offset angles).
- Selection indicator 604 is a super-imposed selection indicator projected by one or more of projecting devices 302 , 306 on respective lenses 108 and/or 110 .
- a camera (not shown) disposed on glasses 102 may be configured to capture the user's field-of-view, and recognize particular objects for selection, such as the soccer ball 600 and/or chair 602 .
- Help displays 606 and/or 608 may be projected on lenses 108 , 110 to provide a user with options for interacting with the real-world Objects 600 , 602 .
- Help displays 606 and 608 may be displayed in a similar manner, and subject to similar restraints, to those of displays 500 and 504 .
- help displays 606 and 608 may be displayed simultaneously, may be displayed only one at a time, or may be displayed such that one of the help displays 606 , 608 is augmented in corresponding overlapping areas of help displays 606 and 608 .
- Help display 606 provides, for example, functions and associated commands for selecting an object recognized by computing system 202 (e.g., via the camera).
- the selection indicator 604 may be displayed over a randomly selected object out of a plurality of objects recognized by the computing system 202 (or, perhaps, displayed over an object the that the glasses 102 detect that the user is most interested in).
- the soccer ball 600 may be initially selected.
- the selection indicator 604 may be displayed in lens 108 and/or lens 110 via respective projecting devices 302 , 304 .
- the Select This Object command 610 of help display 606 may be executed by double-tapping the touch pad 124 with a single finger (illustrated with a symbol comprising a dot within an outer circle).
- Selecting the currently highlighted object in such a manner may allow for further functions to be executed with respect to the selected object. For example, once an object is selected, the selected object may be used as a focus-point for taking a picture via an imaging device (not shown) integrated with the glasses 102 . Additionally or alternatively, an image or information search may be conducted using an image of the selected object. For example, an image of the soccer ball 600 may be used to locate other images of soccer balls via an online search function, to conduct an online product search function to find soccer balls available for purchase, or to obtain information regarding the sport of soccer or soccer balls in general (history, structure, etc.).
- the Choose Another Object command 612 of help display 606 may be executed by a single-tap on the touch pad 124 with a single finger (illustrated with a symbol comprising an empty circle).
- the Choose Another Object command 612 may cycle through each of the plurality of recognized objects within the current field of view. For example, single-tapping touch pad 124 may cause the selection indicator 604 to move from the soccer ball 600 to the chair 602 (and may simultaneously cause the selection indicator 604 to change its shape to accommodate the size and/or geometries of the chair 602 ).
- the Select This Object command 610 may be executed via a double-tap using a single finger on touch pad 124 to find information and/or images with respect to the chair 602 . Other possibilities exist as well.
- the help display 606 may be displayed in lens 108 and/or 110 when the object selection application is first stated, so as to remind the user of the available input commands, executable via touch pad 122 or 124 , to navigate the object selection application. After displaying the help display 606 for some period of time (e.g., 1-5 seconds), the help display 606 may be removed.
- the help display 606 may be displayed only upon demand (perhaps via a particular motion across touch pad 122 or 124 associated with displaying help display 606 , a particular area of touch pad 122 or 124 associated with displaying help display 606 , or an algorithm executing at computing device 202 that detects that a user is having difficulty navigating via touch pad 122 or 124 ).
- Help display 608 provides, for example, functions and associated commands for capturing an image of a scene as viewed through lenses 108 , 110 , and as imaged by the integrated camera (not shown).
- the selection indicator 604 may provide a focus point for an image capture process via commands 614 and 616 .
- the Capture Image command 614 of help display 608 may be executed by a two-finger single-tap on touch pad 124 (illustrated with a symbol comprising two adjacent empty circles), and may cause the camera to capture an image without a flash, using the currently-selected object 600 as the focal point.
- the Capture With Flash command 616 of help display 606 may be executed by a two-finger double-tap on touch pad 124 (illustrated with a symbol comprising two adjacent dotes within respective outer circles), and may cause the camera to capture an image with a flash, using the currently-selected object 600 as the focal point. Similar to the disclosure above relative to help display 504 , the input commands associated with the functions 614 and 616 may be modified by a user, and stored in computing device 202 . Additionally, and similar to the disclosure above relative to help display 606 , help display 608 may only be displayed as necessary, and otherwise, may be removed from lens 110 .
- touch pads 122 and 124 may be used to provide separate, independent input to the glasses 102 .
- touch pad 122 may provide gross motor movement of the selection indicator 604 for image capture focusing purposes
- touch pad 124 may provide fine motor movement of the selection indicator 604 (for the same or different purpose).
- the touch pad 122 may allow a user to move the selection indicator 604 quickly to the top of the user's field of view via a relatively short upwards-swipe across the touch pad 122 (e.g., a full swipe across touch pad 122 in the vertical direction may cause a greater than 50% movement of the selection indicator 604 across a user's field of view).
- the touch pad 124 may allow the user to move the selection indicator 604 in small increments to fine tune the focus selection (e.g., a full swipe across touch pad 124 in the vertical direction may cause a less than 10% movement of the selection indicator 604 across a user's field of view).
- Other applications of using gross and fine motor input between touch pads 122 and 124 could also be implemented.
- FIG. 7 a illustrates an additional or alternative embodiment for interacting with glasses 102 .
- an image capture device 702 and a light source 704 may be coupled to side-arm 116 (in addition to, or in place of, touch pad 122 ).
- the light source 704 and image capture device 702 are illustrated as separate devices, they may be combined into a single device.
- the light source 704 and image capture device 702 work together to illuminate a surface of a user's finger, and to detect the unique, time patterns in the surface of the user's finger.
- the light source 704 may be, for example, a laser or LED that emits light in the visible (e.g., red) or invisible (e.g., infra-red) spectrum.
- the image capture device 702 may be a complementary metal-oxide-semiconductor (CMOS) or charge-coupled device (CCD) image capture device capable of detecting wavelengths in the range emitted by the light source 704 . Interfacing with glasses 102 via image capture device 702 and light source 704 may be accomplished in the same manner as set forth above with respect to FIG. 1-6 .
- FIG. 7 b illustrates another additional or alternative embodiment for interacting with glasses 102 .
- a touch pad 706 may be coupled to side-arm 116 and extend beyond the edges of the side-arm 116 . While this arrangement provides for additional gesturing space and allows a user to create more advanced input patterns, it also blocks more light from a user's field of view, and blocks a user's peripheral vision to a greater extent than the integrated touch pad 122 of FIG. 4 . Thus, in this scenario, the level of translucency and/or transparency of the touch pad 706 may become more important. Additionally, and advantageously, the touch pad 706 in this arrangement may be removable from the side-arm 116 , and may be attached only when needed by a heads-up display user.
- Removable fasteners may include, among others, Velcro, hook and tabs, buttons, snaps, friction fittings, screws, strike and latch fittings, compression fittings, rivets, and grommets. Permanent fasteners could additionally or alternatively be used.
- An electrical connection to the touch pad 706 may be provided via a connector on the outer-surface of the side-arm 116 , and communication between the touch pad 706 and computing device 202 may take place via a wired or wireless connection. Interfacing with glasses 102 via touch pad 706 may be accomplished in the same manner as set forth above with respect to FIG. 1-6 .
- help displays 500 , 504 , 606 , and 608 may also include commands and corresponding symbols relative to movements detectable by movement sensor 408 and/or commands and symbols relative to voice commands detectable by microphone 410 .
- FIG. 8 is a flow-chart illustrating an example method 800 of interfacing with a heads-up display, such as glasses 102 .
- the method 800 includes a first display step 802 , an input step 804 , and a second display step 806 .
- display information is provided to at least one display element of a wearable heads-up display.
- This display information may include one or more supported functions relative to a currently-executing application, and may include, for each function, an associated input command (illustrated via a symbol) that may be executed at an input device to cause the corresponding function to be executed or corresponding selection to be selected.
- the associated input commands may be loaded from a list or database stored at computing device 202 and/or at remote device 210 , and may vary depending upon a determination of the current application being executed by computing device 202 .
- step 804 input information is received from a coupled finger-operable input device regarding a position or movement of a finger along a planar direction relative to a surface of the input device. This input information may be recognized as equal or equivalent to one of the associated input commands included in the display information at step 802 .
- step 806 new display information is provided to at least one display element (and perhaps the same at least one display element as in step 802 ) responsive to receiving the input information.
- FIG. 9 is a functional block diagram of a computing device 202 for supporting the wearable heads-up displays set forth above arranged in accordance with at least some embodiments described herein.
- the computing device 202 may be a personal computer, mobile device, cellular phone, video game system, global positioning system, or other electronic system.
- computing device 202 may typically include one or more processors or controllers (processor) 910 and system memory 920 .
- a memory bus 930 can be used for communicating between the processor 910 and the system memory 920 .
- processor 910 can be of any type including, but not limited to, a microprocessor ( ⁇ P), a microcontroller ( ⁇ C), a digital signal processor (DSP), or any combination thereof.
- a memory controller 915 can also be used with the processor 910 , or in some implementations, the memory controller 915 can be an internal part of the processor 910 .
- system memory 920 can be of any type including but not limited to volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory, etc.) or any combination thereof.
- System memory 920 typically includes one or more applications 922 and program data 924 .
- Application 922 may include algorithms such as input/output device interface algorithms 923 arranged to control and interface with input devices such as finger-operable touch pads, in accordance with the present disclosure
- Other process descriptions, steps, or blocks in flow or message diagrams in the present disclosure should be understood as potentially representing modules, segments, or portions of code which include one or more executable instructions stored in application memory 922 for implementing specific logical functions or steps in the process, and alternate implementations are included within the scope of the preferred embodiments of the methods in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art.
- Program data 924 may include, among other things, display symbols 925 that correspond to commands that may be executed via corresponding finger-operable touch pad operations (or other input interfaces), and that may be included in display data sent to one or more display devices 992 .
- applications stored in application memory 922 can be arranged to operate with program data 924 .
- Computing device 202 can have additional features or functionality, and additional interfaces to facilitate communications between the basic configuration 901 and any devices and interfaces.
- the data storage devices 950 can be removable storage devices 951 , non-removable storage devices 952 , or a combination thereof.
- removable storage and non-removable storage devices include magnetic disk devices such as flexible disk drives and hard-disk drives (HDD), optical disk drives such as compact disk (CD) drives or digital versatile disk (DVD) drives, solid state drives (SSD), and tape drives to name a few.
- HDD hard-disk drives
- CD compact disk
- DVD digital versatile disk
- SSD solid state drives
- Computer storage media can include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, or other data.
- Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory, or other memory technology, CD-ROM, digital versatile disks (DVD), or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage, or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by computing device 202 .
- Computing device 202 can also include output interfaces 960 that may include a graphics processing unit 961 , which can be configured to communicate to various external devices such as display devices 992 (which may include, for example, projecting devices 302 , 306 and/or lenses 108 , 110 ) or speakers via one or more A/V ports 963 .
- External communication circuits 980 may include a network controller 981 , which can be arranged to facilitate communications with one or more other computing devices 990 and/or one or more transmitting and/or receiving devices 991 .
- the communication connection is one example of a communication media.
- Communication media may typically be embodied by computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and includes any information delivery media.
- a “modulated data signal” can be a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal.
- communication media can include wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, radio frequency (RF), infrared (IR) and other wireless media.
- RF radio frequency
- IR infrared
- the term computer readable media as used herein can include both storage media and communication media.
- tangible computer readable media may refer to storage media alone.
- Computing device 202 can be implemented as a portion of a small-form factor portable (or mobile) electronic device such as a cell phone, a multi-chip module (MCM), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a PDA, a personal media player device, a wireless web-watch device, a personal headset device, an application specific device, or a hybrid device that include any of the above functions.
- a small-form factor portable (or mobile) electronic device such as a cell phone, a multi-chip module (MCM), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a PDA, a personal media player device, a wireless web-watch device, a personal headset device, an application specific device, or a hybrid device that include any of the above functions.
- Computing device 202 can also be implemented as a personal computer including both laptop computer and non-laptop computer configurations.
- FIG. 10 is a schematic illustrating a conceptual partial view of an example computer program product 1000 that includes a computer program for executing a computer process on a computing device, arranged according to at least some embodiments presented herein.
- the example computer program product 1000 is provided using a signal bearing medium 1001 .
- the signal bearing medium 1001 may include one or more programming instructions 1002 that, when executed by one or more processors, may provide functionality or portions of the functionality described above with respect to FIGS. 1-8 .
- one or more features of method 800 may be undertaken by one or more instructions associated with the signal bearing medium 1001 .
- the signal bearing medium 1001 may encompass a tangible computer-readable medium 1003 , such as, but not limited to, a hard disk drive, a Compact Disc (CD), a Digital Video Disk (DVD), a digital tape, memory, etc.
- the signal bearing medium 1001 may encompass a computer recordable medium 1004 , such as, but not limited to, memory, read/write (R/W) CDs, R/W DVDs, etc.
- the signal bearing medium 1001 may encompass a communications medium 1005 , such as, but not limited to, a digital and/or an analog communication medium (e.g., a fiber optic cable, a waveguide, a wired communications link, a wireless communication link, etc).
- the signal bearing medium 1001 may be conveyed by a wireless form of the communications medium 1005 (e.g., a wireless communications medium conforming with the IEEE 802.11 standard or other transmission protocol).
- the one or more programming instructions 1002 may be, for example, computer executable and/or logic implemented instructions.
- a computing device such as the computing device 202 of FIG. 9 may be configured to provide various operations, functions, or actions in response to the programming instructions 1002 conveyed to the computing device 202 by one or more of the computer readable medium 1003 , the computer recordable medium 1004 , and/or the communications medium 1005 .
Abstract
Disclosed are systems, methods, and devices for interfacing with a wearable heads-up display via a finger-operable input device. The wearable heads-up display may include a display element for receiving and displaying display information received from a processor, and may also include a wearable frame structure supporting the display element and having a projection extending away from the display element. The projection may be configured to secure the heads-up display to a user's body in a manner such that the display element is disposed within a field of view of the user. A finger-operable input device secured to the wearable frame structure is configured to sense at least one of a position and movement of a finger along a planar direction relative to a surface of the input device, and to provide corresponding input information to the processor.
Description
- The present application claims priority to U.S. patent application Ser. No. 13/115,430, filed on May 25, 2011, the entire contents of which are herein incorporated by reference.
- Unless otherwise indicated herein, the materials described in this section are not prior art to the claims in this application and are not admitted to be prior art by inclusion in this section.
- Various technologies can be utilized to display information to a user of a system. Some systems for displaying information may utilize “heads-up” displays. A heads-up display can be incorporated into a pair of goggles, glasses, a headband, a helmet, or other such device that the user can wear. A heads-up display is typically positioned near the user's eyes and calibrated and/or aligned to the user's field of view to allow the user to review displayed information with little or no head movement. The display may also be transparent or translucent, allowing the user to view and interact with the surrounding environment while wearing the heads-up display. In some cases, the display may not be transparent, but may project a captured image of the environment on the display to simulate transparency or translucency. In other cases, the display may be formed directly on a user's retina via a low-powered laser scanning technique. To generate display information such as images and text for display, a computer processing system may be used. Such heads-up displays have a variety of applications, such as aviation information systems, vehicle navigation systems, and video games, among others.
- For example, in aviation information systems, display information may include airspeed, altitude, a horizon line, heading, turn/bank, and slip/skid indicators. Display information may also include aviation data and symbols for use in approaches and landings. For example, a flight guidance system in the heads-up display can provide visual cues based on navigation aids such as an Instrument Landing System or augmented Global Positioning System. Other aviation uses are possible as well. In the automotive field, display information may include speedometer, tachometer, and navigation system displays. Additionally, night vision information may be displayed to aid in night driving. Other automotive uses are possible as well.
- As more and more applications and uses are being developed for head-mounted display devices, more complex interactions have arisen. For example, in the aviation context, a pilot may need to interface with the information being displayed in the heads-up display, perhaps to switch between a parameter display interface (e.g., displaying airspeed, altitude, a horizon line, heading, etc.) and an approach/landing display interface.
- Other, more complex interactions may also be necessary. For example, a pilot may wish to interact with other pilots in the sky, and/or send information to one or more selected pilots. Current solutions, such as buttons, are limited in their functionality, and generally increase bulk, block light, and/or reduce peripheral vision due to their size, location, and composition requirements.
- Disclosed herein are improved methods and devices for controlling and interfacing with a wearable heads-up display. In an exemplary embodiment, the wearable heads-up display may include a processor, a display element configured to receive display information from the processor and to display the display information, a wearable frame structure supporting the display element and having a projection extending away from the display element, the projection securing the heads-up display to a user's body in a mariner that, when secured, places the display element within a user's field of view, and a finger-operable input device secured to the wearable frame structure and configured to sense at least one of a position and movement of a user's finger along a planar direction relative to a surface of the input device, and to provide corresponding input information to the processor. In this manner, an improved method and device for interfacing with, and providing input to, the wearable heads-up display may be provided. For example, in response to receiving input at the processor from the finger-operable input device, the processor may transmit new display information to the display element. Further input could cause further updates to the display information or may cause the processor to execute other functions.
- In another exemplary embodiment, the display information may include at least one possible input operation that may be made via the input device and one of a function and a selection associated with the input operation. For example, a list of four options may be provided on the display element via the display information, and an indication of an associated input operation to select or execute each one of the four options may be simultaneously provided. In one example, an upwards facing arrow may be displayed next to the first of the four options, indicating that an upwards swipe across the input device may select the first option, while a downwards facing arrow may be displayed next to the second of the four options, indicating that a downwards swipe across the input device may select the second option. Forwards and backwards facing arrows may be displayed on the display element with respect to the remaining two of the four options, and respective forwards and backwards swipes across the input device may select between the remaining two options. Other symbols could be used for other input actions, such as a single circle for a single tap, and a double circle (adjacent or within one another) for a double tap. Other types of swiping patterns (e.g., gestures) and other types of functions are possible as well.
- In one embodiment, the heads-up display may include two display elements coupled together via the wearable frame structure. In one example, the wearable frame structure may be a pair of glasses comprising two projections disposed on opposite sides of the two display elements (e.g., lenses), each projection extending away from each respective lens display element. The finger-operable input device may then be secured to the wearable frame structure via a first of the two projections. Furthermore, a second finger-operable input device may be secured to the wearable frame structure via a second of the two projections and configured to sense at least one of a position and movement of a finger along a planar direction relative to a surface of the second input device, and to provide corresponding input information to the processor. Such a configuration allows the same wearable heads-up display to be operated by both right-handed and left-handed users. Furthermore, the dual-input structure allows for an input system where the first finger-operable input device is configured to provide fine-motor input information to the processor and the second finger-operable input device is configured to provide gross-motor input information to the processor.
- Due to the location of the finger-operable input device(s) on the stern of the glasses, the stems and/or input devices may block or hinder the peripheral vision of the user and/or block light from reaching a user's eyes. In order to prevent such an occurrence, and in one embodiment, the input device(s) may be translucent. In another example, the input device(s) may be substantially transparent. For example, the input devices may be built upon a translucent or substantially transparent glass or plastic substrate. Metal conductor sensing or driving lines may be formed of a substantially transparent metal oxide, such as indium tin oxide (ITO).
- In another embodiment, the input device may include a touch-sensitive pad configured to sense at least one of a position and movement of a finger along a planar direction relative to a surface of the input device via capacitive sensing, resistance sensing, and/or via a surface acoustic wave (SAW) process. In yet another embodiment, the input device may be an optical imaging device and the surface of the optical imaging device a lens provided at an outer surface of the optical imaging device. By capturing images many times per second, the optical imaging device can accurately detect a position and movement of a finger along a planar direction relative to the lens surface of the input device.
- The foregoing summary is illustrative only and is not intended to be in any way limiting. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features will become apparent by reference to the figures and the following detailed description.
- In the figures:
-
FIG. 1 shows an example embodiment of a wearable heads-up display with an integrated finger-tracking input sensor device; -
FIG. 2 shows a block diagram of an example embodiment of a wearable heads-up display system; -
FIG. 3 shows an example embodiment of a wearable heads-up display device including display elements; -
FIG. 4 shows an example embodiment of various input interfaces for a wearable heads-up display device, including an integrated finger-tracking input sensor device; -
FIG. 5 illustrates an example method of interacting with a heads-up display device using the integrated finger-tracking input sensor device ofFIG. 4 ; -
FIG. 6 illustrates another example method of interacting with a heads-up display device using the integrated finger-tracking input sensor device ofFIG. 4 ; -
FIGS. 7 a and 7 h illustrate additional or alternative integrated finger-tracking input sensor devices; -
FIG. 8 is a flow-chart illustrating an example method of providing input to a wearable heads-up display device via an integrated finger-tracking input sensor device; and -
FIG. 9 is a functional block diagram of a computing device for supporting the wearable heads-up display device system ofFIG. 2 ; and -
FIG. 10 is a schematic illustrating a conceptual partial view of an example computer program product. - in the following detailed description, reference is made to the accompanying figures, which form a part hereof. In the figures, similar symbols typically identify similar components, unless context dictates otherwise. The illustrative embodiments described in the detailed description, figures, and claims are not meant to be limiting. Other embodiments may be utilized, and other changes may be made, without departing from the spirit or scope of the subject matter presented herein. It will be readily understood that the aspects of the present disclosure, as generally described herein, and illustrated in the figures, can be arranged, substituted, combined, separated, and designed in a wide variety of different configurations, all of which are explicitly contemplated herein.
- The methods and systems disclosed herein generally relate to wireless directional identification and communication between wearable heads-up displays. First, examples of wearable heads-up displays will be discussed, followed subsequently by discussions of their operation and input interaction.
- 1. Wearable Heads-Up Display Devices and Systems
-
FIG. 1 shows an example embodiment of a wearable heads-up display device. WhileFIG. 1 illustratesglasses 102 as an example of a wearable heads-up display device, other types of wearable heads-up displays could additionally or alternatively be used As illustrated inFIG. 1 ,glasses 102 comprise frame elements including lens-frames respective lens elements center frame support 112, two extending side-arms operable touch pads center support 112 and the two extending side-arms glasses 102 to a user's face via a user's nose and ears, respectively. Each of theframe elements arms glasses 102. Each of side-arms -
FIG. 2 shows an example embodiment of a wearable heads-up display system. As shown inFIG. 2 , a wearable heads-updisplay system 200 may includeglasses 102 coupled to acomputing device 202 via aconnection 206. The structure ofcomputing device 202 will be described in more detail with respect toFIG. 9 . In one embodiment, thecomputing device 202 may be incorporated into theglasses 102 themselves. In another embodiment, thecomputing device 202 may be a head-mounted computing device incorporated into, for example, a hat or helmet, or may be a body-mounted computing device incorporated into, for example, a waist-mounted cell phone or personal digital assistant. Theconnection 206 may be a wired and/or wireless link. A wired link may include, for example, a parallel bus or a serial bus such as a Universal Serial Bus (USB). A wireless link may include, for example, Bluetooth, IEEE 802.11, Cellular (such as (SM, CDMA, UMTS, EV-DO, WiMAX, or LTE), or Zigbee, among other possibilities. Theconnection 206 may function to transmit data and/or commands to and/or from theglasses 102, to transmit input received from finger-operable touch pads respective lenses 108 and/or 110. -
FIG. 3 shows an example embodiment of a wearable heads-up display device including display elements. As shown inFIG. 3 , thelenses Glasses 102 may include aminiature projector 302 coupled to an inside-surface of side-arm 116 and configured to project a display onto an inside-surface oflens 108. For example, the projected display may include acurrent time 304. Additionally or alternatively, asecond projector 306 may be coupled to an inside-surface of side-arm 114 and configured to project a display onto an inside-surface oflens 110. For example, the projected display may include a current battery-level 308 associated with thesystem 200. Additionally, asignal strength indicator 310 may provide a user with an indication of a signal strength associated with a wireless connection (such asconnection 208 with remote device 210). Whileindicator 310 is illustrated as providing a plurality of signal strength bars, other types of signal strength displays could be used, such as a numeric text, a line-graph, etc. - In
FIG. 3 , thelenses projectors projectors lens - Returning to
FIG. 2 , the wearable heads-updisplay system 200 may also communicate with aremote device 210 via theconnection 208.Connection 208 may be a wired and/or wireless link having one or more characteristics described above in relation toconnection 206. Theremote device 210 may be a device associated with the wearable heads-up display system 200 (or a user thereof, not shown), and may be a cell phone, a personal data assistant (PDA), or some other device. For example, theremote device 210 may be a cell phone having Bluetooth capabilities, and may provide information for display onrespective lenses 108 and/or 110, or may provide a target for transmission of data or instructions responsive to input operations received via finger-operable touch pads connection 208 may comprise one or more base stations, routers, switches, LANs, WLANs, WANs, access points, or other network infrastructure. For example,remote device 210 may be accessible via the Internet, and may comprise a computing cluster associated with a particular web service (e.g., social-networking, photo sharing, address book, etc.). -
FIG. 4 shows an example embodiment of various input interfaces forglasses 102 that allows a user to interact with theglasses 102 andcomputing device 202. The input interfaces may comprise one or more of finger-operable touch pad 122, amovement sensor 402, and amicrophone 404, among other possible input elements. WhileFIG. 4 illustrates a side-view of side-arm 116, additional and similar input interfaces may be provided on side-arm 114. For example, and as illustrated inFIG. 3 , an additional finger-operable touch pad 124 may be provided on side-arm 114. - Returning to
FIG. 4 , the finger-operable touch pad 122 may sense at least one of a position and movement of a finger along a planar direction relative to a surface of the touch pad 122 (e.g., parallel to the surface ofFIG. 4 ) via capacitive sensing, resistance sensing, and/or via a surface acoustic wave (SAW) process, among other possibilities. In addition, the finger-operable touch pad 122 may be capable of sensing movement of a finger in a direction normal to the surface of the touch pad 122 (e.g., into the surface ofFIG. 4 ), including perhaps sensing a level of pressure applied to thetouch pad 122. - In a capacitive touch pad, one or more insulating layers are coated with one or more conducting layers, and a driving signal applied to at least one of the one or more conducting layers. As a user's body acts as a conductor, touching the pad with one's finger causes a distortion in at least one of the conducting layer's electrostatic field, measurable as a change in capacitance. Different capacitive technologies may be used to determine the location of the touch. For example, in a surface capacitance method, only oneside of an insulating layer is coated with a conductive layer. A small voltage is then applied to the conductive layer, resulting in an electrostatic field. When a user's finger touches the touch pad surface, a capacitor is dynamically formed, and a controller can determine the location of the touch indirectly from the change in capacitance. Alternatively, in a mutual capacitance method, vertically and horizontally-arranged driving lines (e.g., two conductive layers) are formed separated by an insulating layer. Bringing a finger close to the surface of the array changes the local electrostatic field around an intersection of the separated driving lines, changing the mutual capacitance between driving lines at corresponding intersecting areas. Because the capacitance change can be measured simultaneously at each intersecting point of the driving lines, mutual capacitance can be used to determine touch locations at a plurality of locations (e.g., multi-touch).
- In a resistive touch pad, two electrically conductive layers having horizontal and vertical lines are formed separated by an insulating gap (e.g., glass, plastic, air, etc.), and a voltage gradient is applied to the first conductive layer. When contact is made with the surface of the touch pad, the two conductive layers are pressed together, and the second sheet measures the voltage as distance along the first sheet, providing an X coordinate. After the X contact coordinate has been acquired, a second voltage gradient is applied to the second sheet to ascertain the Y coordinate. These two operations provide the touch location that contact was made.
- in a SAW touch pad, conductive layers are not disposed throughout the pad itself. Rather, transmitting and receiving transducers and reflectors are disposed at edges of the track pad. Waves emitted by the transmitting transducers are reflected across the touch pad in the X and Y directions and to receiving transducers via the reflectors. When a finger touches the screen, portions of the waves are absorbed, causing a touch event and its corresponding location to be detected by control circuitry.
- While several types of touch pads are discussed here, other currently available and other future-developed touch-detection methods are included within the scope of this disclosure.
- As illustrated in
FIG. 4 , a width of the side-arm 116 may be formed thicker in a region in which thetouch pad 122 is formed, and thinner in a region in which thetouch pad 122 is not formed, so as to accommodate sufficient space to detect finger movements in all planar directions (e.g., 360 c), or at the very least, two pairs of diametrically opposed directions such as up, down, forward, and back. - Because the expanded width of the side-
arm 116 in the region of thetouch pad 122 may impede the peripheral vision of the user's eyes and/or may block the entrance of light, the side-arm 116 and/or thetouch pad 122 may be formed of a translucent or substantially transparent material. For example, the side-arm 116 may be formed of a translucent or substantially transparent plastic material such as Acrylic (polymethlmethacrylate), Butyrate (cellulose acetate butyrate), Lexan polycarbonate), and PETG (glycol modified polyethylene terphthalate). Other types of plastics could also be used. Translucent or substantially transparent materials other than plastic could also be used. - The
touch pad 122 may be formed of one or more translucent or transparent insulating (e.g., glass or plastic) layers and one or more translucent or transparent conducting (e.g., metal) layers. The glass may be tempered or toughened glass manufactured through a process of extreme heating and rapid cooling. The plastic may be a polyimide, polyethylene, or polyester based plastic film. Other types of translucent and/or substantially transparent glasses and plastics could also be used. The conducting layer may be formed of a metal oxide, such as Indium Tin Oxide (ITO). Other types of insulating and conducting layers could also be used. - Edges of the
touch pad 122 may be formed to have a raised, indented, or roughened surface, so as to provide tactile feedback to a user when the user's finger reaches the edge of thetouch pad 122. Such a structure may also allow a user (who has limited or no visual cues as to the location of the touch pad 122) to locate thetouch pad 122 on the side-arm 116 quickly, similar to the way in which physical indentions normally provided on the “F” and “J” keys of a keyboard allow a typist to quickly position their fingers correctly on the keyboard. Of course, the raised indented, and/or roughened surface could alternatively or additionally be formed in the side-arm 116 just past the edge of thetouch pad 122. A similar roughened, raised, or indented element may be provided at substantially a center of thetouch pad 122 to provide additional tactile cues to a user. - The
movement sensor 402 may be provided on or in a frame element of theglasses 102, and may act as an input device configured to track a user's movements. Themovement sensor 402 may include one or more of an accelerometer, a magnetometer, or a gyroscope, among other options. An accelerometer is a device that measures acceleration. Single- and multi-axis models can detect magnitude and direction of the acceleration as a vector quantity, and can be used to sense orientation, acceleration, vibration shock, and falling. A gyroscope is a device for measuring or maintaining orientation, based on the principles of conservation of angular momentum. One type of gyroscope, a microelectromechanical system (MEMS) based gyroscope, uses lithographically constructed versions of one or more of a tuning fork, a vibrating wheel, and resonant solids to measure orientation. Other types of gyroscopes could be used as well. A magnetometer is a device used to measure the strength and/or direction of the magnetic field in the vicinity of the device, and can be used to determine a direction in which a person or device is facing. Other types of movement sensors could additionally, or alternatively, be used. - The
movement sensor 402 may be used, for example, to determine when, how much, and perhaps how quickly, a user wearing theglasses 102 turns or moves his or her head or body to the right, left, tilted up, or tilted down. Thesensor 402 may also be able to determine a cardinal direction in which the user is facing. -
Microphone 404 may be any acoustic-to-electric transducer or sensor that converts sound into an electrical signal. For example,microphone 404 may use electromagnetic induction, capacitance change, piezoelectric generation, or light modulation, among other techniques, to produce an electrical voltage signal from mechanical vibration. Themicrophone 404 may communicate with a speech recognition program atcomputing device 202 to allow a user to speak voice commands that cause thecomputing device 202 to take particular action(s). Themicrophone 404 may also be used for other purposes. - While one touch-
sensitive pad 122, onemovement sensor 402, and onemicrophone 404 is illustrated inFIG. 4 , in some embodiments a subset of these devices may be provided. In at least one embodiment, a plurality of touch-sensitive pads may be disposed on the side-arm 116 and/or the side-arm 114. In another embodiment, an array of (same or different) microphones or array of (same or different) movement sensors may be provided on the side-arm 116 and/or the side-arm 114. Additionally, thetouch pad 122 may be provided having a different shape or dimensions than that shown inFIG. 4 . - The input interface may be wiredly or wirelessly coupled to the computing device 202 (perhaps via connection 206) to allow a user to control settings and features of the wearable heads-up
display system 200, to initiate communications with other wearable heads-up displays, to provide positioning and/or movement information fromsensor 402, and/or to control and interact with displays on thelens - 2. Input Interactions Using the Wearable Heads-Up Display System
-
FIGS. 5 and 6 illustrate example input interactions in which one or more of thetouch pads glasses 102.FIG. 5 illustrates a display of input commands and their associated functions relative to a display of text (such as an e-mail), andFIG. 6 illustrates a display of input commands and their associated functions relative to interacting with real-world objects. Of course, these figures are exemplary in nature only, and many other applications and combinations of input commands and associated functions are possible in light of this disclosure. -
FIG. 5 illustrates one embodiment in which interactions with displayed text (such as e-mails in an e-mail inbox) may be accomplished via one or more of thetouch pads touch pad 122 ortouch pad 124 may be operated, and the same input function performed by theglasses 102 in response to the input. One advantage of providingtouch pads glasses 102 is that thesame glasses 102 may be used naturally by both left-handed and right-handed persons. - Although
FIG. 5 illustrates a simultaneous display of different content on bothlenses lenses lens lenses FIG. 5 at any one time. In a further embodiment, bothlenses displays - As illustrated in
FIG. 5 , acontent display 500 projected onlens 110 by projectingdevice 306 may include text of an e-mail, perhaps retrieved from an e-mail inbox associated with a user of theglasses 102 and stored atremote device 210. The e-mail text may represent just one e-mail out of a plurality of available e-mails. Ascroll bar 502 may provide an indication of a relative position of the displayed text relative to a remainder of the e-mail from which the text is retrieved. - In order to aid a user in determining how to use
touch pad content display 500, and prior to showing thecontent display 500, upon request, or at periodic intervals, ahelp display 504 intended to inform the user of available input commands that may be executed on one or more oftouch pads lens 108 via projectingdevice 302. During the time in which thehelp display 504 is being displayed, thecontent display 500 may be removed fromlens 110, corresponding overlapping portions of thecontent display 500 augmented, or a user expected or instructed to close a corresponding eye looking throughlens 110. In this example, thehelp display 504 includes aScroll Up command 506, aScroll Down command 508, aNext E-mail command 510, and aPrevious E-mail command 512. Each of the commands 506-512 is displayed with a corresponding symbol illustrating an input operation that may be executed on one of thetouch pads 122 and/or 124 to execute the associated function. For example, theScroll Down command 508 may be executed by a user swiping their finger acrosstouch pad 124 in a downwards direction (as indicated by the downward facing arrow symbol). In response to receiving the Scroll Down command, thecomputing device 202 may cause thecontent display 500 to be updated with additional text not previously displayed, and may cause thescroll bar 502 to advance downwards in accordance with a length of the e-mail from which the text is pulled. The Scroll Up command 506 (as indicated by the upward facing arrow symbol) may cause thecontent display 500 to be updated with previously displayed text, and may cause thescroll bar 502 to retract upwards in accordance with a length of the e-mail from which the text is pulled. - The
Next E-mail command 510 may cause thecontent display 500 to be updated with text corresponding to a next e-mail (e.g., amore recently received or less recently received e-mail) from the e-mail inbox associated with the user. ThePrevious E-mail command 512 may cause thecontent display 500 to be updated with text corresponding to a previous e-mail that was, perhaps, previously shown in thecontent display 500. Other commands and other types of text, symbols, or images could be used as well. - In one embodiment, the commands and their associated functions may be modified by a user. For example, although in
FIG. 5 a forward-swipe acrosstouch pad 124 is associated with loading a next e-mail, it may be more intuitive for some users that a backward-swipe acrosstouch pad 124 loads the next e-mail. Associations between commands and associated functions may, for example, be stored in a list or database incomputing device 202 and/or atremote device 210. A locally or remotely accessible interface may allow the user to access the stored list or database and modify the associations. For example, the user may be able to access thecomputing device 202 and/or theremote device 210 via a wired or wireless connection, and modify the Next E-mail command to be associated with a backward-swipe acrosstouch pad 124 instead of a forward-swipe. The computing device 202 (or remote device 210) may, in response to recognizing that the newly assigned backward-swipe was previously assigned to another function (the previous e-mail function), request the user to associate a new command with the previous e-mail function. In response, for example, the user may record a forward-swipe acrosstouch pad 124 as being associated with the previous e-mail function. The selection of a new command may be executed by a user selecting the new command from a list, or may be executed by the user entering the new command via thetouch pad 124 when requested (e.g., swiping forward across thetouch pad 124 when requested). In this manner, unique patterns of input commands can be created by a user, and personalized per user, by recording a finger-motion pattern detected at thetouch pad 124 when requested. - In one embodiment, the list or database stored at
computing device 202 may be customized per application, such that, for example, the change to the Next E-mail command is applied to the e-mail function alone, and not to other functions, such as web browsing (e.g., a Next Link or Next Page command). In another embodiment the list or database may store global commands that, once changed, are applied to other related, or to all, functions executed atcomputing device 202. Other possibilities exist as well. - in some embodiments, the
help display 504 may be displayed inlens 108 and/or 110 when the e-mail application is first started, so as to remind the user of the available input commands executable via,touch pad 122 and/or 124 to navigate the e-mail application. After displaying thehelp display 504 for some period of time (e.g., 1-5 seconds), thehelp display 504 may be replaced with the content display 500 (inlens 108 and/or 110). Subsequently, thehelp display 504 may be displayed only upon demand (perhaps via, a particular motion acrosstouch pad help display 504, a particular area oftouch pad help display 504, or an algorithm executing atcomputing device 202 that detects that a user is having difficulty navigating viatouch pad 122 or 124).Computing device 202 may detect that a user is having difficulty navigating based on a number of unrecognized movements acrosstouch pad -
FIG. 6 illustrates an embodiment in which interactions with real world objects may be accomplished via one or more of thetouch pads touch pad touch pads FIG. 6 ,touch pad 124 would be associated with the user's dominant hand, whiletouch pad 122 would be associated with the user's non-dominant hand. Different functions may be assigned to similar input operations executed at arespective touch pad - As shown in
FIG. 6 , real-world objects 600 and 602 are viewable through translucent and/ortransparent lenses world object 600 is illustrated as a soccer ball, while real-world object 602 is illustrated as a chair. Whileobjects lens 108 and 110), it should be understood that there is really only one of eachobject FIG. 6 is meant to reflect the real-world binaural vision characteristics of the human-user (e.g., viewing theobjects Selection indicator 604 is a super-imposed selection indicator projected by one or more of projectingdevices respective lenses 108 and/or 110. A camera (not shown) disposed onglasses 102 may be configured to capture the user's field-of-view, and recognize particular objects for selection, such as thesoccer ball 600 and/orchair 602. Help displays 606 and/or 608 may be projected onlenses world Objects displays displays help displays -
Help display 606 provides, for example, functions and associated commands for selecting an object recognized by computing system 202 (e.g., via the camera). For example, theselection indicator 604 may be displayed over a randomly selected object out of a plurality of objects recognized by the computing system 202 (or, perhaps, displayed over an object the that theglasses 102 detect that the user is most interested in). As shown inFIG. 6 , thesoccer ball 600 may be initially selected. Theselection indicator 604 may be displayed inlens 108 and/orlens 110 via respective projectingdevices help display 606 may be executed by double-tapping thetouch pad 124 with a single finger (illustrated with a symbol comprising a dot within an outer circle). Selecting the currently highlighted object (e.g., the soccer ball inFIG. 6 ) in such a manner may allow for further functions to be executed with respect to the selected object. For example, once an object is selected, the selected object may be used as a focus-point for taking a picture via an imaging device (not shown) integrated with theglasses 102. Additionally or alternatively, an image or information search may be conducted using an image of the selected object. For example, an image of thesoccer ball 600 may be used to locate other images of soccer balls via an online search function, to conduct an online product search function to find soccer balls available for purchase, or to obtain information regarding the sport of soccer or soccer balls in general (history, structure, etc.). - The Choose Another
Object command 612 ofhelp display 606 may be executed by a single-tap on thetouch pad 124 with a single finger (illustrated with a symbol comprising an empty circle). The Choose AnotherObject command 612 may cycle through each of the plurality of recognized objects within the current field of view. For example, single-tapping touch pad 124 may cause theselection indicator 604 to move from thesoccer ball 600 to the chair 602 (and may simultaneously cause theselection indicator 604 to change its shape to accommodate the size and/or geometries of the chair 602). Once thechair 602 is selected, the SelectThis Object command 610 may be executed via a double-tap using a single finger ontouch pad 124 to find information and/or images with respect to thechair 602. Other possibilities exist as well. - Similar to the
help display 504, thehelp display 606 may be displayed inlens 108 and/or 110 when the object selection application is first stated, so as to remind the user of the available input commands, executable viatouch pad help display 606 for some period of time (e.g., 1-5 seconds), thehelp display 606 may be removed. Subsequently, thehelp display 606 may be displayed only upon demand (perhaps via a particular motion acrosstouch pad help display 606, a particular area oftouch pad help display 606, or an algorithm executing atcomputing device 202 that detects that a user is having difficulty navigating viatouch pad 122 or 124). -
Help display 608 provides, for example, functions and associated commands for capturing an image of a scene as viewed throughlenses selection indicator 604 may provide a focus point for an image capture process viacommands Capture Image command 614 ofhelp display 608, for example, may be executed by a two-finger single-tap on touch pad 124 (illustrated with a symbol comprising two adjacent empty circles), and may cause the camera to capture an image without a flash, using the currently-selectedobject 600 as the focal point. The Capture WithFlash command 616 ofhelp display 606, for example, may be executed by a two-finger double-tap on touch pad 124 (illustrated with a symbol comprising two adjacent dotes within respective outer circles), and may cause the camera to capture an image with a flash, using the currently-selectedobject 600 as the focal point. Similar to the disclosure above relative to help display 504, the input commands associated with thefunctions computing device 202. Additionally, and similar to the disclosure above relative to help display 606,help display 608 may only be displayed as necessary, and otherwise, may be removed fromlens 110. - As set forth earlier,
touch pads glasses 102. In the arrangement illustrated onlens 110 inFIG. 6 , and assuming a right-handed user,touch pad 122 may provide gross motor movement of theselection indicator 604 for image capture focusing purposes, andtouch pad 124 may provide fine motor movement of the selection indicator 604 (for the same or different purpose). For example, thetouch pad 122 may allow a user to move theselection indicator 604 quickly to the top of the user's field of view via a relatively short upwards-swipe across the touch pad 122 (e.g., a full swipe acrosstouch pad 122 in the vertical direction may cause a greater than 50% movement of theselection indicator 604 across a user's field of view). On the other hand, thetouch pad 124 may allow the user to move theselection indicator 604 in small increments to fine tune the focus selection (e.g., a full swipe acrosstouch pad 124 in the vertical direction may cause a less than 10% movement of theselection indicator 604 across a user's field of view). Other applications of using gross and fine motor input betweentouch pads -
FIG. 7 a illustrates an additional or alternative embodiment for interacting withglasses 102. As illustrated inFIG. 7 a, animage capture device 702 and alight source 704 may be coupled to side-arm 116 (in addition to, or in place of, touch pad 122). Although thelight source 704 andimage capture device 702 are illustrated as separate devices, they may be combined into a single device. Thelight source 704 andimage capture device 702 work together to illuminate a surface of a user's finger, and to detect the unique, time patterns in the surface of the user's finger. By capturing images of these patterns many times per second (e.g., on the order of hundreds or thousands per second), slight movements in the user's finger can be detected by comparing a reference frame (previous image) to a current frame (present image), and the difference transformed into corresponding movements for input tocomputing device 202. Thelight source 704 may be, for example, a laser or LED that emits light in the visible (e.g., red) or invisible (e.g., infra-red) spectrum. Theimage capture device 702 may be a complementary metal-oxide-semiconductor (CMOS) or charge-coupled device (CCD) image capture device capable of detecting wavelengths in the range emitted by thelight source 704. Interfacing withglasses 102 viaimage capture device 702 andlight source 704 may be accomplished in the same manner as set forth above with respect toFIG. 1-6 . -
FIG. 7 b illustrates another additional or alternative embodiment for interacting withglasses 102. As illustrated inFIG. 7 b, atouch pad 706 may be coupled to side-arm 116 and extend beyond the edges of the side-arm 116. While this arrangement provides for additional gesturing space and allows a user to create more advanced input patterns, it also blocks more light from a user's field of view, and blocks a user's peripheral vision to a greater extent than theintegrated touch pad 122 ofFIG. 4 . Thus, in this scenario, the level of translucency and/or transparency of thetouch pad 706 may become more important. Additionally, and advantageously, thetouch pad 706 in this arrangement may be removable from the side-arm 116, and may be attached only when needed by a heads-up display user. Removable fasteners may include, among others, Velcro, hook and tabs, buttons, snaps, friction fittings, screws, strike and latch fittings, compression fittings, rivets, and grommets. Permanent fasteners could additionally or alternatively be used. An electrical connection to thetouch pad 706 may be provided via a connector on the outer-surface of the side-arm 116, and communication between thetouch pad 706 andcomputing device 202 may take place via a wired or wireless connection. Interfacing withglasses 102 viatouch pad 706 may be accomplished in the same manner as set forth above with respect toFIG. 1-6 . - Although
FIGS. 5-7 focus on input commands relative totouch pads 122 and/or 124, help displays 500, 504, 606, and 608, in one embodiment, may also include commands and corresponding symbols relative to movements detectable bymovement sensor 408 and/or commands and symbols relative to voice commands detectable bymicrophone 410. -
FIG. 8 is a flow-chart illustrating anexample method 800 of interfacing with a heads-up display, such asglasses 102. Themethod 800 includes afirst display step 802, aninput step 804, and asecond display step 806. - At
step 802, display information is provided to at least one display element of a wearable heads-up display. This display information may include one or more supported functions relative to a currently-executing application, and may include, for each function, an associated input command (illustrated via a symbol) that may be executed at an input device to cause the corresponding function to be executed or corresponding selection to be selected. The associated input commands may be loaded from a list or database stored atcomputing device 202 and/or atremote device 210, and may vary depending upon a determination of the current application being executed by computingdevice 202. - At
step 804, input information is received from a coupled finger-operable input device regarding a position or movement of a finger along a planar direction relative to a surface of the input device. This input information may be recognized as equal or equivalent to one of the associated input commands included in the display information atstep 802. Atstep 806, new display information is provided to at least one display element (and perhaps the same at least one display element as in step 802) responsive to receiving the input information. - 3. Example Hardware for a Wearable Heads-Up Display System
-
FIG. 9 is a functional block diagram of acomputing device 202 for supporting the wearable heads-up displays set forth above arranged in accordance with at least some embodiments described herein. Thecomputing device 202 may be a personal computer, mobile device, cellular phone, video game system, global positioning system, or other electronic system. In a very basic configuration 901,computing device 202 may typically include one or more processors or controllers (processor) 910 andsystem memory 920. A memory bus 930 can be used for communicating between theprocessor 910 and thesystem memory 920. Depending on the desired configuration,processor 910 can be of any type including, but not limited to, a microprocessor (μP), a microcontroller (μC), a digital signal processor (DSP), or any combination thereof. Amemory controller 915 can also be used with theprocessor 910, or in some implementations, thememory controller 915 can be an internal part of theprocessor 910. - Depending on the desired configuration, the
system memory 920 can be of any type including but not limited to volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory, etc.) or any combination thereof.System memory 920 typically includes one ormore applications 922 andprogram data 924.Application 922 may include algorithms such as input/output device interface algorithms 923 arranged to control and interface with input devices such as finger-operable touch pads, in accordance with the present disclosure Other process descriptions, steps, or blocks in flow or message diagrams in the present disclosure should be understood as potentially representing modules, segments, or portions of code which include one or more executable instructions stored inapplication memory 922 for implementing specific logical functions or steps in the process, and alternate implementations are included within the scope of the preferred embodiments of the methods in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art. -
Program data 924 may include, among other things, displaysymbols 925 that correspond to commands that may be executed via corresponding finger-operable touch pad operations (or other input interfaces), and that may be included in display data sent to one ormore display devices 992. In some example embodiments, applications stored inapplication memory 922 can be arranged to operate withprogram data 924.Computing device 202 can have additional features or functionality, and additional interfaces to facilitate communications between the basic configuration 901 and any devices and interfaces. For example, thedata storage devices 950 can beremovable storage devices 951,non-removable storage devices 952, or a combination thereof. Examples of removable storage and non-removable storage devices include magnetic disk devices such as flexible disk drives and hard-disk drives (HDD), optical disk drives such as compact disk (CD) drives or digital versatile disk (DVD) drives, solid state drives (SSD), and tape drives to name a few. - Computer storage media can include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, or other data.
-
System memory 920, removable storage media for use withremovable storage devices 951, andnon-removable storage 952 are all examples of computer storage media. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory, or other memory technology, CD-ROM, digital versatile disks (DVD), or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage, or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by computingdevice 202. -
Computing device 202 can also includeoutput interfaces 960 that may include agraphics processing unit 961, which can be configured to communicate to various external devices such as display devices 992 (which may include, for example, projectingdevices lenses 108, 110) or speakers via one or more A/V ports 963.External communication circuits 980 may include anetwork controller 981, which can be arranged to facilitate communications with one or moreother computing devices 990 and/or one or more transmitting and/or receiving devices 991. The communication connection is one example of a communication media. Communication media may typically be embodied by computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and includes any information delivery media. A “modulated data signal” can be a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media can include wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, radio frequency (RF), infrared (IR) and other wireless media. The term computer readable media as used herein can include both storage media and communication media. The term tangible computer readable media may refer to storage media alone. -
Computing device 202 can be implemented as a portion of a small-form factor portable (or mobile) electronic device such as a cell phone, a multi-chip module (MCM), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a PDA, a personal media player device, a wireless web-watch device, a personal headset device, an application specific device, or a hybrid device that include any of the above functions.Computing device 202 can also be implemented as a personal computer including both laptop computer and non-laptop computer configurations. - It should be further understood that arrangements described herein are for purposes of example only. As such, those skilled in the art will appreciate that other arrangements and other elements (e.g. machines, interfaces, functions, orders, and groupings of functions, etc.) can be used instead, and some elements may be omitted altogether according to the desired results. Further, many of the elements that are described are functional entities that may be implemented as discrete or distributed components or in conjunction with other components, in any suitable combination and location.
- The present disclosure is not to be limited in terms of the particular embodiments described in this application, which are intended as illustrations of various aspects. Many modifications and variations can be made without departing from its spirit and scope, as will be apparent to those skilled in the art. Functionally equivalent methods and apparatuses within the scope of the disclosure, in addition to those enumerated herein, will be apparent to those skilled in the art from the foregoing descriptions. Such modifications and variations are intended to fall within the scope of the appended claims.
- In some embodiments, the disclosed methods may be implemented as computer program instructions encoded on a computer-readable storage media or tangible computer-readable storage media in a machine-readable format.
FIG. 10 is a schematic illustrating a conceptual partial view of an examplecomputer program product 1000 that includes a computer program for executing a computer process on a computing device, arranged according to at least some embodiments presented herein. In one embodiment, the examplecomputer program product 1000 is provided using a signal bearing medium 1001. The signal bearing medium 1001 may include one ormore programming instructions 1002 that, when executed by one or more processors, may provide functionality or portions of the functionality described above with respect toFIGS. 1-8 . Thus, for example, referring to the embodiment shown inFIG. 8 , one or more features ofmethod 800 may be undertaken by one or more instructions associated with the signal bearing medium 1001. - In some examples, the signal bearing medium 1001 may encompass a tangible computer-
readable medium 1003, such as, but not limited to, a hard disk drive, a Compact Disc (CD), a Digital Video Disk (DVD), a digital tape, memory, etc. In some implementations, the signal bearing medium 1001 may encompass acomputer recordable medium 1004, such as, but not limited to, memory, read/write (R/W) CDs, R/W DVDs, etc. In some implementations, the signal bearing medium 1001 may encompass a communications medium 1005, such as, but not limited to, a digital and/or an analog communication medium (e.g., a fiber optic cable, a waveguide, a wired communications link, a wireless communication link, etc). Thus, for example, the signal bearing medium 1001 may be conveyed by a wireless form of the communications medium 1005 (e.g., a wireless communications medium conforming with the IEEE 802.11 standard or other transmission protocol). - The one or
more programming instructions 1002 may be, for example, computer executable and/or logic implemented instructions. In some examples, a computing device such as thecomputing device 202 ofFIG. 9 may be configured to provide various operations, functions, or actions in response to theprogramming instructions 1002 conveyed to thecomputing device 202 by one or more of the computer readable medium 1003, thecomputer recordable medium 1004, and/or thecommunications medium 1005. - While various aspects and embodiments have been disclosed herein, other aspects and embodiments will be apparent to those skilled in the art. The various aspects and embodiments disclosed herein are for purposes of illustration and are not intended to be limiting, with the true scope and spirit being indicated by the following claims, along with the full scope of equivalents to which such claims are entitled. It is also to be understood that the terminology used herein is for the purpose of describing particular embodiments only, and is not intended to be limiting.
Claims (20)
1. A system comprising:
a processor;
a display element, wherein the display element is configured to receive display information from the processor and to display the display information;
a wearable frame structure supporting the display element, the wearable frame structure comprising two projections disposed on opposite sides of a front portion and extending away from the front portion, wherein the wearable frame structure is configured to be worn in a position for viewing the display information displayed by the display element, and wherein the display element is coupled to a first projection of the two projections; and
a finger-operable input device secured to the first projection and configured to provide input information to the processor and sense at least one of a position and movement of a finger along a planar direction relative to a surface of the finger-operable input device.
2. The system of claim 1 , wherein the processor is configured to transmit new display information to the display element responsive to the processor receiving the input information.
3. The system of claim 1 , wherein the display information comprises at least one symbol indicating a possible input operation that may be made using the finger-operable input device and one of a function and a selection associated with the possible input operation.
4. The system of claim 1 , wherein the finger-operable input device is translucent.
5. The system of claim 1 , further comprising a second finger-operable input device secured to the wearable frame structure via at least one of the two projections and configured to provide input information to the processor and sense at least one of a position and movement of a finger along a planar direction relative to a surface of the second finger-operable input device.
6. The system of claim 1 , wherein the finger-operable input device comprises a glass substrate.
7. The system of claim 1 , wherein the finger-operable input device comprises a plastic substrate.
8. The system of claim 1 , wherein the display element comprises a projector element.
9. The system of claim 1 , wherein the finger-operable input device comprises a capacitive sensing, resistance sensing, and surface acoustic wave (SAW) touch sensitive pad.
10. The system of claim 1 , wherein the finger-operable input device comprises an optical imaging device.
11. A method comprising:
providing, by a processor, display information to a display element, wherein the display element is supported by a wearable frame structure comprising two projections disposed on opposite sides of a front portion and extending away from the front portion, wherein the wearable frame structure is configured to be worn in a position for viewing the display information displayed by the display element, and wherein the display element is coupled to a first projection of the two projections;
receiving, by the processor, input information provided by a finger-operable input device, wherein the finger-operable input device is configured to sense at least one of a position and movement of a finger along a planar direction relative to a surface of the finger-operable input device, wherein the input information is representative of the at least one of the position and movement of the finger along the planar direction, wherein the finger-operable input device is secured to the wearable frame structure via the first projection; and
providing, by the processor, new display information to the display element responsive to the processor receiving the input information.
12. The method of claim 11 , wherein the display information comprises at least one symbol indicating a possible input operation that may be made using the finger-operable input device and one of a function and a selection associated with the possible input operation, and
wherein receiving input information comprises receiving an indication of the possible input operation.
13. The method of claim 11 , wherein the finger-operable input device is translucent.
14. The method of claim 11 , further comprising receiving, by the processor, input information provided by a second finger-operable input device secured to the wearable frame structure via at least one of the two projections and configured to provide input information to the processor and sense at least one of a position and movement of a finger along a planar direction relative to a surface of the second finger-operable input device.
15. The method of claim 11 , further comprising projecting the display information using a projector element in the display element.
16. An article of manufacture including a computer-readable medium having instructions stored thereon that, upon execution by a computing device, cause the computing device to perform operations comprising:
providing, by a processor, display information to a display element, wherein the display element is supported by a wearable frame structure comprising two projections disposed on opposite sides of a front portion and extending away from the front portion, wherein the wearable frame structure is configured to be worn in a position for viewing the display information displayed by the display element, and wherein the display element is coupled to a first projection of the two projections;
receiving, by the processor, input information provided by a finger-operable input device, wherein the finger-operable input device is configured to sense at least one of a position and movement of a finger along a planar direction relative to a surface of the finger-operable input device, wherein the input information is representative of the at least one of the position and movement of the finger along the planar direction, wherein the finger-operable input device is secured to the wearable frame structure via the first projection; and
providing, by the processor, new display information to the display element responsive to the processor receiving the input information.
17. The article of manufacture of claim 16 , wherein the display information comprises at least one symbol indicating a possible input operation that may be made using the finger-operable input device and one of a function and a selection associated with the possible input operation, and
wherein receiving input information comprises receiving an indication of the possible input operation.
18. The article of manufacture of claim 16 , wherein the operations further comprise receiving, by the processor, input information provided by a second finger-operable input device secured to the wearable frame structure via at least one of the two projections and configured to provide input information to the processor and sense at least one of a position and movement of a finger along a planar direction relative to a surface of the second finger-operable input device.
19. The article of manufacture of claim 16 , wherein the operations further comprise projecting the display information using a projector element in the display element.
20. The article of manufacture of claim 16 , wherein the finger-operable input device is translucent.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/477,547 US20120299870A1 (en) | 2011-05-25 | 2012-05-22 | Wearable Heads-up Display With Integrated Finger-tracking Input Sensor |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/115,430 US8203502B1 (en) | 2011-05-25 | 2011-05-25 | Wearable heads-up display with integrated finger-tracking input sensor |
US13/477,547 US20120299870A1 (en) | 2011-05-25 | 2012-05-22 | Wearable Heads-up Display With Integrated Finger-tracking Input Sensor |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/115,430 Continuation US8203502B1 (en) | 2011-05-25 | 2011-05-25 | Wearable heads-up display with integrated finger-tracking input sensor |
Publications (1)
Publication Number | Publication Date |
---|---|
US20120299870A1 true US20120299870A1 (en) | 2012-11-29 |
Family
ID=46209551
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/115,430 Active US8203502B1 (en) | 2011-05-25 | 2011-05-25 | Wearable heads-up display with integrated finger-tracking input sensor |
US13/477,547 Abandoned US20120299870A1 (en) | 2011-05-25 | 2012-05-22 | Wearable Heads-up Display With Integrated Finger-tracking Input Sensor |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/115,430 Active US8203502B1 (en) | 2011-05-25 | 2011-05-25 | Wearable heads-up display with integrated finger-tracking input sensor |
Country Status (5)
Country | Link |
---|---|
US (2) | US8203502B1 (en) |
EP (1) | EP2715432A4 (en) |
KR (1) | KR101473143B1 (en) |
CN (1) | CN103718082B (en) |
WO (1) | WO2012162304A2 (en) |
Cited By (26)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20140022192A1 (en) * | 2012-07-18 | 2014-01-23 | Sony Mobile Communications, Inc. | Mobile client device, operation method, recording medium, and operation system |
CN103604439A (en) * | 2013-11-19 | 2014-02-26 | 江门市得实计算机外部设备有限公司 | Pedometer capable of realizing function switching by clicking, colliding or vibrating |
WO2014107629A1 (en) * | 2013-01-04 | 2014-07-10 | Vuzix Corporation | Interactive wearable and portable smart devices |
CN103941397A (en) * | 2013-01-23 | 2014-07-23 | 索尼公司 | Head-mounted Display, Display Apparatus, And Input Apparatus |
US8823603B1 (en) | 2013-07-26 | 2014-09-02 | Lg Electronics Inc. | Head mounted display and method of controlling therefor |
WO2014147455A1 (en) * | 2013-03-18 | 2014-09-25 | Minkovitch Zvi | Sports match refereeing system |
DE102013206173A1 (en) * | 2013-04-09 | 2014-10-09 | Bayerische Motoren Werke Aktiengesellschaft | Selection of individual elements for display on data glasses |
WO2014171568A2 (en) * | 2013-04-17 | 2014-10-23 | 한국과학기술원 | Method and apparatus of detecting touch using sound, and device using same |
WO2015005525A1 (en) * | 2013-07-10 | 2015-01-15 | Lg Electronics Inc. | Head mounted display device including multiple user interface formats and method of controlling therefor |
US9013264B2 (en) | 2011-03-12 | 2015-04-21 | Perceptive Devices, Llc | Multipurpose controller for electronic devices, facial expressions management and drowsiness detection |
WO2015077004A1 (en) * | 2013-11-20 | 2015-05-28 | Google Inc. | Systems and methods for performing multi-touch operations on a head-mountable device |
EP2889718A1 (en) | 2013-12-30 | 2015-07-01 | Samsung Electronics Co., Ltd | A natural input based virtual ui system for electronic devices |
US20150186033A1 (en) * | 2013-12-31 | 2015-07-02 | Korea Electronics Technology Institute | Glasses type device with operating means on rim and user input method using the same |
WO2015108234A1 (en) * | 2014-01-15 | 2015-07-23 | Lg Electronics Inc. | Detachable head mount display device and method for controlling the same |
WO2015191079A1 (en) * | 2014-06-13 | 2015-12-17 | Halliburton Energy Services, Inc. | Monitoring hydrocarbon recovery operations using wearable computer machines |
US20150370335A1 (en) * | 2013-02-22 | 2015-12-24 | Sony Corporation | Display control apparatus, display apparatus, display control method, and program |
WO2016027932A1 (en) * | 2014-08-21 | 2016-02-25 | Lg Electronics Inc. | Glass-type mobile terminal and control method thereof |
DE102015204780A1 (en) | 2015-03-17 | 2016-09-22 | Roschi Rohde & Schwarz AG | Portable visual display device for an electrical or electronic measuring device and measuring station arrangement |
US20160274357A1 (en) * | 2013-11-12 | 2016-09-22 | Lg Electronics Inc. | Glass type terminal |
DE102015206250A1 (en) * | 2015-04-08 | 2016-10-13 | Bayerische Motoren Werke Aktiengesellschaft | Operator inputs for systems including touch pads and displays |
US20160320919A1 (en) * | 2012-05-17 | 2016-11-03 | Hong Kong Applied Science and Technology Research Institute Company Limited | Wearable Device with Intelligent User-Input Interface |
US9729767B2 (en) | 2013-03-22 | 2017-08-08 | Seiko Epson Corporation | Infrared video display eyewear |
US10067415B2 (en) | 2014-03-19 | 2018-09-04 | Samsung Electronics Co., Ltd. | Method for displaying image using projector and wearable electronic device for implementing the same |
US20180271383A1 (en) * | 2014-12-18 | 2018-09-27 | Lg Innotek Co., Ltd. | Pulse Measurement Device and Computing Device Using Same |
US10331340B2 (en) | 2014-12-04 | 2019-06-25 | Samsung Electronics Co., Ltd. | Device and method for receiving character input through the same |
US20210263696A1 (en) * | 2015-04-10 | 2021-08-26 | Andy Bevilacqua | Cognitive Improvement System and Method of Testing |
Families Citing this family (125)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP5119636B2 (en) * | 2006-09-27 | 2013-01-16 | ソニー株式会社 | Display device and display method |
AU2011220382A1 (en) | 2010-02-28 | 2012-10-18 | Microsoft Corporation | Local advertising content on an interactive head-mounted eyepiece |
US8467133B2 (en) | 2010-02-28 | 2013-06-18 | Osterhout Group, Inc. | See-through display with an optical assembly including a wedge-shaped illumination system |
US9128281B2 (en) | 2010-09-14 | 2015-09-08 | Microsoft Technology Licensing, Llc | Eyepiece with uniformly illuminated reflective display |
US20150309316A1 (en) | 2011-04-06 | 2015-10-29 | Microsoft Technology Licensing, Llc | Ar glasses with predictive control of external device based on event input |
US9129295B2 (en) | 2010-02-28 | 2015-09-08 | Microsoft Technology Licensing, Llc | See-through near-eye display glasses with a fast response photochromic film system for quick transition from dark to clear |
US9097891B2 (en) | 2010-02-28 | 2015-08-04 | Microsoft Technology Licensing, Llc | See-through near-eye display glasses including an auto-brightness control for the display brightness based on the brightness in the environment |
US9134534B2 (en) | 2010-02-28 | 2015-09-15 | Microsoft Technology Licensing, Llc | See-through near-eye display glasses including a modular image source |
US9091851B2 (en) | 2010-02-28 | 2015-07-28 | Microsoft Technology Licensing, Llc | Light control in head mounted displays |
US9759917B2 (en) | 2010-02-28 | 2017-09-12 | Microsoft Technology Licensing, Llc | AR glasses with event and sensor triggered AR eyepiece interface to external devices |
US9366862B2 (en) | 2010-02-28 | 2016-06-14 | Microsoft Technology Licensing, Llc | System and method for delivering content to a group of see-through near eye display eyepieces |
US9341843B2 (en) | 2010-02-28 | 2016-05-17 | Microsoft Technology Licensing, Llc | See-through near-eye display glasses with a small scale image source |
US9097890B2 (en) | 2010-02-28 | 2015-08-04 | Microsoft Technology Licensing, Llc | Grating in a light transmissive illumination system for see-through near-eye display glasses |
US9285589B2 (en) | 2010-02-28 | 2016-03-15 | Microsoft Technology Licensing, Llc | AR glasses with event and sensor triggered control of AR eyepiece applications |
US9229227B2 (en) | 2010-02-28 | 2016-01-05 | Microsoft Technology Licensing, Llc | See-through near-eye display glasses with a light transmissive wedge shaped illumination system |
US10180572B2 (en) | 2010-02-28 | 2019-01-15 | Microsoft Technology Licensing, Llc | AR glasses with event and user action control of external applications |
US8477425B2 (en) | 2010-02-28 | 2013-07-02 | Osterhout Group, Inc. | See-through near-eye display glasses including a partially reflective, partially transmitting optical element |
US8472120B2 (en) | 2010-02-28 | 2013-06-25 | Osterhout Group, Inc. | See-through near-eye display glasses with a small scale image source |
US8482859B2 (en) | 2010-02-28 | 2013-07-09 | Osterhout Group, Inc. | See-through near-eye display glasses wherein image light is transmitted to and reflected from an optically flat film |
US8488246B2 (en) | 2010-02-28 | 2013-07-16 | Osterhout Group, Inc. | See-through near-eye display glasses including a curved polarizing film in the image source, a partially reflective, partially transmitting optical element and an optically flat film |
US9182596B2 (en) | 2010-02-28 | 2015-11-10 | Microsoft Technology Licensing, Llc | See-through near-eye display glasses with the optical assembly including absorptive polarizers or anti-reflective coatings to reduce stray light |
US20120249797A1 (en) | 2010-02-28 | 2012-10-04 | Osterhout Group, Inc. | Head-worn adaptive display |
US9223134B2 (en) | 2010-02-28 | 2015-12-29 | Microsoft Technology Licensing, Llc | Optical imperfections in a light transmissive illumination system for see-through near-eye display glasses |
US8749573B2 (en) * | 2011-05-26 | 2014-06-10 | Nokia Corporation | Method and apparatus for providing input through an apparatus configured to provide for display of an image |
US8873147B1 (en) * | 2011-07-20 | 2014-10-28 | Google Inc. | Chord authentication via a multi-touch interface |
US9690100B1 (en) * | 2011-09-22 | 2017-06-27 | Sprint Communications Company L.P. | Wireless communication system with a liquid crystal display embedded in an optical lens |
US9626013B2 (en) * | 2011-12-05 | 2017-04-18 | Sony Corporation | Imaging apparatus |
US9110502B2 (en) * | 2011-12-16 | 2015-08-18 | Ryan Fink | Motion sensing display apparatuses |
JP5884502B2 (en) * | 2012-01-18 | 2016-03-15 | ソニー株式会社 | Head mounted display |
US9829715B2 (en) * | 2012-01-23 | 2017-11-28 | Nvidia Corporation | Eyewear device for transmitting signal and communication method thereof |
US9001005B2 (en) | 2012-02-29 | 2015-04-07 | Recon Instruments Inc. | Modular heads-up display systems |
US9069166B2 (en) | 2012-02-29 | 2015-06-30 | Recon Instruments Inc. | Gaze detecting heads-up display systems |
US8907867B2 (en) * | 2012-03-21 | 2014-12-09 | Google Inc. | Don and doff sensing using capacitive sensors |
JP5938977B2 (en) * | 2012-03-23 | 2016-06-22 | ソニー株式会社 | Head mounted display and surgical system |
JP5953963B2 (en) * | 2012-06-13 | 2016-07-20 | ソニー株式会社 | Head-mounted image display device |
US20130339859A1 (en) | 2012-06-15 | 2013-12-19 | Muzik LLC | Interactive networked headphones |
US20180048750A1 (en) * | 2012-06-15 | 2018-02-15 | Muzik, Llc | Audio/video wearable computer system with integrated projector |
US20140002629A1 (en) * | 2012-06-29 | 2014-01-02 | Joshua J. Ratcliff | Enhanced peripheral vision eyewear and methods using the same |
US20140005807A1 (en) * | 2012-06-29 | 2014-01-02 | Black & Decker Inc. | System for Enhancing Operation of Power Tools |
WO2014022230A2 (en) | 2012-07-30 | 2014-02-06 | Fish Robert D | Electronic personal companion |
US9316830B1 (en) | 2012-09-28 | 2016-04-19 | Google Inc. | User interface |
JP5962403B2 (en) * | 2012-10-01 | 2016-08-03 | ソニー株式会社 | Information processing apparatus, display control method, and program |
US10234941B2 (en) * | 2012-10-04 | 2019-03-19 | Microsoft Technology Licensing, Llc | Wearable sensor for tracking articulated body-parts |
US9016857B2 (en) * | 2012-12-06 | 2015-04-28 | Microsoft Technology Licensing, Llc | Multi-touch interactions on eyewear |
US10528135B2 (en) | 2013-01-14 | 2020-01-07 | Ctrl-Labs Corporation | Wearable muscle interface systems, devices and methods that interact with content displayed on an electronic display |
US9791921B2 (en) | 2013-02-19 | 2017-10-17 | Microsoft Technology Licensing, Llc | Context-aware augmented reality object commands |
CN105190578A (en) | 2013-02-22 | 2015-12-23 | 赛尔米克实验室公司 | Methods and devices that combine muscle activity sensor signals and inertial sensor signals for gesture-based control |
US9477315B2 (en) | 2013-03-13 | 2016-10-25 | Honda Motor Co., Ltd. | Information query by pointing |
US9873233B2 (en) * | 2013-03-15 | 2018-01-23 | Johnson & Johnson Vision Care, Inc. | Ophthalmic lens viewing sets for three-dimensional perception of stereoscopic media |
US9361501B2 (en) | 2013-04-01 | 2016-06-07 | Ncr Corporation | Headheld scanner and POS display with mobile phone |
US9317114B2 (en) | 2013-05-07 | 2016-04-19 | Korea Advanced Institute Of Science And Technology | Display property determination |
US10152082B2 (en) | 2013-05-13 | 2018-12-11 | North Inc. | Systems, articles and methods for wearable electronic devices that accommodate different user forms |
KR20150010229A (en) | 2013-07-18 | 2015-01-28 | 삼성디스플레이 주식회사 | Eyeglasses including transparent display and controlling method thereof |
US11426123B2 (en) | 2013-08-16 | 2022-08-30 | Meta Platforms Technologies, Llc | Systems, articles and methods for signal routing in wearable electronic devices that detect muscle activity of a user using a set of discrete and separately enclosed pod structures |
US11921471B2 (en) | 2013-08-16 | 2024-03-05 | Meta Platforms Technologies, Llc | Systems, articles, and methods for wearable devices having secondary power sources in links of a band for providing secondary power in addition to a primary power source |
US10042422B2 (en) | 2013-11-12 | 2018-08-07 | Thalmic Labs Inc. | Systems, articles, and methods for capacitive electromyography sensors |
US20150124566A1 (en) | 2013-10-04 | 2015-05-07 | Thalmic Labs Inc. | Systems, articles and methods for wearable electronic devices employing contact sensors |
CN104423038B (en) * | 2013-08-19 | 2017-07-21 | 联想(北京)有限公司 | Electronic equipment and its focus information acquisition methods |
US9788789B2 (en) | 2013-08-30 | 2017-10-17 | Thalmic Labs Inc. | Systems, articles, and methods for stretchable printed circuit boards |
KR101500130B1 (en) * | 2013-09-02 | 2015-03-06 | 현대자동차주식회사 | Apparatus for Controlling Vehicle installation on Steering wheel |
US20150062164A1 (en) * | 2013-09-05 | 2015-03-05 | Seiko Epson Corporation | Head mounted display, method of controlling head mounted display, computer program, image display system, and information processing apparatus |
US10451874B2 (en) | 2013-09-25 | 2019-10-22 | Seiko Epson Corporation | Image display device, method of controlling image display device, computer program, and image display system |
US10405786B2 (en) | 2013-10-09 | 2019-09-10 | Nedim T. SAHIN | Systems, environment and methods for evaluation and management of autism spectrum disorder using a wearable data collection device |
US9936916B2 (en) | 2013-10-09 | 2018-04-10 | Nedim T. SAHIN | Systems, environment and methods for identification and analysis of recurring transitory physiological states and events using a portable data collection device |
CN103530038A (en) * | 2013-10-23 | 2014-01-22 | 叶晨光 | Program control method and device for head-mounted intelligent terminal |
US9672649B2 (en) * | 2013-11-04 | 2017-06-06 | At&T Intellectual Property I, Lp | System and method for enabling mirror video chat using a wearable display device |
WO2015081113A1 (en) | 2013-11-27 | 2015-06-04 | Cezar Morun | Systems, articles, and methods for electromyography sensors |
WO2015099683A1 (en) * | 2013-12-23 | 2015-07-02 | Empire Technology Development, Llc | Suppression of real features in see-through display |
US10067341B1 (en) | 2014-02-04 | 2018-09-04 | Intelligent Technologies International, Inc. | Enhanced heads-up display system |
JP2017509386A (en) | 2014-02-14 | 2017-04-06 | サルミック ラブス インコーポレイテッド | System, product and method for elastic electrical cable and wearable electronic device using the same |
AU2015218578B2 (en) | 2014-02-24 | 2020-06-25 | Nedim T. Sahin | Systems, environment and methods for evaluation and management of autism spectrum disorder using a wearable data collection device |
KR20150110032A (en) * | 2014-03-24 | 2015-10-02 | 삼성전자주식회사 | Electronic Apparatus and Method for Image Data Processing |
US10199008B2 (en) | 2014-03-27 | 2019-02-05 | North Inc. | Systems, devices, and methods for wearable electronic devices as state machines |
KR102212030B1 (en) * | 2014-05-26 | 2021-02-04 | 엘지전자 주식회사 | Glass type terminal and control method thereof |
US9880632B2 (en) | 2014-06-19 | 2018-01-30 | Thalmic Labs Inc. | Systems, devices, and methods for gesture identification |
US20170322421A1 (en) * | 2014-06-19 | 2017-11-09 | M-I Drilling Fluids Uk Limited | Intrinsically safe safety goggles with data display, user input and data communication |
CN104111658B (en) * | 2014-07-17 | 2016-09-14 | 金陵科技学院 | A kind of unmanned plane being monitored by intelligent glasses shooting and controlling |
WO2016018044A1 (en) * | 2014-07-31 | 2016-02-04 | Samsung Electronics Co., Ltd. | Wearable device and method of controlling the same |
US10540907B2 (en) | 2014-07-31 | 2020-01-21 | Intelligent Technologies International, Inc. | Biometric identification headpiece system for test taking |
US10410535B2 (en) | 2014-08-22 | 2019-09-10 | Intelligent Technologies International, Inc. | Secure testing device |
CN106662746B (en) | 2014-08-22 | 2020-10-23 | 国际智能技术公司 | Secure examination device, system and method |
CN105468620A (en) * | 2014-09-03 | 2016-04-06 | 雅虎公司 | Multimedia interface imaging method and system and wearable equipment |
US20160091996A1 (en) * | 2014-09-30 | 2016-03-31 | Apple Inc. | Liftoff input detection |
US9799301B2 (en) | 2014-10-09 | 2017-10-24 | Nedim T. SAHIN | Method, system, and apparatus for battery life extension and peripheral expansion of a wearable data collection device |
JP2017534132A (en) | 2014-10-10 | 2017-11-16 | ミュージック エルエルシー | Device for sharing user interaction |
US10133900B2 (en) * | 2014-10-30 | 2018-11-20 | Philips Lighting Holding B.V. | Controlling the output of contextual information using a computing device |
US10438106B2 (en) | 2014-11-04 | 2019-10-08 | Intellignet Technologies International, Inc. | Smartcard |
US9535497B2 (en) * | 2014-11-20 | 2017-01-03 | Lenovo (Singapore) Pte. Ltd. | Presentation of data on an at least partially transparent display based on user focus |
US9807221B2 (en) | 2014-11-28 | 2017-10-31 | Thalmic Labs Inc. | Systems, devices, and methods effected in response to establishing and/or terminating a physical communications link |
US20160216792A1 (en) * | 2015-01-26 | 2016-07-28 | Seiko Epson Corporation | Head mounted display, and control method and control program for head mounted display |
US10409072B2 (en) * | 2015-03-09 | 2019-09-10 | Sony Corporation | Wearable display, casing for a wearable display, and manufacturing method for a wearable display |
CN104765156B (en) * | 2015-04-22 | 2017-11-21 | 京东方科技集团股份有限公司 | A kind of three-dimensional display apparatus and 3 D displaying method |
US10078435B2 (en) | 2015-04-24 | 2018-09-18 | Thalmic Labs Inc. | Systems, methods, and computer program products for interacting with electronically displayed presentation materials |
US10197816B2 (en) * | 2015-05-26 | 2019-02-05 | Lumenis Ltd. | Laser safety glasses with an improved imaging system |
US10289239B2 (en) | 2015-07-09 | 2019-05-14 | Microsoft Technology Licensing, Llc | Application programming interface for multi-touch input detection |
US9298283B1 (en) | 2015-09-10 | 2016-03-29 | Connectivity Labs Inc. | Sedentary virtual reality method and systems |
US10289205B1 (en) * | 2015-11-24 | 2019-05-14 | Google Llc | Behind the ear gesture control for a head mountable device |
US10678958B2 (en) | 2015-12-28 | 2020-06-09 | Intelligent Technologies International, Inc. | Intrusion-protected memory component |
JP2016173845A (en) * | 2016-05-16 | 2016-09-29 | ソニー株式会社 | Information processing apparatus, display device, information processing method, and program |
US11216069B2 (en) | 2018-05-08 | 2022-01-04 | Facebook Technologies, Llc | Systems and methods for improved speech recognition using neuromuscular information |
US10990174B2 (en) | 2016-07-25 | 2021-04-27 | Facebook Technologies, Llc | Methods and apparatus for predicting musculo-skeletal position information using wearable autonomous sensors |
CN110291446B (en) * | 2017-02-22 | 2021-10-08 | 三井化学株式会社 | Eyewear |
WO2018168644A1 (en) * | 2017-03-13 | 2018-09-20 | 三井化学株式会社 | Eyewear |
TWM557588U (en) * | 2017-08-01 | 2018-04-01 | 李燕琴 | Wearable device capable of thermal imaging |
CN111033359B (en) | 2017-09-25 | 2022-01-14 | 三井化学株式会社 | Picture frame part |
CN112040858A (en) | 2017-10-19 | 2020-12-04 | 脸谱科技有限责任公司 | System and method for identifying biological structures associated with neuromuscular source signals |
US10599259B2 (en) * | 2017-11-20 | 2020-03-24 | Google Llc | Virtual reality / augmented reality handheld controller sensing |
US10634913B2 (en) * | 2018-01-22 | 2020-04-28 | Symbol Technologies, Llc | Systems and methods for task-based adjustable focal distance for heads-up displays |
US11567573B2 (en) | 2018-09-20 | 2023-01-31 | Meta Platforms Technologies, Llc | Neuromuscular text entry, writing and drawing in augmented reality systems |
US11150730B1 (en) | 2019-04-30 | 2021-10-19 | Facebook Technologies, Llc | Devices, systems, and methods for controlling computing devices via neuromuscular signals of users |
US11961494B1 (en) | 2019-03-29 | 2024-04-16 | Meta Platforms Technologies, Llc | Electromagnetic interference reduction in extended reality environments |
US11493993B2 (en) | 2019-09-04 | 2022-11-08 | Meta Platforms Technologies, Llc | Systems, methods, and interfaces for performing inputs based on neuromuscular control |
US11481030B2 (en) | 2019-03-29 | 2022-10-25 | Meta Platforms Technologies, Llc | Methods and apparatus for gesture detection and classification |
US10937414B2 (en) | 2018-05-08 | 2021-03-02 | Facebook Technologies, Llc | Systems and methods for text input using neuromuscular information |
US11907423B2 (en) | 2019-11-25 | 2024-02-20 | Meta Platforms Technologies, Llc | Systems and methods for contextualized interactions with an environment |
JP6995650B2 (en) * | 2018-01-30 | 2022-01-14 | Dynabook株式会社 | Wearable devices and their operation methods |
DE102018204070A1 (en) | 2018-03-16 | 2019-09-19 | Carl Zeiss Ag | Head-worn visual output device |
US10592001B2 (en) | 2018-05-08 | 2020-03-17 | Facebook Technologies, Llc | Systems and methods for improved speech recognition using neuromuscular information |
KR102184243B1 (en) | 2018-07-06 | 2020-11-30 | 한국과학기술연구원 | System for controlling interface based on finger gestures using imu sensor |
CN112996430A (en) | 2018-08-31 | 2021-06-18 | 脸谱科技有限责任公司 | Camera-guided interpretation of neuromuscular signals |
US11055056B1 (en) | 2018-09-25 | 2021-07-06 | Facebook Technologies, Llc | Split system for artificial reality |
EP3886693A4 (en) | 2018-11-27 | 2022-06-08 | Facebook Technologies, LLC. | Methods and apparatus for autocalibration of a wearable electrode sensor system |
US11733789B1 (en) * | 2019-09-30 | 2023-08-22 | Snap Inc. | Selectively activating a handheld device to control a user interface displayed by a wearable device |
US11868531B1 (en) | 2021-04-08 | 2024-01-09 | Meta Platforms Technologies, Llc | Wearable device providing for thumb-to-finger-based input gestures detected based on neuromuscular signals, and systems and methods of use thereof |
US11934852B1 (en) * | 2022-11-30 | 2024-03-19 | Trimble Solutions Corporation | Providing help content proactively |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060132382A1 (en) * | 2004-12-22 | 2006-06-22 | Jannard James H | Data input management system for wearable electronically enabled interface |
US20060198978A1 (en) * | 2005-02-08 | 2006-09-07 | Antonini Fred A | Elastomeric film |
US20100277563A1 (en) * | 2009-04-29 | 2010-11-04 | Embarq Holdings Company, Llc | Video conferencing eyewear |
Family Cites Families (27)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH05300544A (en) | 1992-04-23 | 1993-11-12 | Sony Corp | Video display device |
WO1994009398A1 (en) | 1992-10-20 | 1994-04-28 | Alec Robinson | Eye screen means with mounted visual display and communication apparatus |
GB2299394A (en) * | 1995-03-31 | 1996-10-02 | Frazer Concepts Ltd | Computer input devices |
JPH10123450A (en) | 1996-10-15 | 1998-05-15 | Sony Corp | Head up display device with sound recognizing function |
EP1027627B1 (en) | 1997-10-30 | 2009-02-11 | MYVU Corporation | Eyeglass interface system |
WO1999042800A1 (en) * | 1998-02-20 | 1999-08-26 | Massachusetts Institute Of Technology | Finger touch sensors and virtual switch panels |
US6424338B1 (en) * | 1999-09-30 | 2002-07-23 | Gateway, Inc. | Speed zone touchpad |
US7958457B1 (en) | 1999-12-20 | 2011-06-07 | Wireless Agents, Llc | Method and apparatus for scheduling presentation of digital content on a personal communication device |
US6925611B2 (en) | 2001-01-31 | 2005-08-02 | Microsoft Corporation | Navigational interface for mobile and wearable computers |
JP2003067128A (en) * | 2001-08-30 | 2003-03-07 | Sharp Corp | Information processor with pad type pointing device |
US6970157B2 (en) | 2002-04-23 | 2005-11-29 | Quadtri Technologies, Llc | Wearable computing, input, and display device |
US7337410B2 (en) * | 2002-11-06 | 2008-02-26 | Julius Lin | Virtual workstation |
WO2004061519A1 (en) | 2002-12-24 | 2004-07-22 | Nikon Corporation | Head mount display |
JP2005072867A (en) * | 2003-08-22 | 2005-03-17 | Matsushita Electric Ind Co Ltd | Head-attached display device |
US20060007056A1 (en) * | 2004-07-09 | 2006-01-12 | Shu-Fong Ou | Head mounted display system having virtual keyboard and capable of adjusting focus of display screen and device installed the same |
KR100594117B1 (en) | 2004-09-20 | 2006-06-28 | 삼성전자주식회사 | Apparatus and method for inputting key using biosignal in HMD information terminal |
US7719521B2 (en) | 2005-08-19 | 2010-05-18 | Microsoft Corporation | Navigational interface providing auxiliary character support for mobile and wearable computers |
JP4961984B2 (en) * | 2006-12-07 | 2012-06-27 | ソニー株式会社 | Image display system, display device, and display method |
US7855718B2 (en) * | 2007-01-03 | 2010-12-21 | Apple Inc. | Multi-touch input discrimination |
DE102007016138A1 (en) | 2007-03-29 | 2008-10-09 | Carl Zeiss Ag | HMD device |
CN201163312Y (en) * | 2008-01-14 | 2008-12-10 | 郭强 | Spectacles display type electronic reading device |
US20100149073A1 (en) | 2008-11-02 | 2010-06-17 | David Chaum | Near to Eye Display System and Appliance |
CN201255787Y (en) * | 2008-09-12 | 2009-06-10 | 天津三星电子有限公司 | Electronic spectacle |
US8957835B2 (en) | 2008-09-30 | 2015-02-17 | Apple Inc. | Head-mounted display apparatus for retaining a portable electronic device with display |
CN103119512A (en) * | 2008-11-02 | 2013-05-22 | 大卫·乔姆 | Near to eye display system and appliance |
KR101578721B1 (en) * | 2008-12-15 | 2015-12-21 | 엘지전자 주식회사 | Navigation device and method for guiding route thereof |
JP5195708B2 (en) * | 2009-09-30 | 2013-05-15 | ブラザー工業株式会社 | Head mounted display |
-
2011
- 2011-05-25 US US13/115,430 patent/US8203502B1/en active Active
-
2012
- 2012-05-22 EP EP12789107.5A patent/EP2715432A4/en not_active Withdrawn
- 2012-05-22 WO PCT/US2012/038968 patent/WO2012162304A2/en active Application Filing
- 2012-05-22 US US13/477,547 patent/US20120299870A1/en not_active Abandoned
- 2012-05-22 KR KR1020137034402A patent/KR101473143B1/en active IP Right Grant
- 2012-05-22 CN CN201280036750.3A patent/CN103718082B/en active Active
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060132382A1 (en) * | 2004-12-22 | 2006-06-22 | Jannard James H | Data input management system for wearable electronically enabled interface |
US20060198978A1 (en) * | 2005-02-08 | 2006-09-07 | Antonini Fred A | Elastomeric film |
US20100277563A1 (en) * | 2009-04-29 | 2010-11-04 | Embarq Holdings Company, Llc | Video conferencing eyewear |
Cited By (51)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9013264B2 (en) | 2011-03-12 | 2015-04-21 | Perceptive Devices, Llc | Multipurpose controller for electronic devices, facial expressions management and drowsiness detection |
US20160320919A1 (en) * | 2012-05-17 | 2016-11-03 | Hong Kong Applied Science and Technology Research Institute Company Limited | Wearable Device with Intelligent User-Input Interface |
US9857919B2 (en) * | 2012-05-17 | 2018-01-02 | Hong Kong Applied Science And Technology Research | Wearable device with intelligent user-input interface |
US9268424B2 (en) * | 2012-07-18 | 2016-02-23 | Sony Corporation | Mobile client device, operation method, recording medium, and operation system |
US20140022192A1 (en) * | 2012-07-18 | 2014-01-23 | Sony Mobile Communications, Inc. | Mobile client device, operation method, recording medium, and operation system |
US9542096B2 (en) | 2012-07-18 | 2017-01-10 | Sony Corporation | Mobile client device, operation method, recording medium, and operation system |
US10007424B2 (en) | 2012-07-18 | 2018-06-26 | Sony Mobile Communications Inc. | Mobile client device, operation method, recording medium, and operation system |
CN105210145A (en) * | 2013-01-04 | 2015-12-30 | 伊奎蒂公司 | Interactive wearable and portable smart devices |
WO2014107629A1 (en) * | 2013-01-04 | 2014-07-10 | Vuzix Corporation | Interactive wearable and portable smart devices |
US10176783B2 (en) | 2013-01-04 | 2019-01-08 | Vuzix Corporation | Interactive wearable and portable smart devices |
US9632318B2 (en) * | 2013-01-23 | 2017-04-25 | Sony Corporation | Head-mounted display including an operating element having a longitudinal direction in a direction of a first axis, display apparatus, and input apparatus |
US20140204062A1 (en) * | 2013-01-23 | 2014-07-24 | Sony Corporation | Head-mounted display, display apparatus, and input apparatus |
CN103941397A (en) * | 2013-01-23 | 2014-07-23 | 索尼公司 | Head-mounted Display, Display Apparatus, And Input Apparatus |
US20150370335A1 (en) * | 2013-02-22 | 2015-12-24 | Sony Corporation | Display control apparatus, display apparatus, display control method, and program |
US10596444B2 (en) | 2013-03-18 | 2020-03-24 | Fb-Mm Ltd. | Sports match refereeing system |
US10967240B2 (en) | 2013-03-18 | 2021-04-06 | Fb-Mm Ltd. | Sports match refereeing system |
WO2014147455A1 (en) * | 2013-03-18 | 2014-09-25 | Minkovitch Zvi | Sports match refereeing system |
US9889367B2 (en) | 2013-03-18 | 2018-02-13 | Zvi Minkovitch | Sports match refereeing system |
US10218884B2 (en) | 2013-03-22 | 2019-02-26 | Seiko Epson Corporation | Infrared video display eyewear |
US9729767B2 (en) | 2013-03-22 | 2017-08-08 | Seiko Epson Corporation | Infrared video display eyewear |
DE102013206173A1 (en) * | 2013-04-09 | 2014-10-09 | Bayerische Motoren Werke Aktiengesellschaft | Selection of individual elements for display on data glasses |
WO2014171568A2 (en) * | 2013-04-17 | 2014-10-23 | 한국과학기술원 | Method and apparatus of detecting touch using sound, and device using same |
WO2014171568A3 (en) * | 2013-04-17 | 2015-05-07 | 한국과학기술원 | Method and apparatus of detecting touch using sound, and device using same |
US9250442B2 (en) | 2013-07-10 | 2016-02-02 | Lg Electronics Inc. | Head mounted display device including multiple user interface formats and method of controlling therefor |
WO2015005525A1 (en) * | 2013-07-10 | 2015-01-15 | Lg Electronics Inc. | Head mounted display device including multiple user interface formats and method of controlling therefor |
US8823603B1 (en) | 2013-07-26 | 2014-09-02 | Lg Electronics Inc. | Head mounted display and method of controlling therefor |
US9874748B2 (en) * | 2013-11-12 | 2018-01-23 | Lg Electronics Inc. | Glass type terminal |
US20160274357A1 (en) * | 2013-11-12 | 2016-09-22 | Lg Electronics Inc. | Glass type terminal |
CN103604439A (en) * | 2013-11-19 | 2014-02-26 | 江门市得实计算机外部设备有限公司 | Pedometer capable of realizing function switching by clicking, colliding or vibrating |
WO2015077004A1 (en) * | 2013-11-20 | 2015-05-28 | Google Inc. | Systems and methods for performing multi-touch operations on a head-mountable device |
US9261700B2 (en) | 2013-11-20 | 2016-02-16 | Google Inc. | Systems and methods for performing multi-touch operations on a head-mountable device |
US9804682B2 (en) | 2013-11-20 | 2017-10-31 | Google Inc. | Systems and methods for performing multi-touch operations on a head-mountable device |
CN105745568A (en) * | 2013-11-20 | 2016-07-06 | 谷歌公司 | Systems and methods for performing multi-touch operations on a head-mountable device |
EP2889718A1 (en) | 2013-12-30 | 2015-07-01 | Samsung Electronics Co., Ltd | A natural input based virtual ui system for electronic devices |
US20150186033A1 (en) * | 2013-12-31 | 2015-07-02 | Korea Electronics Technology Institute | Glasses type device with operating means on rim and user input method using the same |
US9599823B2 (en) | 2014-01-15 | 2017-03-21 | Lg Electronics Inc. | Detachable head mount display device and method for controlling the same |
EP3095005A4 (en) * | 2014-01-15 | 2017-08-09 | LG Electronics Inc. | Detachable head mount display device and method for controlling the same |
US9348142B2 (en) | 2014-01-15 | 2016-05-24 | Lg Electronics Inc. | Detachable head mount display device and method for controlling the same |
WO2015108234A1 (en) * | 2014-01-15 | 2015-07-23 | Lg Electronics Inc. | Detachable head mount display device and method for controlling the same |
US10067415B2 (en) | 2014-03-19 | 2018-09-04 | Samsung Electronics Co., Ltd. | Method for displaying image using projector and wearable electronic device for implementing the same |
GB2540098A (en) * | 2014-06-13 | 2017-01-04 | Halliburton Energy Services Inc | Monitoring hydrocarbon recovery operations using wearable computer machines |
WO2015191079A1 (en) * | 2014-06-13 | 2015-12-17 | Halliburton Energy Services, Inc. | Monitoring hydrocarbon recovery operations using wearable computer machines |
US10378318B2 (en) | 2014-06-13 | 2019-08-13 | Halliburton Energy Services, Inc. | Monitoring hydrocarbon recovery operations using wearable computer machines |
WO2016027932A1 (en) * | 2014-08-21 | 2016-02-25 | Lg Electronics Inc. | Glass-type mobile terminal and control method thereof |
US10331340B2 (en) | 2014-12-04 | 2019-06-25 | Samsung Electronics Co., Ltd. | Device and method for receiving character input through the same |
US20180271383A1 (en) * | 2014-12-18 | 2018-09-27 | Lg Innotek Co., Ltd. | Pulse Measurement Device and Computing Device Using Same |
US10750962B2 (en) * | 2014-12-18 | 2020-08-25 | Lg Innotek Co., Ltd. | Pulse measurement device and computing device using same |
DE102015204780A1 (en) | 2015-03-17 | 2016-09-22 | Roschi Rohde & Schwarz AG | Portable visual display device for an electrical or electronic measuring device and measuring station arrangement |
DE102015206250A1 (en) * | 2015-04-08 | 2016-10-13 | Bayerische Motoren Werke Aktiengesellschaft | Operator inputs for systems including touch pads and displays |
US20210263696A1 (en) * | 2015-04-10 | 2021-08-26 | Andy Bevilacqua | Cognitive Improvement System and Method of Testing |
US11520557B2 (en) * | 2015-04-10 | 2022-12-06 | Advanced Cognitive Concepts | Cognitive improvement system and method of testing |
Also Published As
Publication number | Publication date |
---|---|
KR20140008467A (en) | 2014-01-21 |
CN103718082A (en) | 2014-04-09 |
CN103718082B (en) | 2015-05-13 |
EP2715432A2 (en) | 2014-04-09 |
WO2012162304A2 (en) | 2012-11-29 |
WO2012162304A3 (en) | 2013-04-04 |
EP2715432A4 (en) | 2015-06-03 |
KR101473143B1 (en) | 2014-12-15 |
US8203502B1 (en) | 2012-06-19 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US8203502B1 (en) | Wearable heads-up display with integrated finger-tracking input sensor | |
US9830071B1 (en) | Text-entry for a computing device | |
US9024843B2 (en) | Wearable computer with curved display and navigation tool | |
US20130002724A1 (en) | Wearable computer with curved display and navigation tool | |
US20210181536A1 (en) | Eyewear device with finger activated touch sensor | |
US20220004255A1 (en) | Visibility improvement method based on eye tracking, machine-readable storage medium and electronic device | |
US10114466B2 (en) | Methods and systems for hands-free browsing in a wearable computing device | |
US8643951B1 (en) | Graphical menu and interaction therewith through a viewing window | |
US9223401B1 (en) | User interface | |
US10133407B2 (en) | Display apparatus, display system, method for controlling display apparatus, and program | |
WO2022005687A1 (en) | Augmented reality experiences with object manipulation | |
US9454288B2 (en) | One-dimensional to two-dimensional list navigation | |
US9448687B1 (en) | Zoomable/translatable browser interface for a head mounted device | |
US20140258902A1 (en) | Graphical Interface Having Adjustable Borders | |
US20130117707A1 (en) | Velocity-Based Triggering | |
CN116324581A (en) | Goggles comprising a virtual scene with 3D frames | |
KR20160067622A (en) | Device and method for receiving character input through the same | |
US20190187479A1 (en) | Transmission-type head mounted display apparatus, display control method, and computer program | |
US20240129617A1 (en) | Image capture eyewear with context-based sending | |
US9857965B1 (en) | Resolution of directional ambiguity on touch-based interface gesture | |
JP2018081415A (en) | Input device, input method and computer program |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |