Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
  • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
  • G06V40/18—Eye characteristics, e.g. of the iris
  • G06V40/19—Sensors therefor
• • G—PHYSICS
  • G02—OPTICS
  • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
  • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
  • G02B27/01—Head-up displays
  • G02B27/017—Head mounted
• • G—PHYSICS
  • G02—OPTICS
  • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
  • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
  • G02B27/01—Head-up displays
  • G02B27/017—Head mounted
  • G02B27/0172—Head mounted characterised by optical features
• • G—PHYSICS
  • G02—OPTICS
  • G02C—SPECTACLES; SUNGLASSES OR GOGGLES INSOFAR AS THEY HAVE THE SAME FEATURES AS SPECTACLES; CONTACT LENSES
  • G02C11/00—Non-optical adjuncts; Attachment thereof
  • G02C11/10—Electronic devices other than hearing aids
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
  • G06F1/16—Constructional details or arrangements
  • G06F1/1613—Constructional details or arrangements for portable computers
  • G06F1/1633—Constructional details or arrangements of portable computers not specific to the type of enclosures covered by groups G06F1/1615 - G06F1/1626
  • G06F1/1662—Details related to the integrated keyboard
  • G06F1/1673—Arrangements for projecting a virtual keyboard
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
  • G06F3/012—Head tracking input arrangements
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
  • G06F3/013—Eye tracking input arrangements
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
  • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
  • G06Q30/00—Commerce
  • G06Q30/02—Marketing; Price estimation or determination; Fundraising
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
  • G06Q30/00—Commerce
  • G06Q30/02—Marketing; Price estimation or determination; Fundraising
  • G06Q30/0241—Advertisements
  • G06Q30/0251—Targeted advertisements
  • G06Q30/0261—Targeted advertisements based on user location
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
  • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
  • G06V40/18—Eye characteristics, e.g. of the iris
  • G06V40/197—Matching; Classification
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
  • H04N23/50—Constructional details
  • H04N23/55—Optical parts specially adapted for electronic image sensors; Mounting thereof
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
  • H04N23/60—Control of cameras or camera modules
  • H04N23/63—Control of cameras or camera modules by using electronic viewfinders
  • H04N23/633—Control of cameras or camera modules by using electronic viewfinders for displaying additional information relating to control or operation of the camera
  • H04N23/635—Region indicators; Field of view indicators
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
  • H04N23/60—Control of cameras or camera modules
  • H04N23/66—Remote control of cameras or camera parts, e.g. by remote control devices
  • H04N23/661—Transmitting camera control signals through networks, e.g. control via the Internet
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N5/00—Details of television systems
  • H04N5/44—Receiver circuitry for the reception of television signals according to analogue transmission standards
• • G—PHYSICS
  • G02—OPTICS
  • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
  • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
  • G02B27/01—Head-up displays
  • G02B27/0101—Head-up displays characterised by optical features
  • G02B2027/0138—Head-up displays characterised by optical features comprising image capture systems, e.g. camera
• • G—PHYSICS
  • G02—OPTICS
  • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
  • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
  • G02B27/01—Head-up displays
  • G02B27/017—Head mounted
  • G02B2027/0178—Eyeglass type
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
  • G06F3/014—Hand-worn input/output arrangements, e.g. data gloves

Definitions

• the present disclosure provides a method of collecting biometric information from an individual.
• the method comprises positioning a body part of the individual in front of a sensor.
• the sensor may be a flat plate type sensor for collecting fingerprints and palm prints, or may be an optical device for collecting an iris print.
• a system may comprise an interactive head-mounted eyepiece worn by a user, wherein the eyepiece includes an optical assembly through which the user views a surrounding environment and displayed content, wherein the optical assembly comprises a corrective element that corrects the user's view of the surrounding environment, an integrated processor for handling content for display to the user, and an integrated image source for introducing the content to the optical assembly, wherein the displayed content comprises an interactive control element; and an integrated camera facility that images the surrounding environment, and identifies a user hand gesture as an interactive control element location command, wherein the location of the interactive control element remains fixed with respect to an object in the surrounding environment, in response to the interactive control element location command, regardless of a change in the viewing direction of the user.
• a system may comprise an interactive head-mounted eyepiece worn by a user, wherein the eyepiece includes an optical assembly through which the user views a surrounding environment and displayed content, wherein the optical assembly comprises a corrective element that corrects the user's view of the surrounding environment, an integrated processor for handling content for display to the user, and an integrated image source for introducing the content to the optical assembly; and an integrated camera facility that images a gesture, wherein the integrated processor identifies and interprets the gesture as a command instruction.
• the control instruction may provide manipulation of the content for display, a command communicated to an external device, and the like.
• the local advertisement may be displayed to the user as a banner advertisement, two dimensional graphic, or text. Further, advertisement may be associated with a physical aspect of the surrounding environment. In yet other embodiments, the advertisement may be displayed as an augmented reality associated with a physical aspect of the surrounding environment. The augmented reality
• Another embodiment is a method for stabilizing images.
• the method includes steps of providing an interactive head-mounted eyepiece including a camera and an optical assembly through which a user views a surrounding environment and displayed content, the assembly also comprising a processor for handling content for display to the user and an integrated projector for projecting the content to the optical assembly, and imaging the surrounding environment with the camera to capture an image of an object in the surrounding environment.
• the method also includes steps of displaying, through the optical assembly, the content at a fixed location with respect to the user's view of the imaged object, sensing vibration and movement of the eyepiece, and stabilizing the displayed content with respect to the user's view of the surrounding environment via at least one digital technique.
• a method of gaming includes contacting an online gaming site using the eyepiece, initiating or joining a game of the online gaming site with a plurality of members of the online gaming site, each member using an interactive head- mounted eyepiece system, viewing game content with the optical assembly, and playing the game by manipulating at least one sensor for detecting motion.
• a method of using augmented vision includes providing the eyepiece, controlling the camera through movements, gestures or commands of the user, sending information from the camera to a processor of the interactive head-mounted eyepiece, and viewing visual images using the optical assembly, wherein visual images from the camera and optical assembly are an improvement for the user in at least one of focus, brightness, clarity and magnification.
• an apparatus for collecting biometric information includes a flat plate containing a mosaic sensor, wherein the mosaic sensor has multiple light sources positioned around the perimeter of the flat plate and cameras disposed perpendicular to the flat plate, a keyboard, straps for mounting to a user's forearm, a geo- location module for ascertaining position location, a communications module for wireless interfacing with other communication devices, and a clock for time stamping collected biometric information.
• Fig. 13a depicts an incumbent night vision system
• Fig. 13b depicts the night vision system of the present disclosure
• Fig. 13c illustrates the difference in responsivity between the two.
• Fig. 15B depicts a facial actuation sensor as mounted on the eyepiece.
• Fig. 19 depicts an augmented reality-enabled custom advertisement.
• Fig. 20 an augmented reality-enabled custom artwork.
• Fig. 20A depicts a method for posting messages to be transmitted when a viewer reaches a certain location.
• Fig. 43 shows the components of the pocket bio-kit according to an embodiment.
• Fig. 47 shows a mobile folding biometric enrollment kit according to an embodiment.
• the projector 200 may be an RGB projector.
• the projector 200 may include a housing 202, a heatsink 204 and an RGB LED engine or module 206.
• the RGB LED engine 206 may include LEDs, dichroics, concentrators, and the like.
• a digital signal processor (DSP) (not shown) may convert the images or video stream into control signals, such as voltage drops / current modifications, pulse width modulation (PWM) signals, and the like to control the intensity, duration, and mixing of the LED light.
• PWM pulse width modulation
• the DSP may control the duty cycle of each PWM signal to control the average current flowing through each LED generating a plurality of colors.
• a still image co-processor of the eyepiece may employ noise-filtering, image / video stabilization, and face detection, and be able to make image enhancements.
• An audio back-end processor of the eyepiece may employ buffering, SRC, equalization and the like.
• the digital signal processor may be programmed and/or configured to receive video feed information and configure the video feed to drive whatever type of image source is being used with the optical display 210.
• the DSP may include a bus or other communication mechanism for communicating information, and an internal processor coupled with the bus for processing the information.
• the DSP may include a memory, such as a random access memory (RAM) or other dynamic storage device (e.g., dynamic RAM (DRAM), static RAM (SRAM), and synchronous DRAM (SDRAM)), coupled to the bus for storing information and instructions to be executed.
• RAM random access memory
• DRAM dynamic RAM
• SRAM static RAM
• SDRAM synchronous DRAM
• the interactive head-mounted eyepiece includes an optical assembly through which a user views a surrounding environment and displayed content, wherein the optical assembly includes a corrective element that corrects the user's view of the surrounding environment, a freeform optical waveguide enabling internal reflections, and a coupling lens positioned to direct an image from an optical display, such as an LCoS display, to the optical waveguide.
• the eyepiece further includes an integrated processor for handling content for display to the user and an integrated image source, such as a projector facility, for introducing the content to the optical assembly.
• the image source is a projector
• the projector facility includes a light source and the optical display.
• the image light which may be polarized and collimated, may optionally traverse a display coupling lens 412, which may or may not be the collimator itself or in addition to the collimator, and enter the waveguide 414.
• the waveguide 414 may be a freeform waveguide, where the surfaces of the waveguide are described by a polynomial equation.
• the waveguide may be rectilinear.
• the waveguide 414 may include two reflective surfaces. When the image light enters the waveguide 414, it may strike a first surface with an angle of incidence greater than the critical angle above which total internal reflection (TIR) occurs.
• TIR total internal reflection
• the distance from the pupil to the image light path as it enters the waveguide 502 may be about 50 - 60 mm, which can accommodate a large % of human head breadths.
• the field of view may be larger than the pupil. In embodiments, the field of view may not fill the lens. It should be understood that these dimensions are for a particular illustrative embodiment and should not be construed as limiting.
• the waveguide, snap-on optics, and/or the corrective lens may comprise optical plastic.
• the waveguide snap-on optics, and/or the corrective lens may comprise glass, marginal glass, bulk glass, metallic glass, palladium- enriched glass, or other suitable glass.
• the waveguide 508 and correction lens 510 may be made from different materials selected to result in little to no chromatic aberrations. The materials may include a diffraction grating, a holographic grating, and the like.
• the video output may be streamed to the virtual screen seen by the user.
• the video output may thus be used to help determine the user's location, or may be sent remotely to others to assist in helping to locate the location of the wearer, or for any other purpose.
• Other detection technologies such as GPS, RFID, manual input, and the like, may be used to determine a wearer's location.
• a database may be accessed by the eyepiece for information that may be overlaid, projected, or otherwise displayed with what is being seen. Augmented reality applications and technology will be further described herein.
• replaceable lens 819 has a plurality of snap-fit arms 819a which fit into recesses 820a of frame 820.
• the engagement angle 819b of the arm is greater than 90°, while the engagement angle 820b of the recess is also greater than 90°. Making the angles greater than right angles has the practical effect of allowing removal of lens 819 from the frame 820.
• the lens 819 may need to be removed if the person's vision has changed or if a different lens is desired for any reason.
• the design of the snap fit is such that there is a slight compression or bearing load between the lens and the frame. That is, the lens may be held firmly within the frame, such as by a slight interference fit of the lens within the frame.
• the flip-up / flip-down arrangement enables a modular design for the eyepiece.
• the eyepiece be equipped with a monocular or binocular module 802, but the lens 818 may also be swapped.
• additional features may be included with the module 802, either associated with one or both displays 812.
• either monocular or binocular versions of the module 802 may be display only 902 (monocular), 904 (binocular) or may be equipped with a forward-looking camera 908 (monocular), and 910 & 912 (binocular).
• the module may have additional integrated electronics, such as a GPS, a laser range finder, and the like.
• a binocular electro-optic module 912 is equipped with stereo forward-looking cameras 920 and a laser range finder 918.
• These larger movements may include gross movements of the user, e.g., walking or running, riding in a vehicle.
• Smaller vibrations may also result within the augmented reality eyeglasses, that is, vibrations in the components in the electrical and mechanical linkages that form the path from the camera (input) to the image in the waveguide (output).
• These gross movements may be more important to correct or to account for, rather than, for instance, independent and small movements in the linkages of components downstream from the projector.
• Voltages may be applied to the electrodes in patterns for a short period of time and a check on the focus or astigmatism made. The check may be made, for instance, by an image sensor.
• sensors on the camera or in this case the lens may detect motion of the camera or lens. Motion sensors would include accelerometers, gyroscopes, angular velocity sensors or piezoelectric sensors mounted on the liquid lens or a portion of the optic train very near the liquid lens.
• a table such as a calibration table, is then constructed of voltages applied and the degree of correction or voltages needed for given levels of movement. More sophistication may also be added, for example, by using segmented electrodes in different portions of the liquid so that four voltages may be applied rather than two.
• Fig. 57C An example is depicted in Fig. 57C.
• Four electrodes 3409 are mounted within a liquid lens housing (not shown). Two electrodes are mounted in or near the non-conducting liquid and two are mounted in or near the conducting liquid. Each electrode is independent in terms of the possible voltage that may be applied.
• illumination modulation may be accomplished through filtering and or through magnification.
• filtering techniques may be employed that allow the intensity and or color of the light to be changed such that the optimal or desired illumination is achieved.
• the intensity of the illumination may be modulated by applying greater or less magnification to reach the desired illumination intensity.
• the target MPE dimensions for the system may be 24 mm x 12 mm x
• a black silicon image sensor may have over eight times the signal to noise ration found in costly indium-gallium arsenide image sensors under night sky conditions. Better resolution is also provided by this technology, offering much higher resolution than available using current technology for night vision.
• CMOS-based SWIR have been difficult to interpret, having good heat detection, but poor resolution.
• This problem is solved with a black image silicon SWIR sensor, which relies on much shorter wavelengths. SWIR is highly desirable for battlefield night vision glasses for these reasons.
• FIG. 12 illustrates the effectiveness of black silicon night vision technology, providing both before and after images of seeing through a) dust; b) fog, and c) smoke.
• the images in FIG. 12 demonstrate the performance of the new VIS/NIR/SWIR black silicon sensor.
• Eye control of feature and option selection may be controlled and activated by object recognition software loaded on the system processor.
• Object recognition software may enable augmented reality, combine the recognition output with querying a database, combine the recognition output with a computational tool to determine dependencies / likelihoods, and the like.
• tapping against a hard surface may be interpreted through sensing by accelerometers, force sensors, and the like.
• the position sense device may then transmit motions of the finger as command information, such as moving the finger in the air to move a cursor across the displayed or projected image, moving in quick motion to indicate a selection, and the like.
• the position sense device may send sensed command information directly to the eyepiece for command processing, or the command processing circuitry may be co-located with the position sense device, such as in this example, mounted on the finger as part of an assembly including the sensors of the position sense device.
• the wearer may have a plurality of position sense devices mounted on their body.
• a system may comprise an interactive head-mounted eyepiece worn by a user, wherein the eyepiece includes an optical assembly through which the user views a surrounding environment and displayed content, wherein the optical assembly comprises a corrective element that corrects the user's view of the surrounding environment, an integrated processor for handling content for display to the user, and an integrated image source for introducing the content to the optical assembly; and a tactile control interface mounted on the eyepiece that accepts control inputs from the user through at least one of a user touching the interface and the user being proximate to the interface.
• the control circuitry may include a number of radio frequency oscillators, such as where one oscillator operates at a fixed frequency and another controlled by the user's hand, where the distance from the hand varies the input at the control antenna.
• the user's hand acts as a grounded plate (the user's body being the connection to ground) of a variable capacitor in an L-C (inductance-capacitance) circuit, which is part of the oscillator and determines its frequency.
• the circuit may use a single oscillator, two pairs of heterodyne oscillators, and the like.
• the remote control may be mounted on a weapon for a soldier, such as mounted on a pistol grip, on a muzzle shroud, on a fore grip, and the like, providing remote control to the soldier without the need to remove their hands from the weapon.
• the remote control may be removably mounted to the eyepiece.
• the system is able to determine the location of the house 1508C and provide location information 1514C and a 3D map superimposed onto the user's view of the environment.
• the information associated with an environmental feature may be provided by an external facility, such as communicated with through a wireless communication connection, stored internal to the eyepiece, such as downloaded to the eyepiece for the current location, and the like.
• the perspective of the viewer may be changed to put the viewer 'into' the movie environment with some control of the viewing perspective, where the user may be able to move their head around and have the view change in correspondence to the changed head position, where the user may be able to 'walk into' the image when they physically walk forward, have the perspective change as the user moves the gazing view of their eyes, and the like.
• additional image information may be provided, such as at the sides of the user's view that could be accessed by turning the head.
• the eyepiece may have a plurality of modes of operation where control of the eyepiece is controlled at least in part by positions, shapes, motions of the hand, and the like.
• the eyepiece may utilize hand recognition algorithms to detect the shape of the hand/fingers, and to then associate those hand configurations, possibly in combination with motions of the hand, as commands.
• hand recognition algorithms to detect the shape of the hand/fingers, and to then associate those hand configurations, possibly in combination with motions of the hand, as commands.
• these hand configurations may need to be reused depending upon the mode of operation of the eyepiece.
• eye control may enable the viewer to focus on a certain point on the displayed image from the eyepiece, and because the camera may be able to correlate the viewing direction of the eye to a point on the display, the eyepiece may be able to interpret commands through a combination of where the wearer is looking and an action by the wearer (e.g. blinking, touching an interface device, movement of a position sense device, and the like). For example, the viewer may be able to look at an object on the display, and select that object though the motion of a finger enabled through a position sense device.
• commands through a combination of where the wearer is looking and an action by the wearer (e.g. blinking, touching an interface device, movement of a position sense device, and the like).
• the viewer may be able to look at an object on the display, and select that object though the motion of a finger enabled through a position sense device.
• the input path analyzer determines the characters contacted in the input path, the word matching facility finds a best word match to the sequence of characters contacted and inputs the best word match as input text.
• the hand gesture may be a thumb and index finger gesture 6402 from a first user hand, and the virtual keyboard 6404 projected on the palm of the first user hand, and where the user is able to type on the virtual keyboard with a second user hand.
• the hand gesture 6410 may be a thumb and index finger gesture combination of both user hands, and the virtual keyboard 6408 projected between the user hands as configured in the hand gesture, where the user is able to type on the virtual keyboard using the thumbs of the user's hands.
• Visual cues may provide the wearer of the eyepiece with an automated resource for associating a predetermined external visual cue with a desired outcome in the way of projected content, thus freeing the wearer from searching for the cues themselves.
• a user of the augmented reality glasses may wish to participate in video calling or video conferencing while wearing the glasses.
• Many computers, both desktop and laptop have integrated cameras to facilitate using video calling and conferencing.
• software applications are used to integrate use of the camera with calling or conferencing features.
• the augmented reality glasses providing much of the functionality of laptops and other computing devices, many users may wish to utilize video calling and video conferencing while on the move wearing the augmented reality glasses.
• FIG. 58 depicts a typical camera 5800 for use in video calling or conferencing.
• Such cameras are typically small and could be mounted on a watch 5802, as shown in FIG. 58, cell phone or other portable computing device, including a laptop computer.
• Video calling works by connecting the device controller with the cell phone or other communications device.
• the devices utilize software compatible with the operating system of the glasses and the communications device or computing device.
• the screen of the augmented reality glasses may display a list of options for making the call and the user may gesture using a pointing control device or use any other control technique described herein to select the video calling option on the screen of the augmented reality glasses.
• the present disclosure may provide the wearer with GPS-based content reception, as in Fig. 6.
• augmented reality glasses of the present disclosure may include memory, a global positioning system, a compass or other orienting device, and a camera.
• GPS-based computer programs available to the wearer may include a number of applications typically available from the Apple Inc. App Store for iPhone use. Similar versions of these programs are available for other brands of Smartphone and may be applied to embodiments of the present disclosure. These programs include, for example, SREngine (scene recognition engine), NearestTube, TAT Augmented ID, Yelp, Layar, and TwittARound, as well as other more specialized applications, such as RealSki.
• the user controls the glasses in any of the manners or modes described elsewhere in this patent.
• the user may call up a desired program or application by voice or by indicating a choice on the virtual screen of the augmented reality glasses.
• the augmented glasses may respond to a track pad mounted on the frame of the glasses, as described above.
• the glasses may be responsive to one or more motion or position sensors mounted on the frame. The signals from the sensors are then sent to a microprocessor or microcontroller within the glasses, the glasses also providing any needed signal transducing or processing.
• the program of choice has begun, the user makes selections and enters a response by any of the methods discussed herein, such as signaling "yes” or "no" with a head movement, a hand gesture, a trackpad depression, or a voice command.
• Applications of the present disclosure may provide the wearer with Internet access, such as for browsing, searching, shopping, entertainment, and the like, such as through a wireless communications interface to the eyepiece.
• a wearer may initiate a web search with a control gesture, such as through a control facility worn on some portion of the wearer's body (e.g. on the hand, the head, the foot), on some component being used by the wearer (e.g. a personal computer, a smart phone, a music player), on a piece of furniture near the wearer (e.g. a chair, a desk, a table, a lamp), and the like, where the image of the web search is projected for viewing by the wearer through the eyepiece.
• the wearer may then view the search through the eyepiece and control web interaction though the control facility.
• local advertisements may be stored on a server that contains geo-located local advertisements and specials, and these advertisements may be delivered to the wearer individually as the wearer approaches a particular location, or a set of advertisements may be delivered to the wearer in bulk when the wearer enters a geographic area that is associated with the advertisements so that the advertisements are available when the user nears a particular location.
• the geographic location may be a city, a part of the city, a number of blocks, a single block, a street, a portion of the street, sidewalk, and the like, representing regional, local, hyper-local areas.
• advertisement used herein, but one skilled in the art will appreciate that this can also mean an announcement, a broadcast, a circular, a commercial, a sponsored communication, an endorsement, a notice, a promotion, a bulletin, a message, and the like.
• Figs. 18-20A depict ways to deliver custom messages to persons within a short distance of an establishment that wishes to send a message, such as a retail store.
• embodiments may provide for a way to view custom billboards, such as when the wearer of the eyepiece is walking or driving, by applications as mentioned above for searching for providers of goods and services.
• the billboard 1800 shows an exemplary augmented reality-based advertisement displayed by a seller or a service provider.
• the exemplary advertisement may relate to an offer on drinks by a bar. For example, two drinks may be provided for the cost of just one drink.
• augmented reality-based advertisements and offers With such augmented reality-based advertisements and offers, the wearer's attention may be easily directed towards the billboards.
• the billboards may also provide details about location of the bar such as street address, floor number, phone number, and the like.
• several devices other than eyepiece may be utilized to view the billboards. These devices may include without limitations smartphones, IPHONEs, IPADs, car windshields, user glasses, helmets, wristwatches, headphones, vehicle mounts, and the like.
• a user (wearer in case the augmented reality technology is embedded in the eyepiece) may automatically receive offers or view a scene of the billboards as and when the user passes or drives by the road.
• the user may receive offers or view the scene of the billboards based on his request.
• a viewer may be walking though a clothing store, and as they walk are provided with information on the clothes on the rack, where the information is provided through the product's RFID tag.
• the information may be delivered as a list of information, as a graphic representation, as audio and/or video presentation, and the like.
• the wearer may be food shopping, and advertisement providing facilities may be providing information to the wearer in association with products in the wearer's proximity, the wearer may be provided information when they pick up the product and view the brand, product name, SKU, and the like. In this way, the wearer may be provided a more informative environment in which to effectively shop.
• the wearer may also receive advertisements and offers automatically or based on request regarding current deals, promotions and events in the interested location such as a nearby shopping store.
• the advertisements, deals and offers may appear in proximity of the wearer and options may be displayed for purchasing desired products based on the advertisements, deals and offers.
• the wearer may for example select a product and purchase it through a Google checkout.
• a message or an email may appear on the eyepiece, similar to the one depicted in FIG. 7, with information that the transaction for the purchase of the product has been completed.
• a product delivery status/information may also be displayed.
• the wearer may further convey or alert friends and relatives regarding the offers and events through social networking platforms and may also ask them to join.
• the eyepiece may include an audio device.
• the displayed content may comprise a local advertisement and audio such that the user is also able to hear a message or other sound effects as they relate to the local advertisement.
• the user may hear the bottle open and then the sound of the liquid pouring out of the bottle and onto the rooftop.
• a descriptive message may be played, and or general information may be given as part of the advertisement.
• any audio may be played as desired for the advertisement.
• Another embodiment may allow information concerning auto repairs and maintenance services with the use of the augmented reality enabled devices such as a camera lens fitted in the eyepiece.
• the wearer may receive advertisements related to auto repair shops and dealers by sending a voice command for the request.
• the request may, for example include a requirement of oil change in the vehicle/car.
• the eyepiece may receive information from the repair shop and display to the wearer.
• the eyepiece may pull up a 3D model of the wearer's vehicle and show the amount of oil left in the car through an augmented reality enabled scene/view.
• the eyepiece may show other relevant information also about the vehicle of the wearer such as maintenance requirements in other parts like brake pads.
• the wearer may see 3D view of the wearing brake pads and may be interested in getting those repaired or changed. Accordingly, the wearer may schedule an appointment with a vendor to fix the problem via using the integrated wireless communication capability of the eyepiece.
• the confirmation may be received through an email or an incoming call alert on the eyepiece camera lens.
• the available options may be scrolled by the wearer and the best fit option may be selected via the voice command or text command.
• the wearer may select the Golfer's gift basket.
• a 3D view of the Golfer's gift basket along with a golf course may appear in front of the wearer.
• the virtual 3D view of the Golfer's gift basket and the golf course enabled through the augmented reality may be perceived very close to the real world environment.
• the wearer may finally respond to the address, location and other similar queries prompted through the eyepiece. A confirmation may then be received through an email or an incoming call alert on the eyepiece camera lens.
• augmented reality glasses may be computer video games, such as those furnished by Electronic Arts Mobile, UbiSoft and Activision Blizzard, e.g., World of Warcraft® (WoW).
• WoW World of Warcraft®
• augmented reality glasses may also use gaming applications.
• the screen may appear on an inside of the glasses so that a user may observe the game and participate in the game.
• controls for playing the game may be provided through a virtual game controller, such as a joystick, control module or mouse, described elsewhere herein.
• the game controller may include sensors or other output type elements attached to the user's hand, such as for feedback from the user through acceleration, vibration, force, electrical impulse, temperature, electric field sensing, and the like. Sensors and actuators may be attached to the user's hand by way of a wrap, ring, pad, glove, bracelet, and the like.
• an eyepiece virtual mouse may allow the user to translate motions of the hand, wrist, and/or fingers into motions of the cursor on the eyepiece display, where "motions" may include slow movements, rapid motions, jerky motions, position, change in position, and the like, and may allow users to work in three dimensions, without the need for a physical surface, and including some or all of the six degrees of freedom.
• the location may change for different players in the game, sending one player to one location and another player or players to a different location.
• the game may then have different scenarios for each player or group of players, based on their GPS or magnetometer-provided locations. Each player may also be sent different messages or images based on his or her role, his or her location, or both. Of course, each scenario may then lead to other situations, other interactions, directions to other locations, and so forth. In one sense, such a game mixes the reality of the player's location with the game in which the player is participating.
• AR Invaders available for iPhone applications from the App Store.
• Other games may be obtained from other vendors and for non-iPhone type systems, such as Layar, of Amsterdam and Paris SA, Paris, France, supplier of AR Drone, AR Flying Ace and AR Pursuit.
• games may also be in 3D such that the user can experience 3D gaming.
• the user when playing a 3D game, the user may view a virtual, augmented reality or other environment where the user is able to control his view perspective.
• the user may turn his head to view various aspects of the virtual environment or other environment.
• the perspective of the user may be such that the user is put 'into' a 3D game environment with at least some control over the viewing perspective where the user may be able to move his head and have the view of the game environment change in correspondence to the changed head position.
• the user may interact with or cause reactions from other users playing the game and or 3D elements associated with the other users playing the game.
• Such elements associated with users may include weapons, messages, currency, a 3D image of the user and the like.
• Based on a user's location or other data he or she may encounter, view, or engage, by any means, other users and 3D elements associated with other users.
• 3D gaming may also be provided by software installed in or downloaded to the glasses where the user's location is or is not used.
• the learned behavior and/or actions of a plurality of eyepiece users may be autonomously stored in a collective behavior database, where learned behaviors amongst the plurality of users are available to individual users based on similar conditions. For example, a user may be visiting a city, and waiting for a train on a platform, and the eyepiece of the user accesses the collective behavior database to determine what other users have done while waiting for the train, such as getting directions, searching for points of interest, listening to certain music, looking up the train schedule, contacting the city website for travel information, connecting to social networking sites for entertainment in the area, and the like. In this way, the eyepiece may be able to provide the user with an automated assistant with the benefit of many different user experiences.
• the learned behavior may be used to develop preference profiles, recommendations, advertisement targeting, social network contacts, behavior profiles for the user or groups of users, and the like, for/to the user.
• One or more of the squad vehicle, the platoon commander, the mobile platform, the TOC or the CP will have an integration capability for combining the inputs from the several soldiers and determining a possible location of the hostile fire.
• the signals from each soldier will include the location of the soldier from a GPS capability inherent in the augmented reality glasses or eyepiece.
• the acoustic sensors on each soldier may indicate a possible direction of the noise. Using signals from several soldiers, the direction and possibly the location of the hostile fire may be determined. The soldiers may then neutralize the location.
• Fig. 67 illustrates capturing an iris image for iris recognition.
• the figure illustrates the focus parameters used to analyze the image and includes a geographical location of the individual at the time of biometric data capture.
• Fig. 67 also depicts a sample report that is displayed on the eyepiece.
• the multiple microphones may be arranged in a composite
• the eyepiece may be used as Siglnt Glasses.
• the eyepiece may be used to conspicuously and passively gather signals intelligence for devices and individuals in the user's proximity. Signals intelligence may be gathered automatically or may be triggered when a particular device ID is in proximity, when a particular audio sample is detected, when a particular geo-location has been reached, and the like.
• the mosaic print sensor offers high reliability in harsh environments with excessive sunlight.
• multiple wafer scale optical sensors are digitally stitched together using pixel subtraction.
• the resulting images are engineered to be over 500 dots per inch (dpi).
• Power is supplied by a battery or by parasitically drawing power from other sources using a USB protocol. Formatting is EFTS, EBTS NIST, ISO, and ITL 1-2007 compliant.
• Camera Sensor Support Sensor dynamic range extension, Adaptive defect pixel correction, advanced sharpness enhancement, Geometric distortion correction, advanced color management, HW based face detection, Video stabilization
• Sensors Can accommodate the addition of accelerometer, compass, ambient light, proximity, barometric, and temperature sensors, depending on requirements.
• Audio On board microphone, speakers, and audio/video inputs.
• Keyboard Full tactile QWERTY keyboard with adjustable backlight.
• Additional devices and kits may also incorporate the mosaic sensors and may operate in conjunction with the bio-phone and tactical computer to provide a complete field solution for collection biometric data.
• Biometric data collect is geo-located for monitoring and tracking individual movement. Finger and palm prints, iris images, face images, latent fingerprints, and video may be collected and enrolled in a database using the bio-kit. Algorithms for finger and palm prints, iris images, and face images facilitate these types of data collection. To aid in capturing iris images and latent fingerprint images simultaneously, the bio-kit has IR and UV diodes that actively illuminate an iris or latent fingerprint. In addition, the pocket bio-kit is also fully EFTS/EBTS compliant, including ITL 1-2007 and WSQ. The bio-kit meets MIL-STD-810 for operation in environmental extremes and uses a Linux operating system.
• the fingerprint, palm print, geo-location, and POI enrollments device includes an integrated computer, QWERTY keyboard, and display.
• the display is designed to allow easy operation in strong sunlight and uses an LCD screen or LED indicator to alert the operator of successful fingerprint and palm print capture.
• the display uses transflective QVGA color, with a backlit LCD screen to improve readability.
• the device is lightweight and compact, weighing 16 oz. and measuring 5" x 2.5" at the mosaic sensor. This compact size and weight allows the device to slip into an LBV pocket or be strapped to a user's forearm, as shown in Fig. 44.
• all POIs are tagged with geo-location information at the time of capture.
• the eyepiece is capable of communicating wirelessly with the bio- phone sensor and forearm mounted devices using a 256-bit AES encrypted UWB. This also allows the device to communicate with a laptop or combat radio, as well as network to CPs, TOCs, and biometric databases.
• the eyepiece is ABIS, EBTS, EFTS, and JPEG 2000 compatible.
• Fig. 46 illustrates a further embodiment of the fingerprint, palm print, geo-location, and POI enrollment device.
• the device is 16 oz and uses a 5" x 2.5" active fingerprint and palm print capacitance sensor.
• the sensor is capable of enrolling 10 fingers, a palm, 4 finger slap, and finger tip prints at 500 dpi.
• a 0.6 - 1 GHz processor with 430 MHz DSP provides rapid enrollment and data capture.
• the device is ABIS, EBTS, EFTS, and JPEG 2000 compatible and features networked GPS for highly accurate location of persons of interest.
• the device communicates wirelessly over a 256-bit AES encrypted UWB, laptop, or combat radio. Database information may also be stored on the device, allowing in the field comparison without uploading information. This onboard data may also be shared wirelessly with other devices, such as a laptop or combat radio.
• Data may be stored on the forearm device since the device can utilize Mil-con data storage caps for increased storage capacity. Data entry is performed on the QWERTY keyboard and may be done wearing gloves.
• the mosaic sensor described above may also be incorporated into a mobile, folding biometric enrollment kit, as shown in Fig. 47.
• the mobile folding biometric enrollment kit 4700 folds into itself and is sized to fit into a tactical vest pocket, having dimensions of 8 x 12 x 4 inches when unfolded.
• Fig. 48 illustrates how the eyepiece and forearm mounted device interface to provide a complete system for biometric data collection.
• Fig. 50 illustrates a thin-film finger and palm print collection device.
• the device can record four fingerprint slaps and rolls, palm prints, and fingerprints to the NIST standard. Superior quality finger print images can be captured with either wet or dry hands.
• the device is reduced in weight and power consumption compared to other large sensors.
• the sensor is self-contained and is hot swappable.
• the configuration of the sensor may be varied to suit a variety of needs, and the sensor may be manufactured in various shapes and dimensions.
• Weight & Size 16 oz. forearm straps or inserts into LBV pocket
• Tags all collected biometric data with time, date, and location
• Figs. 52-54 depict use of the devices incorporating a sensor for collecting biometric data.
• Fig. 52 shows capture of a two stage palm print.
• Fig. 53 shows collection using a fingertip tap.
• Fig. 54 demonstrates a slap and roll print being collected.
• fingerprints may be taken by persons using a polarized light source and retrieving images of the fingerprints using reflected polarized light in two planes.
• fingerprints may be taken by persons using a light source and retrieving images of the fingerprints using
• multispectral processing e.g., using two imagers at two different locations with different inputs.
• the different inputs may be caused by using different filters or different sensors/imagers.
• Applications of this technology may include biometric checks of unknown persons or subjects in which the safety of the persons doing the checking may be at issue.
• the menu may then offer selections as to which database to use for comparison, which device(s) to use for storage, etc.
• the touchless or contactless biometric data gathering system may be controlled by any of the methods described herein.

Landscapes

• Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• General Engineering & Computer Science (AREA)
• Business, Economics & Management (AREA)
• Human Computer Interaction (AREA)
• Multimedia (AREA)
• Strategic Management (AREA)
• Finance (AREA)
• Development Economics (AREA)
• Accounting & Taxation (AREA)
• Signal Processing (AREA)
• Computer Hardware Design (AREA)
• Optics & Photonics (AREA)
• Game Theory and Decision Science (AREA)
• Entrepreneurship & Innovation (AREA)
• Economics (AREA)
• Marketing (AREA)
• General Business, Economics & Management (AREA)
• Health & Medical Sciences (AREA)
• Ophthalmology & Optometry (AREA)
• General Health & Medical Sciences (AREA)
• Acoustics & Sound (AREA)
• Otolaryngology (AREA)
• User Interface Of Digital Computer (AREA)
• Information Transfer Between Computers (AREA)

Priority Applications (5)

Applications Claiming Priority (12)

Publications (1)

Family

ID=44505790

Family Applications (2)

Family Applications Before (1)

Country Status (7)

Cited By (61)

Families Citing this family (1542)