US11475177B2 - Method and apparatus for improved position and orientation based information display - Google Patents

Method and apparatus for improved position and orientation based information display Download PDF

Info

Publication number
US11475177B2
US11475177B2 US17/398,804 US202117398804A US11475177B2 US 11475177 B2 US11475177 B2 US 11475177B2 US 202117398804 A US202117398804 A US 202117398804A US 11475177 B2 US11475177 B2 US 11475177B2
Authority
US
United States
Prior art keywords
smart device
transceiver
location
interest
transceivers
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
US17/398,804
Other versions
US20210374300A1 (en
Inventor
Michael A. Wodrich
Michael S. Santarone
Jason E. Duff
Fred Flitsch
Randall Pugh
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Middle Chart LLC
Original Assignee
Middle Chart LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US15/703,310 external-priority patent/US20180239313A1/en
Priority claimed from US15/887,637 external-priority patent/US20180239840A1/en
Priority claimed from US16/142,275 external-priority patent/US10433112B2/en
Priority claimed from US16/161,823 external-priority patent/US10467353B2/en
Priority claimed from US16/165,517 external-priority patent/US10733334B2/en
Priority claimed from US16/171,593 external-priority patent/US10620084B2/en
Priority claimed from US16/176,002 external-priority patent/US10268782B1/en
Priority claimed from US16/297,383 external-priority patent/US10762251B2/en
Priority claimed from PCT/US2019/033134 external-priority patent/WO2020068177A1/en
Priority claimed from US16/503,878 external-priority patent/US10776529B2/en
Priority claimed from US16/504,919 external-priority patent/US10740502B2/en
Priority claimed from US16/528,104 external-priority patent/US10671767B2/en
Priority claimed from US16/657,660 external-priority patent/US20200065433A1/en
Priority claimed from US16/688,775 external-priority patent/US10628617B1/en
Priority claimed from US16/775,233 external-priority patent/US20210041440A1/en
Priority claimed from US16/817,926 external-priority patent/US10831945B2/en
Priority claimed from US16/898,602 external-priority patent/US10949579B2/en
Application filed by Middle Chart LLC filed Critical Middle Chart LLC
Priority to US17/398,804 priority Critical patent/US11475177B2/en
Publication of US20210374300A1 publication Critical patent/US20210374300A1/en
Assigned to Middle Chart, LLC reassignment Middle Chart, LLC ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: Flitsch, Fred, PUGH, RANDALL, DUFF, JASON E., SANTARONE, MICHAEL S., Wodrich, Michael A.
Application granted granted Critical
Publication of US11475177B2 publication Critical patent/US11475177B2/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F30/00Computer-aided design [CAD]
    • G06F30/10Geometric CAD
    • G06F30/13Architectural design, e.g. computer-aided architectural design [CAAD] related to design of buildings, bridges, landscapes, production plants or roads
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01CMEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/20Instruments for performing navigational calculations
    • G01C21/206Instruments for performing navigational calculations specially adapted for indoor navigation
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S11/00Systems for determining distance or velocity not using reflection or reradiation
    • G01S11/16Systems for determining distance or velocity not using reflection or reradiation using difference in transit time between electrical and acoustic signals
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S15/00Systems using the reflection or reradiation of acoustic waves, e.g. sonar systems
    • G01S15/87Combinations of sonar systems
    • G01S15/876Combination of several spaced transmitters or receivers of known location for determining the position of a transponder or a reflector
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S19/00Satellite radio beacon positioning systems; Determining position, velocity or attitude using signals transmitted by such systems
    • G01S19/38Determining a navigation solution using signals transmitted by a satellite radio beacon positioning system
    • G01S19/39Determining a navigation solution using signals transmitted by a satellite radio beacon positioning system the satellite radio beacon positioning system transmitting time-stamped messages, e.g. GPS [Global Positioning System], GLONASS [Global Orbiting Navigation Satellite System] or GALILEO
    • G01S19/42Determining position
    • G01S19/48Determining position by combining or switching between position solutions derived from the satellite radio beacon positioning system and position solutions derived from a further system
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S3/00Direction-finders for determining the direction from which infrasonic, sonic, ultrasonic, or electromagnetic waves, or particle emission, not having a directional significance, are being received
    • G01S3/02Direction-finders for determining the direction from which infrasonic, sonic, ultrasonic, or electromagnetic waves, or particle emission, not having a directional significance, are being received using radio waves
    • G01S3/14Systems for determining direction or deviation from predetermined direction
    • G01S3/46Systems for determining direction or deviation from predetermined direction using antennas spaced apart and measuring phase or time difference between signals therefrom, i.e. path-difference systems
    • G01S3/50Systems for determining direction or deviation from predetermined direction using antennas spaced apart and measuring phase or time difference between signals therefrom, i.e. path-difference systems the waves arriving at the antennas being pulse modulated and the time difference of their arrival being measured
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S5/00Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations
    • G01S5/02Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations using radio waves
    • G01S5/0205Details
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S5/00Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations
    • G01S5/02Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations using radio waves
    • G01S5/0257Hybrid positioning
    • G01S5/0263Hybrid positioning by combining or switching between positions derived from two or more separate positioning systems
    • G01S5/0264Hybrid positioning by combining or switching between positions derived from two or more separate positioning systems at least one of the systems being a non-radio wave positioning system
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S5/00Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations
    • G01S5/02Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations using radio waves
    • G01S5/14Determining absolute distances from a plurality of spaced points of known location
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S5/00Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations
    • G01S5/18Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations using ultrasonic, sonic, or infrasonic waves
    • G01S5/30Determining absolute distances from a plurality of spaced points of known location
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q99/00Subject matter not provided for in other groups of this subclass
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/05Geographic models
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00Manipulating 3D models or images for computer graphics
    • G06T19/006Mixed reality
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S15/00Systems using the reflection or reradiation of acoustic waves, e.g. sonar systems
    • G01S15/88Sonar systems specially adapted for specific applications
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88Lidar systems specially adapted for specific applications
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88Lidar systems specially adapted for specific applications
    • G01S17/89Lidar systems specially adapted for specific applications for mapping or imaging
    • G01S17/8943D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S19/00Satellite radio beacon positioning systems; Determining position, velocity or attitude using signals transmitted by such systems
    • G01S19/01Satellite radio beacon positioning systems transmitting time-stamped messages, e.g. GPS [Global Positioning System], GLONASS [Global Orbiting Navigation Satellite System] or GALILEO
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/48Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
    • G01S7/481Constructional features, e.g. arrangements of optical elements
    • G01S7/4817Constructional features, e.g. arrangements of optical elements relating to scanning
    • GPHYSICS
    • G08SIGNALLING
    • G08BSIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B21/00Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
    • G08B21/02Alarms for ensuring the safety of persons
    • G08B21/0202Child monitoring systems using a transmitter-receiver system carried by the parent and the child
    • G08B21/0263System arrangements wherein the object is to detect the direction in which child or item is located
    • GPHYSICS
    • G08SIGNALLING
    • G08BSIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B21/00Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
    • G08B21/02Alarms for ensuring the safety of persons
    • G08B21/0202Child monitoring systems using a transmitter-receiver system carried by the parent and the child
    • G08B21/0272System arrangements wherein the object is to detect exact location of child or item using triangulation other than GPS

Definitions

  • the present invention relates to methods and apparatus for enhanced determination of a direction of interest based upon orientation of an apparatus. More specifically, the present invention presents methods and apparatus for indicating a direction based upon wireless communications between transceivers and using that to guide information display.
  • a geospatial position it is known for a geospatial position to be ascertained based upon triangulation techniques.
  • triangulation techniques may be based upon artificial location references, such as satellites and/or cell towers.
  • location determination within a structure is limited due to variations in communication signal strength within a structure.
  • positioning systems lack an accurate functionality for determining a direction of interest to a user.
  • the present invention combines methods and apparatus for enhanced determination of a position of a Smart Device and one or both of a direction of interest of an Agent and a direction of orientation of a Smart Device.
  • the enhanced determination of a position of the Smart Device; and one or both of a direction of interest of an Agent and a direction of orientation of a Smart Device may be accomplished via registration and storage of aspects of an environment surrounding the Smart Device at a first instance in time.
  • the registered aspects are subsequently referenced and compared to registration and storage of similar (or same) aspects at a second instance in time.
  • the comparison of the registered aspects may be combined with wireless communications to provide enhanced location and orientation of the Smart Device.
  • enhanced location and orientation may be augmented via operation of devices for determining physical attitude of the Smart Device (e.g.
  • pitch and yaw and bank typically denoted as typically denoted as ⁇ , ⁇ , ⁇ , or ⁇ , ⁇ , ⁇ ), wherein a change in pitch, yaw or bank may be tracked from an original position and orientation of the Smart Device to calculate a subsequent position and orientation of the Smart Device
  • aspects of an environment may include, for example, one or more of physical features in a proximate area to a Smart Device or other Node that are quantifiable via: sonic (including ultrasonic) waves; LIDAR measurements; magnetic force readings and visual light wavelength measurements.
  • Instruments useful in determining physical features in proximity to a Smart Devices may include, for example, one or more of: accelerometers, charged coupled devices (CCD); LiDAR transmitters and receivers; enhanced magnetic sensors, magneto-resistive sensors, Hall Effect sensors or transducers, MEMS (micro electromechanical sensor) field sensor, anisotropic magneto resistive effect transducers, and giant magneto resistive effect transducers or other mechanism to measure a magnetic field.
  • Location determination may be performed using combinations of positionally fixed Nodes communicating via wireless communication devices and protocols.
  • the present invention provides for multiple complimentary ways for determining an orientation of a smart device relative to a structure based upon supporting a first transceiver and a second transceiver via an agent at a first geospatial position within radio communication distance to transceivers located within the structure and generating respective logical communications between the first transceiver and each of a first set of three reference point transceivers, the three reference point transceivers fixedly positioned at disparate points in the structure.
  • a controller may execute software code to determining X, Y and Z coordinates of the first transceiver based upon the respective logical communications between the first transceiver and each of a first set of three reference point transceivers and generate respective logical communications between the second transceiver and each of a second set of three reference point transceivers.
  • the controller may also determine X, Y and Z coordinates of the second transceiver based upon the respective logical communications between the first transceiver and each of a first set of three reference point transceivers.
  • a magnetic field sensor may generate a first quantification of a magnetic field encompassing the smart device such that a location of the smart device is generated based upon at least one of: X, Y and Z coordinates of the first transceiver, and the X, Y and Z coordinates of the second transceiver.
  • a first direction of interest may be generated based upon at least two of: the X, Y and Z coordinates of the first transceiver, the X, Y and Z coordinates of the second transceiver, and the quantification of the magnetic field encompassing the smart device.
  • Data associated with the location of the smart device and the first direction of interest may be accessed and a user interface may be generated that includes the data associated with the location of the smart device and/or a direction of interest.
  • the smart device may be reoriented and a second quantification of the magnetic field encompassing the smart device may be quantified such that a second direction of interest may be generated based upon the second quantification of the magnetic field relative to the smart device.
  • the magnetic field sensor may include a Hall Effect sensor and the transceivers may be embodied in one or more of: a smart watch, a smart ring, and a smart wand.
  • the smart device may be reoriented and a topographical representation of at least a portion of an environment within an area capable of wireless communication the smart device may be generated, such as, for example using LIDAR or sonic measurements, and generating a second direction of interest based upon the topographical representation.
  • Apparatus for determining an orientation relative to a structure may include: a controller comprising a processor and a digital storage, said controller in logical communication with a user interface display; a first transceiver supportable by an Agent at a first geospatial position and in logical communication with the controller; and a second transceiver supportable by an Agent at the first geospatial position and in logical communication with the controller; a magnetic field sensor in logical communication with the controller; and executable code stored on the digital storage and executable by the processor to cause the controller to perform the method steps involved in generating a position and direction of interest.
  • the controller may be operative to: generate respective logical communications between the first transceiver and each of a first set of three reference point transceivers, the three reference point transceivers fixedly positioned at disparate points in the structure; determine X, Y and Z coordinates of the first transceiver based upon the respective logical communications between the first transceiver and each of a first set of three reference point transceivers; generate respective logical communications between the second transceiver and each of a second set of three reference point transceivers; determine X, Y and Z coordinates of the second transceiver based upon the respective logical communications between the first transceiver and each of a first set of three reference point transceivers; generate a first quantification of a magnetic field encompassing the smart device with a magnetic field sensor; generate a location of the smart device based upon at least one of: X, Y and Z coordinates of the first transceiver, and the X, Y and Z coordinates of the second transcei
  • the controller may generate a second quantification of the magnetic field encompassing the smart device following a reorientation of the smart device; and generate a second direction of interest based upon the second quantification of the magnetic field relative to the smart device.
  • FIG. 1 illustrates location determination with wireless communication to reference point transceivers.
  • FIG. 2 illustrates location determination using sonic transceivers.
  • FIG. 3 illustrates methods of orienteering by device movement.
  • FIGS. 4A-4D illustrate exemplary configurations of antenna arrays.
  • FIG. 5A illustrates an exemplary smart device with an array of antennas.
  • FIG. 5B illustrates a direction determination of a smart device.
  • FIG. 5C illustrates embodiments of the invention that include stereoscopic cameras.
  • FIG. 6 illustrates apparatus that may be used to implement aspects of the present invention including executable software.
  • FIG. 7 illustrates an exemplary smart device that may be used to implement aspects of the present invention including executable software.
  • FIG. 8A-8C illustrate movements for determining a direction of interest of a smart device.
  • FIG. 9 illustrates method steps that may be implemented according to some aspects of the present invention.
  • FIG. 10 illustrates an Agent supporting a smart device.
  • FIG. 11 illustrates method steps that may be implemented in some embodiments of the present invention.
  • the present invention relates to methods and apparatus for enhanced wireless determination of a position and direction of a smart device.
  • Wireless transceivers controlled by the smart device communicate with reference point transceivers to generate data sufficient to determine relative positions of the wireless transceivers and a direction of interest.
  • Operation of one or both of a magnetic force sensor and LIDAR may be operative to verify the position and direction of the Smart Device.
  • Reference Point Transceivers 101 - 104 are shown deployed within or proximate to a Structure 106 , to determine a location 107 of a Transceiver 105 supported an Agent 100 within or proximate to the Structure 106 .
  • Reference Point Transceivers 101 - 104 may be fixed in a certain location within or proximate to the Structure 106 and define a wireless communication area 111 .
  • the Reference Point Transceivers 101 - 104 may transceive in a manner suitable for determination of a position of one or more Transceivers 105 supported by an Agent 100 , such as, for example, a Transceiver 105 in or associated with a Smart Device, headgear or Tag supported by the Agent 100 . Transceiving may be conducted via one or more wireless transmission modalities between a portable Transceiver 105 supported by the Agent 100 and one or more Reference Point Transceivers 101 - 104 .
  • Transceivers 105 supported by the Agent 100 may be included in, and/or be in logical communication with, a Smart Device, such as a smart phone, tablet, headgear, ring, watch, wand, pointer, badge, Tag, Node or other Agent supportable device with portable Transceiver 105 able to transceive with the Reference Point Transceivers 101 - 104 .
  • a Smart Device such as a smart phone, tablet, headgear, ring, watch, wand, pointer, badge, Tag, Node or other Agent supportable device with portable Transceiver 105 able to transceive with the Reference Point Transceivers 101 - 104 .
  • the Reference Point Transceivers 101 - 104 may include devices, such as, for example, a radio transmitter, radio receiver, a light generator, a light receiver, a pattern recognizing device.
  • a radio frequency transceiver may transmitters and receivers operative to communicate via wireless modalities such as, for example: WiFi, Bluetooth, Ultrawideband, Ultrasonic, infrared, or other communication modality capable of logical communication between Transceivers 101 - 105 .
  • a Reference Point Transceivers 101 - 104 may include a multi-modality transceiver, that communicates, more locally via a first modality, such as UWB, Bluetooth, WiFi, ANT, Zigbee, BLE, Z Wave, 6LoWPAN, Thread, WiFi, WiFi-ah, NFC (near field communications), sub Giga Hertz, Dash 7, Wireless HART or similar modality and to a greater distance via a second modality, such as cellular communication modalities (e.g. 3G, 4G 5G and the like), sub GHz, IP Protocol modalities and the like which may provide access to a distributed network, such as the Internet.
  • a first modality such as UWB, Bluetooth, WiFi, ANT, Zigbee, BLE, Z Wave, 6LoWPAN, Thread, WiFi, WiFi-ah, NFC (near field communications), sub Giga Hertz, Dash 7, Wireless HART or similar modality and to a greater distance via a second modality, such
  • Wireless communications between Transceivers 101 - 105 may engage in logical communications to provide data capable of generating one or more of: Cartesian Coordinates, X, Y, Z coordinates, vector values, AoA, AoD, RTT, RSS, a GPS position, or other data that may be utilized for one or more of: locating one or both of an Agent 100 ; indicating a direction of interest; and identify a Structure or defined area 106 .
  • a precise location may be determined via triangulation based upon a measured distance from three or more Reference Point Transceivers 101 - 104 .
  • a radio transmission or light signal may be measured and compared from the three reference position identifiers 101 - 103 .
  • Other embodiments may include a device recognizable via image analysis and a camera or other Image Capture Device, such as a CCD device, may capture an image of three or more Reference Point Transceivers 101 - 104 .
  • Image analysis may recognize the identification of each of three or more of the Reference Point Transceivers 101 - 104 and a size ratio of the respective image captured Reference Point Transceivers 101 - 104 may be utilized to calculate a precise position.
  • a height designation may be made via triangulation using the position identifiers as reference to a known height or a reference height.
  • Transceivers 101 - 105 may include circuitry and logic capable of transceiving in a single modality, or multiple disparate modalities.
  • a Reference Point Transceiver 101 - 104 and/or an Agent supported Transceiver 105 may include multiple transceiver device, including, transmitters and receivers.
  • a modality, as used in conjunction with a Transceiver, transmitter and/or receiver refers to one or both of a bandwidth of wireless communication and a protocol associated with a bandwidth.
  • a modality, as used in relation to a Transceiver, transmitter and/or receiver may include: WiFi; WiFi RTT; Bluetooth; UWB; Ultrasonic, sonic, infrared; ANT, Zigbee, BLE, Z Wave, 6LoWPAN, Thread, WiFi, Wi-Fi 33-ah, NFC (near field communications), sub Giga Hertz, Dash 7, Wireless HART or other logical communication medium.
  • Triangulation essentially includes determining an intersection of three distances 108 - 110 , each distance 108 - 110 calculated from a reference point 101 - 104 to an Agent supported device 105 .
  • the presence invention allows for a first distance 108 , to be determined based upon a wireless communication in a first modality; and a second distance 109 and a third distance 110 determined based upon a wireless communication in a same or different modality as the first modality.
  • a first distance 108 may be determined based upon a wireless communication using UWB; a second distance 109 may be determined based upon a wireless communication using Bluetooth; and a third communication may be determined based upon a wireless communication using ultrasonic communication (other combinations of same and/or different communication modalities are also within the scope of the present invention).
  • a location 107 may be determined via triangulation based upon a measured distance from three or more position identifiers 101 - 103 to the transceiver 105 supported by the Agent 100 . For example, timing associated with a radio transmission or light signal may be measured and compared from the three reference position identifiers 101 - 103 .
  • Other embodiments may include a device recognizable via image analysis and a camera or other Image Capture Device, such as a CCD device, may capture an image of three or more position identifiers 101 - 104 .
  • Additional embodiments may include image analysis of image data captured via a CCD included in a Smart Device to recognize the identification of each of three or more of the position identifiers 101 - 104 and a size ratio of the respective image captured position identifiers 101 - 104 may be utilized to calculate a precise position.
  • a height designation may be made via triangulation using the position identifiers as reference to a known height or a reference height.
  • triangulation may be utilized to determine a relative elevation of the Smart Device as compared to a reference elevation of the reference points.
  • the position 107 of the Agent supported Transceiver 105 may be ascertained via one or more of: triangulation; trilateration; and multilateration (MLT) techniques.
  • a geospatial location based upon triangulation may be generated based upon a controller receiving a measurement of angles between the position and known points at either end of a fixed baseline.
  • a point of a geospatial location may be determined based upon generation of a triangle with one known side and two known angles.
  • a geospatial location based upon multilateration may be generated based on a controller receiving measurement of a difference in distance to two reference positions, each reference position being associated with a known location.
  • Wireless signals may be available at one or more of: periodically, within determined timespans and continually. The determination of the difference in distance between two reference positions provides multiple potential locations at the determined distance.
  • a controller may be used to generate a plot of potential locations.
  • the potential determinations generally form a curve. Specific embodiments will generate a hyperbolic curve.
  • the controller may be programmed to execute code to locate a relatively exact position along a generated curve, which is used to generate a geospatial location.
  • the multilateration system thereby receives as input multiple measurements of distance to reference points, wherein a second measurement taken to a second set of stations (which may include one station of a first set of stations) is used to generate a second curve. A point of intersection of the first curve and the second curve may be used to indicate a specific location.
  • the location of a Transceiver 101 - 105 may be determined and/or aided via discernment of data based upon a physical artifact, such as, for example a visually discernable feature, shape or printed aspect located within the Structure 106 .
  • Discernment of the physical artifact may, for example, be based upon topographical renditions of physical aspects included in the Structure, such as those measured using LIDAR, a magnetic force, image data (or a point cloud derived from image data).
  • a pattern on a surface may convey a reference point by a recognizable pattern (which may be unique to the setting), Vernier or three dimensional structure as non limiting examples.
  • a Smart Device ascertaining a physical reference mark and a distance of the Smart Device to the mark may determine a relative location in space to a coordinate system of the marks.
  • Marks tied to a geospatial coordinate system may be utilized to determine a relative location.
  • a number of methods may be executed to determine a distance from the Smart Device to a mark such as, for example, a sense reflection of light beams (preferably laser beams), electromagnetic beams of wavelength outside of the visible band such as IR, UV, Radio and the like, or sound based emanations.
  • a carefully placed reference point Node may function as a transceiver of signals.
  • a Node may receive and transmit signals in a radio frequency band of the electromagnetic spectrum.
  • a Node may detect an incoming signal and coincidently broadcast a radio frequency wireless communication.
  • Frequencies utilized for wireless communication may include those within the electromagnetic spectrum radio frequencies used in UWB, WiFi, and Bluetooth modalities, as well as IR, Visible and UV light as examples.
  • sound emanations may also be used as a communication mechanism between a smart device and a Reference Point Transceiver 101 - 104 .
  • the Reference Point Transceiver 101 - 104 may function to communicate data with their electromagnetic or sonic transmissions. Such communications may provide identifying information unique to the Node, data related to the synchronization of timing at different well located reference points, and may also function as general data communication nodes.
  • a triangulation calculation of the position of a Smart Device or a Node may result from a system of multiple reference position Nodes communicating timing signals to or from the Smart Device or Node.
  • Methods of calculating positions via wireless communications may include one or more of: RTT, RSSI, AoD, AoA, timing signal differential and the like, Triangulation or other mathematical techniques may also be employed in determining a location.
  • the process of determination of a position based upon triangulation with the reference points may be accomplished, for example via executable software interacting with the controller in a Smart Device, such as, for example via running an app on the Smart Device.
  • Reference points may be coded via identifiers, such as a UUID (Universally Unique Identifier), or other identification vehicle.
  • identifiers such as a UUID (Universally Unique Identifier), or other identification vehicle.
  • Reference Position Transceivers 101 - 104 may be deployed in a defined area 106 to determine a location 107 of an Agent 100 within or proximate to the defined wireless communication area 111 .
  • Reference Position Transceivers 101 - 104 may be fixed in a certain location and transceive in a manner suitable for a triangulation determination of the position of the Agent. Transceiving may occur via wireless transmission to one or more Transceivers 105 supported by the Agent 100 .
  • Transceivers 105 supported by the Agent 100 may be included in, or be in logical communication with, a Smart Device with Transceivers 105 able to transceive with the Reference Position Transceivers 101 - 104 .
  • the Reference Position Transceivers 101 - 104 may include devices such as a radio transmitter, radio receiver, a light generator, or an image-recognizable device (i.e., an apparatus set out in a distinctive pattern recognizable by a camera).
  • a radio transmitter may include a UWB Node, Wi-Fi, Bluetooth or other communication device for entering into logical communication between Transceivers 101 - 105 .
  • Reference Position Transceivers 101 - 104 may include a Wi-Fi router that additionally provides access to a distributed network, such as the Internet.
  • Cartesian coordinates may be used as data that may be utilized for one or more of: locating one or both of an Agent 100 ; indicating a direction of interest; and identifying a Structure 106 or defined area 106 .
  • a radio transmitter may include a router or other Wi-Fi device.
  • the radio transmitter may include transmissions via a Ultra Wideband (“UWB”) frequencies including, for example, 3.5-6.5 GHz; on Wi-Fi frequencies (300 MHz-60 GHz), sub GHz frequencies or other modality.
  • UWB Ultra Wideband
  • a light generator may distribute light at human-safe intensities and at virtually any frequency known in the art. Such frequencies include, without limitation, infrared, ultraviolet, visible, or nonvisible light.
  • the light beacon may comprise a laser, which may transmit light at any of the aforementioned frequencies in a coherent beam.
  • a Smart Device 101 ND/OR smart receptacle may measure A timing signal transmitted by a Reference Point Transceiver 101 - 104 within a different error tolerance than it may measure the receipt into a photodetector of infrared laser light.
  • the standard location calculation comprises a weighted average of the location as calculated by five modalities, but one modality yields a location greater than two standard deviations from the average computed location, then that modality may not be considered for future weighted location calculations.
  • the radio transmitters and/or transceiver in the Smart Device may comprise multiple antennas that transmit signals in a staggered fashion to reduce noise.
  • the antennas may comprise varying lengths to accommodate desirable wavelengths. Further, dead reckoning may be used to measure location, using traditional methods of numerical integration.
  • a precise location may be determined based upon wireless transmissions between Nodes, such as between an Smart Device and the Reference Position Transceivers. Timing determinations—as well as signal qualities like angle of arrival, angle of departure, transmission strength, transmission noise, and transmission interruptions—may be considered in generating relative positions of Nodes. Additional considerations may include AI and unstructured queries of transmissions between Nodes and triangulation logic based upon a measured distance from three or more Reference Position Transceivers 101 - 104 . For example, a radio transmission or light emission may be measured, and timing associated with the radio transmission or light to determine a distance between Nodes. Distances from three Reference Position Transceivers AA 01 -AA 03 may be used to generate a position of a Node in consideration. Other methodologies include determination of a distance from one or more Nodes and a respective angle of arrival and/or angle of departure of a radio or light transmission between the Node in consideration and another Node (Reference Point Node or dynamic position Node).
  • position determination in a Structure or on a Property contemplates determination of a geospatial location using triangulation, trilateration, or multilateration techniques.
  • a geospatial location relative to one or more known reference points is generated.
  • the geospatial location in space may be referred to as having an X,Y position indicating a planar designation (e.g. a position on a flat floor), and a Z position (e.g. a level within a Structure, such as a second floor) may be generated based upon indicators of distance from reference points. Indicators of distance may include a comparison of timing signals received from wireless references.
  • a geospatial location may be generated relative to the reference points.
  • a geospatial location with reference to a larger geographic area is associated with the reference points, however, in many embodiments, a controller will generate a geospatial location relative to the reference point(s) and it is not relevant where the position is located in relation to a greater geospatial area.
  • Cartesian coordinates polar coordinates may be used, as further described below.
  • a geospatial location based upon triangulation may be generated based upon a controller receiving a measurement of angles between the position and known points at either end of a fixed baseline.
  • a point of a geospatial location may be determined based upon generation of a triangle with one known side and two known angles.
  • triangulation essentially includes determining an intersection of three distances 108 - 210 , each distance 108 - 110 calculated from a reference point 101 - 104 to an Agent-supported device 105 .
  • the present invention allows for a first distance 108 to be determined based upon a wireless communication in a first modality; and a second distance 109 and a third distance 110 determined based upon a wireless communication in a same or different modality as the first modality.
  • a first distance 108 may be determined based upon a wireless communication using UWB; a second distance 109 may be determined based upon a wireless communication using Bluetooth; and a third communication may be determined based upon a wireless communication using ultrasonic communication (other combinations of same and/or different communication modalities are also within the scope of the present invention).
  • a geospatial location based upon trilateration may be generated based upon a controller receiving wireless indicators of distance and geometry of geometric shapes, such as circles, spheres, triangles and the like.
  • a geospatial location based upon multilateration may be generated based on a controller receiving a measurement of a difference in distance to two reference positions, each reference position being associated with a known location.
  • Wireless signals may be available at one or more of: periodically, within determined timespans, and continually.
  • the determination of the difference in distance between two reference positions provides multiple potential locations at the determined distance.
  • a controller (such as one in the Smart Device) may be used to generate a plot of potential locations.
  • the potential determinations generally form a curve. Specific embodiments will generate a hyperbolic curve.
  • the controller may be programmed to execute code to locate an exact position along a generated curve, which is used to generate a geospatial location.
  • the multilateration thereby receives as input multiple measurements of distance to reference points, wherein a second measurement taken to a second set of stations (which may include one station of a first set of stations) is used to generate a second curve. A point of intersection of the first curve and the second curve is used to indicate a specific location.
  • the distances may be triangulated based on measurements of UWB, Wi-Fi or sub GHz strength at two points.
  • Transceiver signals propagate outward as a wave, ideally according to an inverse square law.
  • a crucial feature of the present invention relies on measuring relative distances between two points. In light of the speed of Wi-Fi waves and the real-time computations involved in orienteering; these computations need to be as computationally simple as possible.
  • various coordinate systems may be desirable. In particular, if the Smart Device moves only in a planar direction while the elevation is constant, or only at an angle relative to the ground, the computation less complicated.
  • One exemplary coordinate system includes a polar coordinate system.
  • a three-dimensional polar coordinate system is a spherical coordinate system.
  • a spherical coordinate system typically comprises three coordinates: a radial coordinate, a polar angle, and an azimuthal angle (r, ⁇ , and ⁇ , respectively, though ⁇ and ⁇ are occasionally swapped conventionally).
  • Point 1 is considered the origin for a spherical coordinate system (i.e., the point (0, 0, 0)).
  • Each Transceiver 101 - 105 emitter e 1 , e 2 , e 3 can be described as points (r 1 , ⁇ 1 , ⁇ 1 ), (r 2 , ⁇ 2 , ⁇ 2 ), and (r 3 , ⁇ 3 , ⁇ 3 ), respectively.
  • Each of the r i 's (1 ⁇ i ⁇ 3) represent the distance between the Transceiver 101 - 105 emitter and the Transceiver 101 - 105 receiver on the Smart Device 101 or Smart Receptacle (see FIG. 5A ).
  • an azimuth may include an angle, such as a horizontal angle determined in an arcuate manner from a reference plane or other base direction line, such as an angle formed between a reference point or reference direction; and line (ray or vector) such as a ray or vector generated from or continuing to a Smart Device.
  • an angle such as a horizontal angle determined in an arcuate manner from a reference plane or other base direction line, such as an angle formed between a reference point or reference direction; and line (ray or vector) such as a ray or vector generated from or continuing to a Smart Device.
  • the ray or vector may be generally directed from a Reference Position Transceiver towards, and/or intersect one or more of: an item of interest; a point of interest; an architectural aspect (such as a wall, beam, header, corner, arch, doorway, window, etc.); an installed component that may act as a reference in an augmented virtual model (AVM) (such as, for example, an electrical outlet, a light fixture, a plumbing fixture, an architectural aspect; an item of equipment; an appliance; a multimedia device, etc.); another Reference Position Transceiver or other identifiable destination.
  • AVM augmented virtual model
  • a spherical coordinate system may include Reference Position Transceiver that is capable of determining an angle of departure of a location signal and a Transceiver that is capable of determining an angle of arrival of the location signal; one or both of which may be used to facilitate determination of an applicable azimuth.
  • an angle of departure and an angle of arrival may therefore be registered by a Transceiver that is transmitting and/or receiving wireless signals (e.g. radio frequency, UWB, Bluetooth 5.1, sonic frequency, or light frequency).
  • wireless signals e.g. radio frequency, UWB, Bluetooth 5.1, sonic frequency, or light frequency
  • locating an Agent occurs in or proximate to a Structure in which Reference Position Transceivers, (including, for example, one or more of: Wi-Fi Transceivers, UWB Transceivers, Bluetooth Transceivers, infrared Transceivers and ultrasonic Transceivers) may be located above and/or below an Agent.
  • Reference Position Transceivers including, for example, one or more of: Wi-Fi Transceivers, UWB Transceivers, Bluetooth Transceivers, infrared Transceivers and ultrasonic Transceivers
  • a cylindrical coordinate system may be more appropriate.
  • a cylindrical coordinate system typically comprises three coordinates: a radial coordinate, an angular coordinate, and an elevation (r, ⁇ , and z, respectively).
  • a cylindrical coordinate system may be desirable where, for example, all Wi-Fi emitters have the same elevation. Angles may be determined as described above.
  • transceivers 101 - 105 including arrays of antennas may be used to measure an angle of radio communication (e.g. angle of arrival and/or angle of departure).
  • an angle of radio communication e.g. angle of arrival and/or angle of departure.
  • Various configurations of transmitting antennas and receiving antennas may be used.
  • a radio transmission may be transmitted with a single antenna and received with a receiver with an array of antennas, the phase or timing difference of arriving signals can be used to calculate the angle at which the signals emerged.
  • a transmitter may contain an array of antennas and may send a pattern of signals through the array that arrive at a receiver with a single antenna where the angle of departure (AoD) is communicated.
  • AoD angle of departure
  • Measurement of angle of arrival may be performed as mentioned by calculation of time difference of arrival at the antennas in an array or alternatively can be performed by rotation of antenna elements.
  • Some modalities such as those modalities that adhere to the Bluetooth 5.1 or BL5.1 standards, allow a Smart Device 101 , Smart Receptacle or other Node to determine an angle of arrival (AoA) or an angle of departure (AoD) for a wireless transmission.
  • An array of antennas may be used to measure aspects of the Bluetooth signaling that may be useful to calculate these AoA and AoD parameters.
  • the system may be used to determine angles in one or two dimensions depending on the design of the antenna. The result may be significant improvement in pinpointing the location of origin of a signal.
  • An array of antennas may be positioned relative to each other and a transmitting transceiver to allow for extraction of an AoA/AoD.
  • Such an array may include a rectangular array; a polar or circular array; a linear array; and a patterned array, where a number of antennas are deployed in a pattern conducive to a particular environment for transceiving.
  • Antennas may be separated by characterized distances from each other, and in some examples, a training protocol for the antenna array results in antenna positioning incorporating superior angle and location precision.
  • Some transceivers may transceive in 2.4-2.482 GHz frequency bands, and thus the radiofrequency transmissions may have wavelengths in the roughly 125 mm length scale.
  • Transceivers 101 - 105 may include antenna arrays combined with batteries and circuitry to form complete self-contained devices. Antenna arrays and methods of using the same for determining position and direction of a Smart Device or other Node are described in U.S. Ser. No. 16/775,223, the contents of which are incorporated herein by reference.
  • an Agent supported transceiver 105 may be located at a position and may transmit a signal of the various types as have been described. Nodes, such as reference point transceivers 101 - 104 located at reference points in the wireless communication area around the position of the Agent 100 may receive the transmission and determine the angle of arrival of that transmission. Similarly, transmission associated with other transceivers 101 - 103 may also determine an angle of arrival of transmissions. In some embodiments, transceiver 101 - 103 may communicate with one or more of: each other, a smart device, a controller or other processor to mathematically process multiple angles and locations of the transceivers and calculate a position of a transmission emanation. In examples where calculations are not performed at a smart device, a communication to the smart device of the calculated position may be communicated.
  • a direction of interest indicated by Smart Device 101 or a Smart Receptacle may be determined based upon a movement of Smart Device 101 or a Smart Receptacle 502 .
  • a device with a controller and an accelerometer such as mobile Smart Device, may include a user display that allows a direction to be indicated by movement of the device from a determined location acting as a base position towards an As Built feature in an extended position.
  • the Smart Device may first determine a first position based upon triangulation with the reference points and a second position (extended position) also based upon triangulation with the reference points. These position determinations may proceed as described above.
  • the process of determination of a position based upon triangulation with the reference points may be accomplished, for example via executable software interacting with the controller in the Smart Device, such as, for example via running an app on the Smart Device.
  • Logical communications relevant to location determination may include, for example, one or more of: timing signals; SIM information; received signal strength; GPS data; raw radio measurements; Cell-ID; round trip time of a signal; phase; and angle of received/transmitted signal; time of arrival of a signal; a time difference of arrival; and other data useful in determining a location.
  • captured data may be compared to a library of stored data using image recognition software to ascertain and/or affirm a specific location, elevation and direction of an image capture location and proper alignment with the virtual model.
  • a position of a user may be determined by any of the means described herein.
  • a user may position a camera of an associated smart device to be pointing in a direction of interest and obtain an image.
  • the image may be passed on to a server with access to database of images containing stored images of the space around the user.
  • Algorithms on the server may compare the stored images to the image captured by the user, and may calculate adjustments to the comparative image based on where the reference image was taken in relationship to the location of the user.
  • a direction may be inferred with known location of objects in the reference image.
  • the differences in features of the user obtained image compared to a reference image may be used to calculate a direction of interest based upon a location at which the reference image was obtained.
  • stored images may be obtained at multiple angles to improve accuracy of orienteering. These examples may include sensor arrays, audio capture arrays and camera arrays with multiple data collection angles. In some examples a full 360 degree camera perspective may be obtained by such arrays. In some directional arrays, a sensor array (including image capture sensors) may include at least 120 degrees of data capture. By collecting such image collections as the sensor/camera systems are moved a database of image perspectives may be formed and utilized to assist in orienteering as described.
  • Non-limiting examples may include image based identification where a device with some imaging means, including but not limited to a mobile device camera, tablet device camera, computer camera, security camera, or AR headset camera may image points of interest in a direction of interest. These points of interest may be identified.
  • Image recognition software may be used to identify the visualized landscape by its identifying features.
  • Machine learning may be used to train systems using this software to identify specific features of the environment in question.
  • laser scanning and/or LiDAR may be performed during the acquisition of imagery for a reference database.
  • a resulting three dimensional shape model may be modelled with the captured imagery to help in the comparison to user data.
  • Three dimensional shape can be used to infer comparative imagery at different angles of acquisition than exist in a database.
  • a device of a user may have the means of performing laser scanning or LiDAR scanning of the environment as well as obtaining images.
  • the resultant three dimensional data or a composite of the three dimensional data and the imagery may be used to recognize features and determine a direction that the user was facing when they collected the image.
  • the results of scanning may be stored and presented in different manners.
  • the scanned data may be represented by a point cloud representation, in other examples an estimated topographic surface representation may be used to visualize the three dimensional shape data obtained.
  • outward facing planes of the surface topography may have the captured imagery superimposed upon them.
  • the resulting image and three dimensional models may be used to calculate a direction of interest or a device field of view in a dynamic sense or alternatively upon user request.
  • other methods of capturing spatially accurate information may include the use of drones and optical scanning techniques which may include high resolution imagery obtained from multiple viewpoints. Scanning may be performed with light based methods such as a CCD camera. Other methods may include infrared, ultraviolet, acoustic, and magnetic and electric field mapping techniques.
  • a single distance to a point of interest in an image which may be obtained by a laser, other collimated light source, sound source or the like, may be used with models of the environment of the user.
  • a comparison of the imagery and the measurement of the distance of the user to a prominent feature in the image may allow for an orientation of the user to be determined algorithmically.
  • radio frequencies used for wireless communication include sound waves used to perform one or more of: location determination; movement tracking in interior or exterior areas; and distance calculation.
  • Sound wave transmissions include a number of significant attributes, which may translate into a benefit for a given set of circumstances when used for RF based location determination.
  • sonic waves may be deployed independently, or in combination with electromagnetic transmissions and reception of logical communications utilizing other bandwidths, such as bandwidths associated with Ultrawideband, WiFi, Bluetooth, Ultrawideband, ANT, infrared or almost any wavelength in the Industrial, Scientific and Medical bands (sometimes referred to as “ISM Bands”).
  • bandwidths such as bandwidths associated with Ultrawideband, WiFi, Bluetooth, Ultrawideband, ANT, infrared or almost any wavelength in the Industrial, Scientific and Medical bands (sometimes referred to as “ISM Bands”).
  • the present invention also provides for sonic wave emanations may be used to compliment electromagnetic emanations based upon a tendency that sonic waves generally do not efficiently penetrate walls other physical items or structures.
  • Sonic transceivers may be particularly advantageous in a defined area where location can be unambiguously determined to be within a particular room (the use of multiple bandwidth transmitting and receiving for different purposes is further discussed below). Sound wave interaction with a solid surface, such as a wall, may require that for optimal performance, transceiver/transmitters/receivers to be located in each room where location detection is desired.
  • a reflected sonic transmission may be received and analyzed to determine an effect of the reflected nature of the transmission.
  • frequencies of effective indoor sonic location detection may be at ultrasonic bandwidths (commonly used bandwidths include frequencies of between 1 MHz and 10 MHz, although frequencies of less than 50 kHz to greater than 200 MHz are possible).
  • the utilized frequencies may be either below or above audible detection by people or other animals in the location; such as at frequencies above 20 kHz.
  • Sound waves may be used to perform one or more of: location determination, movement tracking in interior or exterior locations, and distance calculation from a position to a Smart Device 101 , which may be accomplished based upon transmission and receipt of sonic transmission.
  • Sound wave transmissions include several significant attributes, which may be beneficial for a given set of circumstances when used for radiofrequency-based location determination.
  • sonic waves may be deployed independently, or in combination with, transmissions and reception of logical communications utilizing other bandwidths, such as bandwidths associated with Wi-Fi, Bluetooth, ANT, infrared or almost any wavelength in the Industrial, Scientific and Medical bands (sometimes referred to as “ISM Bands”).
  • bandwidths such as bandwidths associated with Wi-Fi, Bluetooth, ANT, infrared or almost any wavelength in the Industrial, Scientific and Medical bands (sometimes referred to as “ISM Bands”).
  • a sonic Transceiver 200 may transmit a sonic transmission 202 and determine a location 201 based upon receiving an echo 203 back from an Agent supported transceiver 204 .
  • Walls 205 may also generate reflected sonic emanations 206 .
  • an Agent supported device 204 may support receivers, transmitters or transceivers that interact with ultrasonic transceivers fixedly secured to a reference point position, such as via mechanical mounting within a room environment.
  • An ultrasonic positioning system may have indoor positioning accuracy at centimeter, millimeter, and even sub millimeter accuracy.
  • Multiple ultrasonic Transceivers may transceive from an Agent supported device to communicate with fixed reference point transceivers may transmit signals. Arrival of the sound transmissions may be accurately timed and converted to distances.
  • distance determinations may be improved with knowledge of temperatures in the environment containing the sound transceiving. For example, temperature may be measured at one or more of: an Agent supported smart device, a Reference Point position, and an ambient environment.
  • synced timing apparatus is able to generate a location of a stationary Agent to within centimeter accuracy using sonic wave transmissions and reception and preferably within several millimeters of accuracy.
  • sensors are able to detect frequency shifts within the sonic emanations which may add information about the relative rate of movement of the Agent, which may then in turn allow for correction to the timing signals.
  • a combination of radio frequency emissions and ultrasonic emissions may be used. For example, a compliment of radio frequency emissions/receptions and ultrasonic of radio frequency emissions and ultrasonic emissions may be reconciled to generate more accurate location determination.
  • a radio frequency signal may be used to transmit syncing signals that establish a time that ultrasonic signals are transmitted. Since, the electromagnetic transmissions may be orders of magnitude faster than sound transmissions, the electromagnetic transmissions relatively small time of travel from the Transceivers to the Agent may be negligible and therefore used as “zero-time” setpoints as received at the Agent supported Transceiver.
  • a controller determining a location may use not only relative arrival times, but also a delta time between a radiofrequency transmission and ultrasonic transmission to determine a distance from a transmitting Transceiver.
  • An array of such ultrasonic and/or radiofrequency transceivers provide increased accuracy in triangulating a location of the Agent.
  • RF communications may not only transmit a syncing pulse, but also transmit digital data about various aspects of a defined area, such as the defined area's identification, its relative and/or absolute location in space and other refinements.
  • data related to improved distance calculation may also be transmitted by RF communication such as temperature of the environment, humidity and the like which may influence the speed of sound in the environment as a non-limiting example. In some examples such a system may result in millimeter level accuracy of position determination.
  • the process may be iterated to refine the direction of each of the ultrasonic transmitters and maximize signal levels of the transmission which may provide additional information in the calculation of a position.
  • RF and WiFi transmissions may be used for data communications and syncing as have been described.
  • an iterative process may be used to track the Agent supported device 204 moves through space. Stationary Agents may be tracked with submillimeter accuracy in some embodiments.
  • a User 300 may position a Transceiver 305 in a first position 301 proximate to a portion in a space of interest 325 .
  • the first position 301 of the Transceiver 305 may be determined and recorded.
  • the User 300 may then relocate the Transceiver 305 to a second position 302 in a general direction of the portion in space of interest.
  • a controller in an external system or in an associated Transceiver 305 may generate one or both of a ray and a vector towards the portion of a space of interest.
  • the vector may have a length determined by a controller that is based upon a distance to a feature of interest in space as represented in a model on the controller in the direction of the generated vector.
  • the vector may represent a distance 303 from the second position 302 to the space of interest 325 along the axis defined by a line between the first position 301 and the second position 302 .
  • a ray may include just a starting point and a direction.
  • a device with a controller and an accelerometer such as mobile phone, tablet or other Smart Device that includes a Transceiver 305
  • a user display that allows a direction to be indicated by movement of the device from a determined location acting as a base position towards an point in a direction of interest or representing a center of a field of view of the device. The movement may occur to a second location in an extended position.
  • the Smart Device determines a first position 301 based upon triangulation with the reference points. The process of determination of a position based upon triangulation with the reference points may be accomplished, for example via executable software interacting with a controller in the Smart Device, such as, for example by running an app on the Transceiver 305 .
  • FIGS. 4A-D a series of exemplary devices employing matrices (or arrays) of antennas for use with Nodes that communicate wirelessly, such as, via exemplary UWB, Sonic, Bluetooth, a Wi-Fi or other modality, is illustrated.
  • Linear antenna arrays 401 are illustrated in FIG. 4 A.
  • Rectangular antenna arrays 402 are illustrated in FIG. 4B .
  • Circular antenna arrays 403 are illustrated in FIG. 4C , other shapes for arrays are within the scope of the invention.
  • an antenna array may be omni-directional or directional.
  • a Smart Device 405 may include one or more Nodes 406 internal to the Smart Device 405 or fixedly attached or removably attached to the Smart Device 405 .
  • Each Node 406 may include antenna arrays combined with a power source and circuitry to form complete self-contained devices.
  • the Nodes 406 or a controller may determine an RTT, time of arrival, AoA and/or AoD or other related angular determinations based upon values for variables involved in wireless communications.
  • a composite device 404 may be formed when a Node 406 with a configuration of antennas, such as the illustrated exemplary circular configuration of antennas 407 , is attached to a Smart Device 405 .
  • the Node 406 attached to the Smart Device 405 may communicate information from and to the Smart Device 405 including calculated results received from or about another Node 406 , such as a Node 406 fixed as a Reference Point Transceiver or a Node with dynamic locations, wherein the wireless communications are conducive to generation of data useful for determination of a position (e.g. timing data, angles of departure and/or arrival, amplitude, strength, phase change, etc.).
  • a combination of angles from multiple fixed reference points to the antenna array can allow for a location of a user in space.
  • An array of antennas positioned at a reference point may allow for the accurate receipt of orientation information from a transmitter.
  • a combination devices with arrays of antennas may be used to calculation a position.
  • a single Node with an array of antennas can be used for orienteering and determining a direction of interest.
  • Each of the antennas in such an array receiving a signal from a source will have different phase aspects of the received signal at the antennas due to different distances that the emitted signal passes through. The phase differences can be turned into a computed angle that the source makes with the antenna array.
  • one or both of a Smart Device 501 and a Smart Receptacle 502 may incorporate multiple Transceivers 503 - 510 and a direction of interest may be ascertained by generating a vector 527 passing through a respective position of each of at least two of the transceivers (as illustrated through transceiver 505 and 507 ).
  • the respective positions of each of the transceivers 503 - 510 supported by the Smart Device 501 and/or Smart Receptacle 502 may be ascertained according to the methods presented herein, including for example via triangulation, trilateration, signal strength analysis, RTT, AoD, AoA, topography recognition.
  • one or both of the Smart Device 501 and the Smart Receptacle 502 incorporating Transceivers 503 - 510 may be rotated in a manner (such as, for example in a clockwise or counterclockwise movement 520 522 relative to a display screen) that repositions one or more Transceivers 503 - 510 from a first position to a second position.
  • a vector 526 may be generated at an angle that is zero degrees 524 with a plane of a display 515 or perpendicular 525 or some other designated angle in relation to the smart device 501 and an associated display screen 515 .
  • an angle in relation to the smart device is perpendicular 525 to a display screen 515 and thereby viewable via a forward looking camera (or other CCD or LIDAR device) on the smart device.
  • a mirror or other angle altering device may be used in conjunction with a CCD, LIDAR or other energy receiving device.
  • Movements of a Smart Device 501 equipped with an antenna array can be determined via relative positions of the antenna and/or via operation of an accelerometer 524 within the Smart Device 501 or Smart Receptacle 502 .
  • Rough movement sense may be inferred with a single source to the antenna array. However, with multiple sources, the positional movement of each of the antennas can be used to sense many types of movements including translations and rotations.
  • a user may position the smart device 501 such that an object in a direction of interest is within in the CCD view.
  • the smart device may then be moved to reposition one or more of the transceivers 503 - 510 from a first position to a second position and thereby capture the direction of interest via a generation of a vector in the direction of interest.
  • a magnetic force detection device 523 such as a magnetometer.
  • a registration of a magnetic force may be determined in relation to a particular direction of interest 524 - 525 and a subsequent determination of magnetic force referenced or provide a subsequent orientation of the Smart Device 501 or Smart Receptacle 502 .
  • the magnetic force detection device 523 may be used combination with, or in place of directional movement of the Smart Device transceivers 503 - 507 to quantify a direction of interest to a user.
  • Embodiments may include an electronic and/or magnetic sensor to indicate a direction of interest when a Smart Device 501 and/or Smart Receptacle 502 is aligned in a direction of interest. Alignment may include, for example, pointing a specified side of a Smart Device 501 and/or Smart Receptacle 502 , or pointing an arrow or other symbol displayed upon a user interface on the Smart Device 501 towards a direction of interest.
  • a magnetic force detection device 523 may detect a magnetic field particular to a setting that a Smart Device is located.
  • a particular structure or other setting may have a magnetic force that is primarily subject to the earth's magnetic field or may be primarily subject to electromagnetic forces from equipment, power lines, or some other magnetic influence or disturbance.
  • An initial quantification of a magnetic influence at a first instance in time may be completed and may be compared to a subsequent quantification of magnetic influence at a later instance in time. In this manner an initial direction of interest indicated by a position, orientation, pitch and yaw of a Node, such as a Smart Device may be compared to a subsequent position, orientation, pitch and yaw of the Smart Device.
  • an initial position, pitch and yaw of a Smart Device 501 may be described as a relative angle to a presiding magnetic force.
  • presiding magnetic forces include, magnetic influences of electrical charges, Earth's magnetic field, magnetized materials, permanent magnetic material, strong magnetic fields, ferromagnetism, ferrimagnetism, antiferromagnetism, paramagnetism, and diamagnetism, or electric fields that are generated at reference nodes at known positions which may be locally used to indicate a direction of interest.
  • Smart devices may include electronic magnetic sensors as part of their device infrastructure.
  • the magnetic sensors may typically include sensing elements deployed along three axis.
  • the magnetic sensors may be supplemented with electronic accelerometers, such as MEMS accelerometers.
  • the magnetic sensors may measure a sensed magnetic field perpendicular to the body of the sensor through a Hall Effect sensor.
  • a Hall Effect sensor may be built into silicon to generate a relatively sensitive sensing capability for magnetic fields.
  • electrons and holes flowing in a region of the silicon may interact with the regional magnetic field and build up on the fringes of the conduction region, thus generating a measurable voltage potential.
  • anisotropic magnetoresistance sensors may sensitively detect the magnetic field at the device as a significant change in resistance of structure elements in the device.
  • giant magnetoresistance (GMR) sensors may detect the magnetic field.
  • the GMR sensors may detect a magnetic field with a current-perpendicular-to-plane (CPP) GMR configuration.
  • CIP current in plane
  • the resulting three axis magnetic sensors may perform a sensitive compass function to determine a direction of a specified portion of the smart device and/or an edge of the smart device relative to the local magnetic field environment.
  • a specified portion of the Smart Device may be indicated via a user interface presented on a screen of the Smart Device.
  • a vector in a direction of interest 525 may be based upon a rocking motion 533 - 534 of the smart device 501 , such as a movement of an upper edge 518 in a forward arcuate movement 533 .
  • the lower edge 519 may also be moved in a complementary arcuate movement 534 or remain stationary.
  • the movement of one or both the upper edge 518 - 519 also results in movement of one or more transceivers 503 - 510 and/or registration in an onboard accelerometer 534 .
  • the movement of the transceivers 503 - 510 will preferably be a sufficient distance to register disparate geospatial positions based upon wireless transmissions and/or sufficient to register movement via the accelerometer 534 .
  • a direction dimension may be based upon one or more of: a wireless position of two or more transceivers, a movement of a device, a magnetic force determination, a LIDAR transmission and receiving, CCD energy determinations and other assessments of an environment containing the Smart Device and/or Smart Receptacle.
  • a device with a controller and an accelerometer such as mobile Smart Device, may include a user display that allows a direction to be indicated by movement of the device from a determined location acting as a base position towards a feature in the intended direction where the movement results in an extended position.
  • the Smart Device may first determine a first position based upon triangulation with the reference points and a second position (extended position) also based upon triangulation with the reference points.
  • facing a mobile device towards an area in a Structure and movement of the mobile device in a particular pattern may be used to ascertain a specific area in space to be interpreted by various methods.
  • FIG. 5C an illustration of an Agent 550 utilizing an oriented stereoscopic camera system 5551 to orient a direction of interest is shown.
  • the stereoscopic camera system 551 may obtain two different images from different viewpoints 552 - 553 which may be used to create topographical shape profiles algorithmically.
  • a controller may obtain the image and topographic data and algorithmically compare them to previously stored images and topographic data in the general environment of the user. The resulting comparison of the imagery and topography may determine an orientation in space of the Agent 501 and in some examples determine a device field of view. The controller may utilize this determined field of view for various functionality as described herein.
  • controller 600 may be included in one or more of: a wireless tablet or handheld device, a server, a rack mounted processor unit.
  • the controller may be included in one or more of the apparatus described above, such as a Smart Device, Smart Tablet, Headgear, Smart Watch Smart Ring or other Smart Device.
  • the controller 600 includes a processor unit 620 , such as one or more semiconductor based processors, coupled to a communication device 610 configured to communicate via a communication network (not shown in FIG. 6 ).
  • the communication device 610 may be used to communicate, for example, via a distributed network such as a cellular network, an IP Protocol network, the Internet or other distributed logic communication network.
  • the processor 620 is also in communication with a storage device 630 .
  • the storage device 630 may comprise any appropriate information storage device, including combinations of digital data storage devices (e.g., Solid State Drives and hard disk drives), optical storage devices, and/or semiconductor memory devices such as Random Access Memory (RAM) devices and Read Only Memory (ROM) devices.
  • the storage device 630 can store a software program 640 with executable logic for controlling the processor 620 .
  • the processor 620 performs instructions of the software program 640 , and thereby operates in accordance with the present invention.
  • the processor 620 may also cause the communication device 610 to transmit information, including, in some instances, control commands to operate apparatus to implement the processes described above.
  • the storage device 630 can additionally store related data in a database 650 and database 660 , as needed.
  • the smart device 702 comprises an optical capture device 708 to capture an image and convert it to machine-compatible data, and an optical path 706 , typically a lens, an aperture or an image conduit to convey the image from the rendered document to the optical capture device 708 .
  • the optical capture device 708 may incorporate a Charge-Coupled Device (CCD), a Complementary Metal Oxide Semiconductor (CMOS) imaging device, or an optical Sensor 724 of another type.
  • CCD Charge-Coupled Device
  • CMOS Complementary Metal Oxide Semiconductor
  • a microphone 710 and associated circuitry may convert the sound of the environment, including spoken words, into machine-compatible signals.
  • Input facilities may exist in the form of buttons, scroll wheels, or other tactile Sensors such as touch-pads.
  • input facilities may include a touchscreen display.
  • Audible feedback 734 may come from a loudspeaker or other audio transducer.
  • Tactile feedback may come from a vibrate module 736 .
  • a magnetic force sensor 737 such as a Hall Effect Sensor, solid state device, MEMS device or other silicon based or micro-electronic apparatus.
  • a motion Sensor 738 and associated circuitry converts motion of the smart device 702 into a digital value or other machine-compatible signals.
  • the motion Sensor 738 may comprise an accelerometer that may be used to sense measurable physical acceleration, orientation, vibration, and other movements.
  • motion Sensor 738 may include a gyroscope or other device to sense different motions.
  • a location Sensor 740 and associated circuitry may be used to determine the location of the device.
  • the location Sensor 740 may detect Global Position System (GPS) radio signals from satellites or may also use assisted GPS where the mobile device may use a cellular network to decrease the time necessary to determine location.
  • GPS Global Position System
  • the location Sensor 740 may use radio waves to determine the distance from known radio sources such as cellular towers to determine the location of the smart device 702 . In some embodiments these radio signals may be used in addition to GPS.
  • the smart device 702 comprises logic 726 to interact with the various other components, possibly processing the received signals into different formats and/or interpretations.
  • Logic 726 may be operable to read and write data and program instructions stored in associated storage or memory 730 such as RAM, ROM, flash, SSD, or other suitable memory. It may read a time signal from the clock unit 728 .
  • the smart device 702 may have an on-board power supply 732 . In other embodiments, the smart device 702 may be powered from a tethered connection to another device or power source.
  • the smart device 702 also includes a network interface 716 to communicate data to a network and/or an associated computing device.
  • Network interface 716 may provide two-way data communication.
  • network interface 716 may operate according to the internet protocol.
  • network interface 716 may be a local area network (LAN) card allowing a data communication connection to a compatible LAN.
  • network interface 716 may be a cellular antenna and associated circuitry which may allow the mobile device to communicate over standard wireless data communication networks.
  • network interface 716 may include a Universal Serial Bus (USB) to supply power or transmit data. In some embodiments other wireless links may also be implemented.
  • USB Universal Serial Bus
  • a reader may scan some coded information from a location marker in a facility with the smart device 702 .
  • the coded information may include for example, a hash code, bar code, RFID or other data storage device.
  • the scan may include a bit-mapped image via the optical capture device 708 .
  • Logic 726 causes the bit-mapped image to be stored in memory 730 with an associated time-stamp read from the clock unit 728 .
  • Logic 726 may also perform optical character recognition (OCR) or other post-scan processing on the bit-mapped image to convert it to text.
  • OCR optical character recognition
  • Logic 726 may optionally extract a signature from the image, for example by performing a convolution-like process to locate repeating occurrences of characters, symbols or objects, and determine the distance or number of other characters, symbols, or objects between these repeated elements. The reader may then upload the bit-mapped image (or text or other signature, if post-scan processing has been performed by logic 726 ) to an associated computer via network interface 716 .
  • a reader may recite words to create an audio file by using microphone 710 as an acoustic capture port.
  • Logic 726 causes audio file to be stored in memory 730 .
  • Logic 726 may also perform voice recognition or other post-scan processing on the audio file to convert it to text.
  • the reader may then upload the audio file (or text produced by post-scan processing performed by logic 726 ) to an associated computer via network interface 716 .
  • a directional sensor 741 may also be incorporated into the smart device 702 .
  • the directional device may be a compass and be based upon a magnetic reading, or based upon network settings.
  • the magnetic sensor may include three axes of magnetic sensitive elements and may also be coupled with an accelerometer in the directional sensor 741 .
  • a LiDAR sensing system 751 may also be incorporated into the smart device 702 .
  • the LiDAR system may include a scannable laser light (or other collimated) light source which may operate at nonvisible wavelengths such as in the infrared.
  • An associated sensor device, sensitive to the light of emission may be included in the system to record time and strength of returned signal that is reflected off of surfaces in the environment of the smart device 702 .
  • FIGS. 8A-8C a support 800 for a smart device 801 is illustrated.
  • the support remains stationary in relation to a ground plane.
  • One or more position devices 803 - 808 are shown located within, on or proximate to the smart device 801 .
  • FIG. 8A generally linear movement 814 - 815 from a first position to a second position is illustrated.
  • a cessation of movement in a general direction is determined via an accelerometer included in or operated by the smart device 801 .
  • the support 800 may activate a user interactive device, such as a button on a touch screen, or a switch to indicate one or both of the first position and the second position.
  • the transceivers 803 - 808 enter into logical communication with multiple wireless positional reference transceivers 810 - 813 .
  • a direction of interest will include an item of interest 809 , such as an apparatus or other piece of equipment.
  • a direction of interest 814 may include a vector with a direction pointing towards the item of interest 809 .
  • the vector length will be sufficient to reach the item of interest 809 .
  • a vector indicating a direction of interest 814 may be used to reference stored data, such as, for example, digital data stored in a database, an AVM, URL, cloud storage or other storage medium.
  • a user interface may include a drop down menu that includes potential items of interest 809 in a general direction of the vector.
  • a selection of an item of interest may be used to facilitate determination of a length of the vector 814 .
  • a movement of a smart device 801 may be arcuate in nature 814 C so long as arcuate movement 814 C results in sufficient distance of movement of one or more position devices 803 - 808 .
  • geospatial location services are used to determine geospatial location such as a location of the structure with a position and direction of interest. Geospatial services may be used to determine a user's location relative to the structure and directions thereto.
  • the methods used may include, by way of non-limiting example, one or more of: satellite-based global positioning systems (GPS), cell tower triangulation, radio signal triangulation, Wi-Fi signal location services, infrared transmitters and the like.
  • GPS global positioning systems
  • cell tower triangulation cell tower triangulation
  • radio signal triangulation radio signal triangulation
  • Wi-Fi signal location services infrared transmitters and the like.
  • Geospatial location services will be cross-referenced with database registry of as built virtually modeled facilities and may be used in conjunction with a network of registered service technicians to route the nearest available service technician to the structure experiencing equipment area of interest. Service technician may register with the system to accept geospatial location tracking services by the system.
  • the service technician's entry into the structure will be registered. Registration of entry into the structure may be achieved through multiple methods, which may include, by way of non-limiting example, on or more of: WiFi gateway detection, infrared detection, magnetic door locking systems, Bluetooth services, and the like.
  • system Upon entry into the structure requesting the service call, system will register the service technician's entry into the structure.
  • a support unit for a smart device such as service technician or an unmanned vehicle may be tacked via a change in triangulation values and/or an accelerometer and a position and direction within the structure is tracked.
  • the methods used may be, by means of non-limiting example, one or more of: use of data gleaned from accelerometers located on or in possession of service technicians, Wifi services, radio frequency (RF) triangulation, Bluetooth technology, infrared detection, RFID badges, and the like.
  • RF radio frequency
  • a smart device will be registered as entering within structure. Following the smart device entry into structure.
  • a smart device may be associated with one or both of a person and an entity.
  • the smart device is pre-registered by the system with detailed instructions regarding a reason for the device to be at a particular location.
  • the reason may be, for example, one or more of: a service call placed from structure to system detailing current equipment area of interest, service calls from structure detailing non-specific area of interests and symptomatic data indicating equipment area of interest, a service call placed by self-assessing equipment utilizing internet of things (IoT) and machine learning functionality to ascertain area of interests and predictive analytics to anticipate area of interests, and the like.
  • IoT internet of things
  • the system may integrate data reports into the AVM and relay as much to the smart device in the field.
  • the smart device may arrive at the structure without prior knowledge of a purpose.
  • system Upon entry into the structure and registration of the smart device as described in method steps 901 through 904 , system will relay data gleaned from the AVM, operational data uploaded to the system through IoT processes, and other experiential data reported to the system and thereby relayed to the smart device on site.
  • Methods for relation of such data to the onsite smart device may include, by means of non-limiting example, referential data based on proprietary orienteering processes to determine smart device location within structure, which location will be cross-referenced with AVM data.
  • a position within or proximate to the structure may be determined via positioning identifiers.
  • the position within or proximate to the structure is determined and detailed instructions directing smart device to the source of an area of interest is relayed by the system to the smart device directly or by means of smart device application.
  • the methods used may be, by means of non-limiting example, one or more of: augmented reality overlays displayed on heads-up displays or other wearable technologies, augmented reality overlays displayed on smart devices, direct instructional vectoring provided to the smart device by the system over Wifi internet connection or LTE signal, virtual reality walkthrough instructions provided to smart device on site or prior to arrival at the structure, updated map/schematic displays detailing the structure and directing the smart device to the source of the subject area of interest by means of vectoring and orienteering processes.
  • a smart device's location within the structure along an XY axis will be tracked and recorded by the system by means of fixed or adaptive orienteering apparatus within the structure.
  • orienteering apparatus may include, by means of non-limiting example, on or more of: radio frequency (e.g. UWB, WiFi, sub GHz, etc.) triangulation, infrared position detection, radio frequency (RF) detection, RF ID tracking, onboard accelerometers located on the smart device or carried smart devices, quantification of magnetic fields, topographical pattern recognition, and the like.
  • the smart device's location within the structure along the Z axis will be determined.
  • the methods used may be, by means of non-limiting example, one or more of: Smart Device integrated magnetometers, Smart Device integrated barometers, Smart Device integrated accelerometers, Smart Device integrated tilt sensors, used in conjunction with in-structure XY axis position processes described in method step 097 above, along with data detailed in a database or model of the structure.
  • Method steps 901 through 909 work in conjunction to track and direct the smart device to an area of interest; as the smart device approaches an area of interest, the smart device will be oriented to the area of interest.
  • the system will determine the smart device's direction of interest using, by means of non-limiting example, one or more of the following methods: generation of a vector based upon multiple locations of transceiver(s), wherein the transceiver locations may be associated with a single transceiver at two different locations, or disparate transceivers located at respective locations; a transceiver location and an AoA or AoD combined with a distance; infrared pointers, laser direction finding devices, Smart Device integrated CCDs; LIDAR Wifi triangulation, and the like.
  • the smart device's distance to the subject area of interest will be determined.
  • the methods used may be, by means of non-limiting example, one or more of: time of flight of a transmitted signal, RTT, phase differentiation, depth orientation of the smart device 1001 .
  • One or more position devices 1003 - 1008 are shown located within, on or proximate to the smart device 1001 .
  • the smart device 1001 may include an electronic magnetic sensor 1050 .
  • the electronic magnetic sensor 1050 may quantify an ambient magnetic field and generate an orientation of the smart device 1001 along three axis of the ambient magnetic field that correlates with the quantified ambient magnetic field.
  • a direction of interest 1014 may be quantified in terms of an angle relative to the three axis of the ambient magnetic field.
  • the transceivers 1003 - 1008 enter into logical communication with multiple wireless positional reference transceivers 1010 - 1013 to generate data sufficient to quantify a physical position of the transceivers 1010 - 1013 and an orientation of the smart device.
  • a direction of interest will permit identification of an item of interest 1009 , such as an apparatus or other piece of equipment or a direction into the environment.
  • a direction of interest 1014 may include a vector with a direction pointing towards the area of interest 1009 .
  • the vector length may be sufficient to reach the item of interest 1009 .
  • an item may be assigned a “Virtual Tag,” in which a Smart Device can identify an object in its field of view and capture its location coordinates based on the position of the Smart Device.
  • a “Virtual Tag” or “icon” may be generated in the Smart Device's field of view at those coordinates and saved in the digital memory. Information regarding the object may also be saved with the spatial coordinates.
  • This information can be transmitted to the monitoring system so that others may find the object by routing to the particular coordinates.
  • the Smart Device when that Smart Device, or another Smart Device in logical communication with the monitoring system, views the particular location, the Smart Device will identify that a virtual tag exists at that location with, for example, an icon. The icon may also be interactive so that information associated with the object may be known.
  • the monitoring system may be in logical communication with both physical tags such as Smart Devices or Nodes and with Virtual Tags created by a Smart Device. This will be described in further detail below.
  • Creating of Virtual Tags may, in some embodiments, performed using the following steps: in a first step 1101 , an orientation home position is established.
  • the Smart Device initiates transceiving and at step 1103 , a coordinate system is selected. If Cartesian coordinate system is selected, at step 1104 , the Smart Device may triangulate with Reference Point Transceivers/Nodes to determine the position of the Image Capture Device in the Smart Device (step 1106 ). If a polar coordinate system is selected, at Step 1505 , an angle and distance involved in transceiving is measured, and the camera position is determined (step 1106 ).
  • the Image Capture Device may be a camera, including a charge-coupled device (CCD), or a LIDAR apparatus, including a solid-state LIDAR or a MEMS-based LIDAR.
  • step 1107 the position of any Nodes or Reference Point Transceivers is verified, and in step 1108 , the direction of the Image Capture Device is determined based on the methods described herein, using the Smart Device and any Nodes or Reference Point Transceivers nearby.
  • the Image Capture Device captures an image. This may be done by capture of wireless energy of a first wavelength is received into a wireless receiver in the Smart Device. In exemplary embodiments, this step may comprise receiving image data based on visible light into a camera of the smart device.
  • the wireless energy may be dispersed over a one-, two-, or three-dimensional space in a defined physical area, and may be received into a one-, two-, or three-dimensional array in the receiver.
  • the wireless energy may take the form of electromagnetic radiation, such as light in the human-visible light spectrum (generally having a wavelength between 380 nm-740 nm), ultraviolet light (generally having a wavelength between 10.0 nm-400 nm), or infrared light (generally having a wavelength between 740 nm-2.00 mm).
  • the set of wireless energy available to the wireless receiver is the smart device's Field of View. In step 1110 , the Field of View may change as the angle or position of the Smart Device changes.
  • the Field of View is quantified.
  • a pattern of digital values is generated based upon receipt of wireless energy into the wireless receiver. This pattern of digital values may be based on one or more qualities of the received wireless energy, including its intensity, spatial dispersion, wavelength, or angle of arrival. The pattern may be placed into an appropriate array. For example, if the display of the smart device is a two-dimensional display, then the pattern of digital values may comprise a two-dimensional representation of the image data received. In some embodiments, the pattern of digital values may be based on an aggregated set of values from an array of receivers.
  • the digital value assigned to a given entry in the array may be based on a weighted average of intensity of wireless energy received at a plurality of the receivers in the array.
  • the set of digital values within the Field of View is the Digital Field of View.
  • the coordinates of the space in the Digital Field of View may also be determined. Any suitable method may be used, but in some embodiments, LIDAR may be used to determine the presence and distance from the Smart Device of various items in the Field of View and therefore determine the position of items.
  • items or space in the Field of View may be “tagged” such as via a touch screen or cursor.
  • the location coordinates for the Virtual Tags may be known and transmitted, for example, to a monitoring system or controller.
  • step 1111 when a Smart Device that is part of the monitoring system approaches the location of the Virtual Tag, the Smart Device will identify that the Virtual Tag is within the Smart Device's Field of View. In step 1115 , when this occurs, the Smart Device will generate an icon in the position of the Virtual Tag. This icon may be interactive so that information regarding the location or Virtual Tag may be ascertained. For example, the Virtual Tag may provide a location and environmental history of the product at the coordinates, as will be discussed in further detail below.
  • Agent refers to a person or automation capable of supporting a Smart Device at a geospatial location relative to a Ground Plane.
  • Ambient Data refers to data and data streams captured in an environment proximate to a Vantage Point and/or an equipment item that are not audio data or video data.
  • Ambient Data include, but are not limited to Sensor perception of: temperature, humidity, particulate, chemical presence, gas presence, light, electromagnetic radiation, electrical power, moisture and mineral presence.
  • Analog Sensor and “Digital Sensor” as used herein include a Sensor operative to quantify a state in the physical world in an analog representation.
  • As Built refers to details of a physical structure associated with a specific location within the physical structure or parcel and empirical data captured in relation to the specific location.
  • As Built Features refers to a feature in a virtual model or AVM that is based at least in part upon empirical data captured at or proximate to a correlating physical location of the feature.
  • Examples of As Built Features include placement of structural components such as a wall, doorway, window, plumbing, electrical utility, machinery and/or improvements to a parcel, such as a well, septic, electric or water utility line, easement, berm, pond, wet land, retaining wall, driveway, right of way and the like.
  • Augmented Virtual Model (sometimes referred to herein as “AVM”): as used herein is a digital representation of a real property parcel including one or more three dimensional representations of physical structures suitable for use and As Built data captured descriptive of the real property parcel.
  • An Augmented Virtual Model includes As Built Features of the structure and may include improvements and features contained within a Processing Facility.
  • Bluetooth as used herein means the Wireless Personal Area Network (WPAN) standards managed and maintained by Bluetooth SIG. Unless otherwise specifically limited to a subset of all Bluetooth standards, the Bluetooth will encompass all Bluetooth standards (e.g. Bluetooth 4.0; 5.0; 5.1 and BLE versions).
  • Directional Indicator shall mean a quantification of a direction generated via one or both of: analogue and digital indications.
  • Directional Image Data refers to image data captured from a Vantage Point with reference to a direction.
  • Image data may include video data.
  • Directional Audio refers to audio data captured from a Vantage Point within or proximate to a property and from a direction.
  • Digital Sensor as used herein includes a Sensor operative to quantify a state in the physical world in a digital representation.
  • Ground Plane refers to horizontal plane from which a direction of interest may be projected.
  • Image Capture Device or “Scanner” as used herein refers to apparatus for capturing digital or analog image data
  • an Image capture device may be one or both of: a two dimensional camera (sometimes referred to as “2D”) or a three dimensional camera (sometimes referred to as “3D”).
  • an Image Capture Device includes a charged coupled device (“CCD”) camera.
  • Intelligent Automation as used herein refers to a logical processing by a device, system, machine or equipment item (such as data gathering, analysis, artificial intelligence, and functional operation) and communication capabilities.
  • MultiModal refers to the ability of a device to communication using multiple protocols and/or bandwidths. Examples of multimodal may include being capable of communication using two to more of: Ultra Wideband, Bluetooth; Bluetooth Low Energy; WiFi; WiFi RT; GPS; ultrasonic; infrared protocols and/or mediums.
  • Node as used herein means a device including at least a processor, a digital storage and a wireless transceiver.
  • Proximate as used herein means physically located within range of a relevant modality of wireless communication.
  • “Sensor” as used herein refers to one or more of a solid state, electro-mechanical, and mechanical device capable of transducing a physical condition or property into an analogue or digital representation and/or metric.
  • Smart Device as used herein includes an electronic device including, or in logical communication with, a processor and digital storage and capable of executing logical commands.
  • Smart Receptacle as used herein includes a case or other receiver of a smart device with components capable of receiving wireless transmissions from multiple wireless positional reference transceivers.
  • the smart receptacle will include a wireless transmitter and/or a physical connector for creating an electrical path for carrying one or both of electrical power and logic signals between an associated Smart Device and the Smart Receptacle.
  • “Structure” as used herein refers to a manmade assembly of parts connected in an ordered way. Examples of a Structure in this disclosure include a building; a sub-assembly of a building; a bridge, a roadway, a train track, a train trestle, an aqueduct; a tunnel a dam, and a retainer berm.
  • Transceive refers to an act of at least one of: transmitting and receiving data.
  • Transceiver refers to an electronic device capable of one or both of wirelessly transmitting and receiving data.
  • Ultra Wideband sometimes referred to herein as “UWB” means a wireless communication modality which may engage a bandwidth generally greater than 500 MHz and transmit wireless energy in pulses.
  • Vantage Point refers to a specified location which may be an actual location within a physical environment or a virtual representation of the actual location within a physical environment.
  • Vector refers to a magnitude and a direction as may be represented and/or modeled by a directed line segment with a length that represents the magnitude and an orientation in space that represents the direction.
  • Virtual Tag as used here shall mean a reference point identified with a location in a virtual model of a space. It may have common aspects with an artificial reality anchor, reference anchor and the like. The tag may have numerous features and content associated with it including data values, user functionality, image data, video data and the like.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Remote Sensing (AREA)
  • General Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Theoretical Computer Science (AREA)
  • Geometry (AREA)
  • Computer Hardware Design (AREA)
  • Software Systems (AREA)
  • Computer Graphics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Business, Economics & Management (AREA)
  • General Business, Economics & Management (AREA)
  • Architecture (AREA)
  • Civil Engineering (AREA)
  • Structural Engineering (AREA)
  • Computational Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Pure & Applied Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Automation & Control Theory (AREA)
  • Position Fixing By Use Of Radio Waves (AREA)

Abstract

Apparatus and methods for enhanced wireless determination of a position and direction of a smart device are describe which support the display of a virtual tag upon a user interface of the smart device. Wireless transceivers controlled by the smart device communicate with reference point transceivers to generate data sufficient to determine relative positions of the wireless transceivers and a direction of interest. Operation of LIDAR may be operative to verify the position and direction of the Smart Device as well as a topography of the environment.

Description

CROSS REFERENCE TO RELATED APPLICATIONS
The present application claims priority to Non Provisional patent application Ser. No. 17/183,062, filed Feb. 23, 2021 and entitled METHOD AND APPARATUS FOR ENHANCED POSITION AND ORIENTATION BASED INFORMATION DISPLAY, which in turn claims Non Provisional patent application Ser. No. 16/898,602, filed Jun. 11, 2020 and entitled METHOD AND APPARATUS FOR ENHANCED POSITION AND ORIENTATION DETERMINATION which in turn claims the benefit of Provisional Patent Application Ser. No. 62/769,133, filed Nov. 19, 2018 and entitled METHODS AND APPARATUS FOR ORIENTEERING; and to Non Provisional patent application Ser. No. 16/142,275, filed Sep. 26, 2018 and entitled METHODS AND APPARATUS FOR ORIENTEERING; and to Non Provisional patent application Ser. No. 16/503,878, filed Jul. 5, 2019 and entitled METHOD AND APPARATUS FOR ENHANCED AUTOMATED WIRELESS ORIENTEERING; and to Non Provisional patent application Ser. No. 16/688,775, filed Nov. 19, 2019, entitled METHOD AND APPARATUS FOR WIRELESS DETERMINATION OF POSITION AND ORIENTATION OF A SMART DEVICE and issued on Apr. 21, 2020 as U.S. Pat. No. 10,628,617; and to Non-Provisional patent application Ser. No. 16/721,906, filed Dec. 19, 2019 and entitled METHOD AND APPARATUS FOR DETERMINING A DIRECTION OF INTEREST; and to Non-Provisional patent application Ser. No. 16/775,223, filed Jan. 29, 2020 and entitled SPATIAL SELF-VERIFYING ARRAY OF NODES; and to Non-Provisional patent application Ser. No. 16/817,926, filed Mar. 13, 2020 and entitled APPARATUS FOR OPERATION OF CONNECTED INFRASTRUCTURE; the contents of each of which are relied upon and incorporated herein by reference.
FIELD OF THE INVENTION
The present invention relates to methods and apparatus for enhanced determination of a direction of interest based upon orientation of an apparatus. More specifically, the present invention presents methods and apparatus for indicating a direction based upon wireless communications between transceivers and using that to guide information display.
BACKGROUND OF THE INVENTION
It is known for a geospatial position to be ascertained based upon triangulation techniques. Such triangulation techniques may be based upon artificial location references, such as satellites and/or cell towers. However, location determination within a structure is limited due to variations in communication signal strength within a structure. In addition, such positioning systems lack an accurate functionality for determining a direction of interest to a user.
SUMMARY OF THE INVENTION
Accordingly, the present invention combines methods and apparatus for enhanced determination of a position of a Smart Device and one or both of a direction of interest of an Agent and a direction of orientation of a Smart Device. The enhanced determination of a position of the Smart Device; and one or both of a direction of interest of an Agent and a direction of orientation of a Smart Device may be accomplished via registration and storage of aspects of an environment surrounding the Smart Device at a first instance in time. The registered aspects are subsequently referenced and compared to registration and storage of similar (or same) aspects at a second instance in time. The comparison of the registered aspects may be combined with wireless communications to provide enhanced location and orientation of the Smart Device. In some embodiments, enhanced location and orientation may be augmented via operation of devices for determining physical attitude of the Smart Device (e.g. pitch and yaw and bank, typically denoted as typically denoted as α, β, γ, or φ, θ, ψ), wherein a change in pitch, yaw or bank may be tracked from an original position and orientation of the Smart Device to calculate a subsequent position and orientation of the Smart Device
Aspects of an environment may include, for example, one or more of physical features in a proximate area to a Smart Device or other Node that are quantifiable via: sonic (including ultrasonic) waves; LIDAR measurements; magnetic force readings and visual light wavelength measurements.
Instruments useful in determining physical features in proximity to a Smart Devices may include, for example, one or more of: accelerometers, charged coupled devices (CCD); LiDAR transmitters and receivers; enhanced magnetic sensors, magneto-resistive sensors, Hall Effect sensors or transducers, MEMS (micro electromechanical sensor) field sensor, anisotropic magneto resistive effect transducers, and giant magneto resistive effect transducers or other mechanism to measure a magnetic field. Location determination may be performed using combinations of positionally fixed Nodes communicating via wireless communication devices and protocols.
The present invention provides for multiple complimentary ways for determining an orientation of a smart device relative to a structure based upon supporting a first transceiver and a second transceiver via an agent at a first geospatial position within radio communication distance to transceivers located within the structure and generating respective logical communications between the first transceiver and each of a first set of three reference point transceivers, the three reference point transceivers fixedly positioned at disparate points in the structure.
A controller may execute software code to determining X, Y and Z coordinates of the first transceiver based upon the respective logical communications between the first transceiver and each of a first set of three reference point transceivers and generate respective logical communications between the second transceiver and each of a second set of three reference point transceivers. The controller may also determine X, Y and Z coordinates of the second transceiver based upon the respective logical communications between the first transceiver and each of a first set of three reference point transceivers.
A magnetic field sensor may generate a first quantification of a magnetic field encompassing the smart device such that a location of the smart device is generated based upon at least one of: X, Y and Z coordinates of the first transceiver, and the X, Y and Z coordinates of the second transceiver. In addition, a first direction of interest may be generated based upon at least two of: the X, Y and Z coordinates of the first transceiver, the X, Y and Z coordinates of the second transceiver, and the quantification of the magnetic field encompassing the smart device.
Data associated with the location of the smart device and the first direction of interest; may be accessed and a user interface may be generated that includes the data associated with the location of the smart device and/or a direction of interest.
In some embodiments, the smart device may be reoriented and a second quantification of the magnetic field encompassing the smart device may be quantified such that a second direction of interest may be generated based upon the second quantification of the magnetic field relative to the smart device.
In some embodiments, the magnetic field sensor may include a Hall Effect sensor and the transceivers may be embodied in one or more of: a smart watch, a smart ring, and a smart wand.
In some embodiments, the smart device may be reoriented and a topographical representation of at least a portion of an environment within an area capable of wireless communication the smart device may be generated, such as, for example using LIDAR or sonic measurements, and generating a second direction of interest based upon the topographical representation.
Apparatus for determining an orientation relative to a structure may include: a controller comprising a processor and a digital storage, said controller in logical communication with a user interface display; a first transceiver supportable by an Agent at a first geospatial position and in logical communication with the controller; and a second transceiver supportable by an Agent at the first geospatial position and in logical communication with the controller; a magnetic field sensor in logical communication with the controller; and executable code stored on the digital storage and executable by the processor to cause the controller to perform the method steps involved in generating a position and direction of interest.
More specifically the controller may be operative to: generate respective logical communications between the first transceiver and each of a first set of three reference point transceivers, the three reference point transceivers fixedly positioned at disparate points in the structure; determine X, Y and Z coordinates of the first transceiver based upon the respective logical communications between the first transceiver and each of a first set of three reference point transceivers; generate respective logical communications between the second transceiver and each of a second set of three reference point transceivers; determine X, Y and Z coordinates of the second transceiver based upon the respective logical communications between the first transceiver and each of a first set of three reference point transceivers; generate a first quantification of a magnetic field encompassing the smart device with a magnetic field sensor; generate a location of the smart device based upon at least one of: X, Y and Z coordinates of the first transceiver, and the X, Y and Z coordinates of the second transceiver; generate a first direction of interest based upon at least two of: the X, Y and Z coordinates of the first transceiver, the X, Y and Z coordinates of the second transceiver, and the quantification of the magnetic field encompassing the smart device; access data associated with the location of the smart device and the first direction of interest; and generate a user interface on the smart device, the user interface comprising the data associated with the location of the smart device.
In addition, in some embodiments, the controller may generate a second quantification of the magnetic field encompassing the smart device following a reorientation of the smart device; and generate a second direction of interest based upon the second quantification of the magnetic field relative to the smart device.
The details of one or more examples of the invention are set forth in the accompanying drawings and the description below. The accompanying drawings that are incorporated in and constitute a part of this specification illustrate several examples of the invention and, together with the description, serve to explain the principles of the invention: other features, objects, and advantages of the invention will be apparent from the description, drawings, and claims herein.
DESCRIPTION OF THE DRAWINGS
The accompanying drawings, that are incorporated in and constitute a part of this specification, illustrate several embodiments of the invention and, together with the description, serve to explain the principles of the invention:
FIG. 1 illustrates location determination with wireless communication to reference point transceivers.
FIG. 2 illustrates location determination using sonic transceivers.
FIG. 3 illustrates methods of orienteering by device movement.
FIGS. 4A-4D illustrate exemplary configurations of antenna arrays.
FIG. 5A illustrates an exemplary smart device with an array of antennas.
FIG. 5B illustrates a direction determination of a smart device.
FIG. 5C illustrates embodiments of the invention that include stereoscopic cameras.
FIG. 6 illustrates apparatus that may be used to implement aspects of the present invention including executable software.
FIG. 7 illustrates an exemplary smart device that may be used to implement aspects of the present invention including executable software.
FIG. 8A-8C illustrate movements for determining a direction of interest of a smart device.
FIG. 9 illustrates method steps that may be implemented according to some aspects of the present invention.
FIG. 10 illustrates an Agent supporting a smart device.
FIG. 11 illustrates method steps that may be implemented in some embodiments of the present invention.
DETAILED DESCRIPTION
The present invention relates to methods and apparatus for enhanced wireless determination of a position and direction of a smart device. Wireless transceivers controlled by the smart device communicate with reference point transceivers to generate data sufficient to determine relative positions of the wireless transceivers and a direction of interest. Operation of one or both of a magnetic force sensor and LIDAR may be operative to verify the position and direction of the Smart Device.
Referring now to FIG. 1, aspects of a system for enhanced wireless position and orientation are illustrated. Reference Point Transceivers 101-104 are shown deployed within or proximate to a Structure 106, to determine a location 107 of a Transceiver 105 supported an Agent 100 within or proximate to the Structure 106. Reference Point Transceivers 101-104 may be fixed in a certain location within or proximate to the Structure 106 and define a wireless communication area 111. The Reference Point Transceivers 101-104 may transceive in a manner suitable for determination of a position of one or more Transceivers 105 supported by an Agent 100, such as, for example, a Transceiver 105 in or associated with a Smart Device, headgear or Tag supported by the Agent 100. Transceiving may be conducted via one or more wireless transmission modalities between a portable Transceiver 105 supported by the Agent 100 and one or more Reference Point Transceivers 101-104.
By way of non-limiting example, Transceivers 105 supported by the Agent 100 may be included in, and/or be in logical communication with, a Smart Device, such as a smart phone, tablet, headgear, ring, watch, wand, pointer, badge, Tag, Node or other Agent supportable device with portable Transceiver 105 able to transceive with the Reference Point Transceivers 101-104.
The Reference Point Transceivers 101-104 may include devices, such as, for example, a radio transmitter, radio receiver, a light generator, a light receiver, a pattern recognizing device. A radio frequency transceiver may transmitters and receivers operative to communicate via wireless modalities such as, for example: WiFi, Bluetooth, Ultrawideband, Ultrasonic, infrared, or other communication modality capable of logical communication between Transceivers 101-105.
In some embodiments, a Reference Point Transceivers 101-104 may include a multi-modality transceiver, that communicates, more locally via a first modality, such as UWB, Bluetooth, WiFi, ANT, Zigbee, BLE, Z Wave, 6LoWPAN, Thread, WiFi, WiFi-ah, NFC (near field communications), sub Giga Hertz, Dash 7, Wireless HART or similar modality and to a greater distance via a second modality, such as cellular communication modalities (e.g. 3G, 4G 5G and the like), sub GHz, IP Protocol modalities and the like which may provide access to a distributed network, such as the Internet.
Wireless communications between Transceivers 101-105 may engage in logical communications to provide data capable of generating one or more of: Cartesian Coordinates, X, Y, Z coordinates, vector values, AoA, AoD, RTT, RSS, a GPS position, or other data that may be utilized for one or more of: locating one or both of an Agent 100; indicating a direction of interest; and identify a Structure or defined area 106.
A precise location may be determined via triangulation based upon a measured distance from three or more Reference Point Transceivers 101-104. For example, a radio transmission or light signal may be measured and compared from the three reference position identifiers 101-103. Other embodiments may include a device recognizable via image analysis and a camera or other Image Capture Device, such as a CCD device, may capture an image of three or more Reference Point Transceivers 101-104. Image analysis may recognize the identification of each of three or more of the Reference Point Transceivers 101-104 and a size ratio of the respective image captured Reference Point Transceivers 101-104 may be utilized to calculate a precise position. Similarly, a height designation may be made via triangulation using the position identifiers as reference to a known height or a reference height.
Transceivers 101-105 may include circuitry and logic capable of transceiving in a single modality, or multiple disparate modalities. Similarly, a Reference Point Transceiver 101-104 and/or an Agent supported Transceiver 105 may include multiple transceiver device, including, transmitters and receivers.
A modality, as used in conjunction with a Transceiver, transmitter and/or receiver refers to one or both of a bandwidth of wireless communication and a protocol associated with a bandwidth. By way of non-limiting example, a modality, as used in relation to a Transceiver, transmitter and/or receiver may include: WiFi; WiFi RTT; Bluetooth; UWB; Ultrasonic, sonic, infrared; ANT, Zigbee, BLE, Z Wave, 6LoWPAN, Thread, WiFi, Wi-Fi 33-ah, NFC (near field communications), sub Giga Hertz, Dash 7, Wireless HART or other logical communication medium.
Triangulation essentially includes determining an intersection of three distances 108-110, each distance 108-110 calculated from a reference point 101-104 to an Agent supported device 105. The presence invention allows for a first distance 108, to be determined based upon a wireless communication in a first modality; and a second distance 109 and a third distance 110 determined based upon a wireless communication in a same or different modality as the first modality. For example, a first distance 108 may be determined based upon a wireless communication using UWB; a second distance 109 may be determined based upon a wireless communication using Bluetooth; and a third communication may be determined based upon a wireless communication using ultrasonic communication (other combinations of same and/or different communication modalities are also within the scope of the present invention).
A location 107 may be determined via triangulation based upon a measured distance from three or more position identifiers 101-103 to the transceiver 105 supported by the Agent 100. For example, timing associated with a radio transmission or light signal may be measured and compared from the three reference position identifiers 101-103. Other embodiments may include a device recognizable via image analysis and a camera or other Image Capture Device, such as a CCD device, may capture an image of three or more position identifiers 101-104.
Additional embodiments, may include image analysis of image data captured via a CCD included in a Smart Device to recognize the identification of each of three or more of the position identifiers 101-104 and a size ratio of the respective image captured position identifiers 101-104 may be utilized to calculate a precise position. Similarly, a height designation may be made via triangulation using the position identifiers as reference to a known height or a reference height. In a similar fashion, triangulation may be utilized to determine a relative elevation of the Smart Device as compared to a reference elevation of the reference points.
In some embodiments, the position 107 of the Agent supported Transceiver 105 may be ascertained via one or more of: triangulation; trilateration; and multilateration (MLT) techniques.
In some embodiments, a geospatial location based upon triangulation may be generated based upon a controller receiving a measurement of angles between the position and known points at either end of a fixed baseline. By way of non-limiting example, a point of a geospatial location may be determined based upon generation of a triangle with one known side and two known angles. Moreover, a geospatial location based upon multilateration may be generated based on a controller receiving measurement of a difference in distance to two reference positions, each reference position being associated with a known location. Wireless signals may be available at one or more of: periodically, within determined timespans and continually. The determination of the difference in distance between two reference positions provides multiple potential locations at the determined distance. A controller may be used to generate a plot of potential locations. In some embodiments, the potential determinations generally form a curve. Specific embodiments will generate a hyperbolic curve.
The controller may be programmed to execute code to locate a relatively exact position along a generated curve, which is used to generate a geospatial location. The multilateration system thereby receives as input multiple measurements of distance to reference points, wherein a second measurement taken to a second set of stations (which may include one station of a first set of stations) is used to generate a second curve. A point of intersection of the first curve and the second curve may be used to indicate a specific location.
In another aspect, in some embodiments, the location of a Transceiver 101-105 may be determined and/or aided via discernment of data based upon a physical artifact, such as, for example a visually discernable feature, shape or printed aspect located within the Structure 106. Discernment of the physical artifact may, for example, be based upon topographical renditions of physical aspects included in the Structure, such as those measured using LIDAR, a magnetic force, image data (or a point cloud derived from image data). A pattern on a surface may convey a reference point by a recognizable pattern (which may be unique to the setting), Vernier or three dimensional structure as non limiting examples. A Smart Device ascertaining a physical reference mark and a distance of the Smart Device to the mark may determine a relative location in space to a coordinate system of the marks.
Marks tied to a geospatial coordinate system may be utilized to determine a relative location. A number of methods may be executed to determine a distance from the Smart Device to a mark such as, for example, a sense reflection of light beams (preferably laser beams), electromagnetic beams of wavelength outside of the visible band such as IR, UV, Radio and the like, or sound based emanations.
In some examples, a carefully placed reference point Node may function as a transceiver of signals. For example, a Node may receive and transmit signals in a radio frequency band of the electromagnetic spectrum. In a simple form, a Node may detect an incoming signal and coincidently broadcast a radio frequency wireless communication. Frequencies utilized for wireless communication may include those within the electromagnetic spectrum radio frequencies used in UWB, WiFi, and Bluetooth modalities, as well as IR, Visible and UV light as examples.
In some embodiments, sound emanations may also be used as a communication mechanism between a smart device and a Reference Point Transceiver 101-104. In some examples, the Reference Point Transceiver 101-104 may function to communicate data with their electromagnetic or sonic transmissions. Such communications may provide identifying information unique to the Node, data related to the synchronization of timing at different well located reference points, and may also function as general data communication nodes. A triangulation calculation of the position of a Smart Device or a Node may result from a system of multiple reference position Nodes communicating timing signals to or from the Smart Device or Node. Methods of calculating positions via wireless communications may include one or more of: RTT, RSSI, AoD, AoA, timing signal differential and the like, Triangulation or other mathematical techniques may also be employed in determining a location.
The process of determination of a position based upon triangulation with the reference points may be accomplished, for example via executable software interacting with the controller in a Smart Device, such as, for example via running an app on the Smart Device.
Reference points may be coded via identifiers, such as a UUID (Universally Unique Identifier), or other identification vehicle.
Reference Position Transceivers 101-104 may be deployed in a defined area 106 to determine a location 107 of an Agent 100 within or proximate to the defined wireless communication area 111. Reference Position Transceivers 101-104 may be fixed in a certain location and transceive in a manner suitable for a triangulation determination of the position of the Agent. Transceiving may occur via wireless transmission to one or more Transceivers 105 supported by the Agent 100. By way of non-limiting example, Transceivers 105 supported by the Agent 100 may be included in, or be in logical communication with, a Smart Device with Transceivers 105 able to transceive with the Reference Position Transceivers 101-104.
The Reference Position Transceivers 101-104 may include devices such as a radio transmitter, radio receiver, a light generator, or an image-recognizable device (i.e., an apparatus set out in a distinctive pattern recognizable by a camera). A radio transmitter may include a UWB Node, Wi-Fi, Bluetooth or other communication device for entering into logical communication between Transceivers 101-105. In some embodiments, Reference Position Transceivers 101-104 may include a Wi-Fi router that additionally provides access to a distributed network, such as the Internet. Cartesian coordinates (including Cartesian coordinates generated relative to a GPS or other reference point), or any other coordinate system, may be used as data that may be utilized for one or more of: locating one or both of an Agent 100; indicating a direction of interest; and identifying a Structure 106 or defined area 106. A radio transmitter may include a router or other Wi-Fi device. The radio transmitter may include transmissions via a Ultra Wideband (“UWB”) frequencies including, for example, 3.5-6.5 GHz; on Wi-Fi frequencies (300 MHz-60 GHz), sub GHz frequencies or other modality. A light generator may distribute light at human-safe intensities and at virtually any frequency known in the art. Such frequencies include, without limitation, infrared, ultraviolet, visible, or nonvisible light. Further, the light beacon may comprise a laser, which may transmit light at any of the aforementioned frequencies in a coherent beam.
This plurality of modalities allows for increased accuracy because each modality may have a different degree of reliability. For example, a Smart Device 101 ND/OR smart receptacle may measure A timing signal transmitted by a Reference Point Transceiver 101-104 within a different error tolerance than it may measure the receipt into a photodetector of infrared laser light. This has at least two principle benefits. First, a location calculation may, in some embodiments, be a weighted average of the location calculated from each modality. Second, outliers may be shed. For example, if the standard location calculation comprises a weighted average of the location as calculated by five modalities, but one modality yields a location greater than two standard deviations from the average computed location, then that modality may not be considered for future weighted location calculations.
Additionally, the radio transmitters and/or transceiver in the Smart Device may comprise multiple antennas that transmit signals in a staggered fashion to reduce noise. By way of non-limiting example, if there are three antennas, then they may transmit a signal in intervals of 20 milliseconds. Given this rate of transmission, a detected time of arrival may be used to determine the distance between the transmitter and the antenna (i.e., the Smart Device). Moreover, the antennas may comprise varying lengths to accommodate desirable wavelengths. Further, dead reckoning may be used to measure location, using traditional methods of numerical integration.
A precise location may be determined based upon wireless transmissions between Nodes, such as between an Smart Device and the Reference Position Transceivers. Timing determinations—as well as signal qualities like angle of arrival, angle of departure, transmission strength, transmission noise, and transmission interruptions—may be considered in generating relative positions of Nodes. Additional considerations may include AI and unstructured queries of transmissions between Nodes and triangulation logic based upon a measured distance from three or more Reference Position Transceivers 101-104. For example, a radio transmission or light emission may be measured, and timing associated with the radio transmission or light to determine a distance between Nodes. Distances from three Reference Position Transceivers AA01-AA03 may be used to generate a position of a Node in consideration. Other methodologies include determination of a distance from one or more Nodes and a respective angle of arrival and/or angle of departure of a radio or light transmission between the Node in consideration and another Node (Reference Point Node or dynamic position Node).
In some embodiments of the invention, position determination in a Structure or on a Property contemplates determination of a geospatial location using triangulation, trilateration, or multilateration techniques. In some embodiments, a geospatial location relative to one or more known reference points is generated. The geospatial location in space may be referred to as having an X,Y position indicating a planar designation (e.g. a position on a flat floor), and a Z position (e.g. a level within a Structure, such as a second floor) may be generated based upon indicators of distance from reference points. Indicators of distance may include a comparison of timing signals received from wireless references. A geospatial location may be generated relative to the reference points. In some embodiments, a geospatial location with reference to a larger geographic area is associated with the reference points, however, in many embodiments, a controller will generate a geospatial location relative to the reference point(s) and it is not relevant where the position is located in relation to a greater geospatial area. In addition to these Cartesian coordinates, polar coordinates may be used, as further described below.
A geospatial location based upon triangulation may be generated based upon a controller receiving a measurement of angles between the position and known points at either end of a fixed baseline. A point of a geospatial location may be determined based upon generation of a triangle with one known side and two known angles.
Referring again to FIG. 1, triangulation essentially includes determining an intersection of three distances 108-210, each distance 108-110 calculated from a reference point 101-104 to an Agent-supported device 105. The present invention allows for a first distance 108 to be determined based upon a wireless communication in a first modality; and a second distance 109 and a third distance 110 determined based upon a wireless communication in a same or different modality as the first modality. For example, a first distance 108 may be determined based upon a wireless communication using UWB; a second distance 109 may be determined based upon a wireless communication using Bluetooth; and a third communication may be determined based upon a wireless communication using ultrasonic communication (other combinations of same and/or different communication modalities are also within the scope of the present invention).
A geospatial location based upon trilateration may be generated based upon a controller receiving wireless indicators of distance and geometry of geometric shapes, such as circles, spheres, triangles and the like.
A geospatial location based upon multilateration may be generated based on a controller receiving a measurement of a difference in distance to two reference positions, each reference position being associated with a known location. Wireless signals may be available at one or more of: periodically, within determined timespans, and continually. The determination of the difference in distance between two reference positions provides multiple potential locations at the determined distance. A controller (such as one in the Smart Device) may be used to generate a plot of potential locations. In some embodiments, the potential determinations generally form a curve. Specific embodiments will generate a hyperbolic curve.
The controller may be programmed to execute code to locate an exact position along a generated curve, which is used to generate a geospatial location. The multilateration thereby receives as input multiple measurements of distance to reference points, wherein a second measurement taken to a second set of stations (which may include one station of a first set of stations) is used to generate a second curve. A point of intersection of the first curve and the second curve is used to indicate a specific location.
In exemplary embodiments, as described herein, the distances may be triangulated based on measurements of UWB, Wi-Fi or sub GHz strength at two points. Transceiver signals propagate outward as a wave, ideally according to an inverse square law. Ultimately, a crucial feature of the present invention relies on measuring relative distances between two points. In light of the speed of Wi-Fi waves and the real-time computations involved in orienteering; these computations need to be as computationally simple as possible. Thus, depending upon a specific application and mechanism for quantifying a condition or location, such as a measurement, various coordinate systems may be desirable. In particular, if the Smart Device moves only in a planar direction while the elevation is constant, or only at an angle relative to the ground, the computation less complicated.
One exemplary coordinate system includes a polar coordinate system. One example of a three-dimensional polar coordinate system is a spherical coordinate system. A spherical coordinate system typically comprises three coordinates: a radial coordinate, a polar angle, and an azimuthal angle (r, θ, and φ, respectively, though θ and φ are occasionally swapped conventionally).
By way of non-limiting example, suppose Point 1 is considered the origin for a spherical coordinate system (i.e., the point (0, 0, 0)). Each Transceiver 101-105 emitter e1, e2, e3 can be described as points (r1, θ1, φ1), (r2, θ2, φ2), and (r3, θ3, φ3), respectively. Each of the ri's (1≤i≤3) represent the distance between the Transceiver 101-105 emitter and the Transceiver 101-105 receiver on the Smart Device 101 or Smart Receptacle (see FIG. 5A).
It is understood that in some embodiments, an azimuth may include an angle, such as a horizontal angle determined in an arcuate manner from a reference plane or other base direction line, such as an angle formed between a reference point or reference direction; and line (ray or vector) such as a ray or vector generated from or continuing to a Smart Device. In preferred embodiments, the ray or vector may be generally directed from a Reference Position Transceiver towards, and/or intersect one or more of: an item of interest; a point of interest; an architectural aspect (such as a wall, beam, header, corner, arch, doorway, window, etc.); an installed component that may act as a reference in an augmented virtual model (AVM) (such as, for example, an electrical outlet, a light fixture, a plumbing fixture, an architectural aspect; an item of equipment; an appliance; a multimedia device, etc.); another Reference Position Transceiver or other identifiable destination.
Accordingly, in some embodiments, a spherical coordinate system may include Reference Position Transceiver that is capable of determining an angle of departure of a location signal and a Transceiver that is capable of determining an angle of arrival of the location signal; one or both of which may be used to facilitate determination of an applicable azimuth.
According to various embodiments of the present invention, one or both of an angle of departure and an angle of arrival may therefore be registered by a Transceiver that is transmitting and/or receiving wireless signals (e.g. radio frequency, UWB, Bluetooth 5.1, sonic frequency, or light frequency).
In some embodiments, locating an Agent occurs in or proximate to a Structure in which Reference Position Transceivers, (including, for example, one or more of: Wi-Fi Transceivers, UWB Transceivers, Bluetooth Transceivers, infrared Transceivers and ultrasonic Transceivers) may be located above and/or below an Agent. In these embodiments, a cylindrical coordinate system may be more appropriate. A cylindrical coordinate system typically comprises three coordinates: a radial coordinate, an angular coordinate, and an elevation (r, θ, and z, respectively). A cylindrical coordinate system may be desirable where, for example, all Wi-Fi emitters have the same elevation. Angles may be determined as described above.
In some embodiments, transceivers 101-105 including arrays of antennas may be used to measure an angle of radio communication (e.g. angle of arrival and/or angle of departure). Various configurations of transmitting antennas and receiving antennas may be used. For example, a radio transmission may be transmitted with a single antenna and received with a receiver with an array of antennas, the phase or timing difference of arriving signals can be used to calculate the angle at which the signals emerged. In angle of departure schemes, a transmitter may contain an array of antennas and may send a pattern of signals through the array that arrive at a receiver with a single antenna where the angle of departure (AoD) is communicated.
Measurement of angle of arrival may be performed as mentioned by calculation of time difference of arrival at the antennas in an array or alternatively can be performed by rotation of antenna elements.
Some modalities, such as those modalities that adhere to the Bluetooth 5.1 or BL5.1 standards, allow a Smart Device 101, Smart Receptacle or other Node to determine an angle of arrival (AoA) or an angle of departure (AoD) for a wireless transmission. An array of antennas may be used to measure aspects of the Bluetooth signaling that may be useful to calculate these AoA and AoD parameters. By calibrating an antenna system, the system may be used to determine angles in one or two dimensions depending on the design of the antenna. The result may be significant improvement in pinpointing the location of origin of a signal.
An array of antennas may be positioned relative to each other and a transmitting transceiver to allow for extraction of an AoA/AoD. Such an array may include a rectangular array; a polar or circular array; a linear array; and a patterned array, where a number of antennas are deployed in a pattern conducive to a particular environment for transceiving. Antennas may be separated by characterized distances from each other, and in some examples, a training protocol for the antenna array results in antenna positioning incorporating superior angle and location precision. Some transceivers may transceive in 2.4-2.482 GHz frequency bands, and thus the radiofrequency transmissions may have wavelengths in the roughly 125 mm length scale. A collection of antennas separated by significantly less than the wavelength may function by comparing a phase of RF transmissions arriving at the antennas. An accurate extraction of phase differences can yield a difference in path length that when accumulated can lead to a solution for the angles involved. In some embodiments, Transceivers 101-105 may include antenna arrays combined with batteries and circuitry to form complete self-contained devices. Antenna arrays and methods of using the same for determining position and direction of a Smart Device or other Node are described in U.S. Ser. No. 16/775,223, the contents of which are incorporated herein by reference.
In an example, an Agent supported transceiver 105 may be located at a position and may transmit a signal of the various types as have been described. Nodes, such as reference point transceivers 101-104 located at reference points in the wireless communication area around the position of the Agent 100 may receive the transmission and determine the angle of arrival of that transmission. Similarly, transmission associated with other transceivers 101-103 may also determine an angle of arrival of transmissions. In some embodiments, transceiver 101-103 may communicate with one or more of: each other, a smart device, a controller or other processor to mathematically process multiple angles and locations of the transceivers and calculate a position of a transmission emanation. In examples where calculations are not performed at a smart device, a communication to the smart device of the calculated position may be communicated.
In certain embodiments of the invention, a direction of interest indicated by Smart Device 101 or a Smart Receptacle (see FIG. 5A) may be determined based upon a movement of Smart Device 101 or a Smart Receptacle 502. For example, a device with a controller and an accelerometer, such as mobile Smart Device, may include a user display that allows a direction to be indicated by movement of the device from a determined location acting as a base position towards an As Built feature in an extended position. In some implementations, the Smart Device may first determine a first position based upon triangulation with the reference points and a second position (extended position) also based upon triangulation with the reference points. These position determinations may proceed as described above. The process of determination of a position based upon triangulation with the reference points may be accomplished, for example via executable software interacting with the controller in the Smart Device, such as, for example via running an app on the Smart Device. Logical communications relevant to location determination may include, for example, one or more of: timing signals; SIM information; received signal strength; GPS data; raw radio measurements; Cell-ID; round trip time of a signal; phase; and angle of received/transmitted signal; time of arrival of a signal; a time difference of arrival; and other data useful in determining a location.
In another aspect, captured data may be compared to a library of stored data using image recognition software to ascertain and/or affirm a specific location, elevation and direction of an image capture location and proper alignment with the virtual model. In an example, a position of a user may be determined by any of the means described herein. A user may position a camera of an associated smart device to be pointing in a direction of interest and obtain an image. The image may be passed on to a server with access to database of images containing stored images of the space around the user. Algorithms on the server may compare the stored images to the image captured by the user, and may calculate adjustments to the comparative image based on where the reference image was taken in relationship to the location of the user. Based on the determination that the calculated adjusted image compared to the image obtained by the user in the direction of interest, a direction may be inferred with known location of objects in the reference image. In some variations, the differences in features of the user obtained image compared to a reference image may be used to calculate a direction of interest based upon a location at which the reference image was obtained.
In some examples, stored images may be obtained at multiple angles to improve accuracy of orienteering. These examples may include sensor arrays, audio capture arrays and camera arrays with multiple data collection angles. In some examples a full 360 degree camera perspective may be obtained by such arrays. In some directional arrays, a sensor array (including image capture sensors) may include at least 120 degrees of data capture. By collecting such image collections as the sensor/camera systems are moved a database of image perspectives may be formed and utilized to assist in orienteering as described.
Non-limiting examples may include image based identification where a device with some imaging means, including but not limited to a mobile device camera, tablet device camera, computer camera, security camera, or AR headset camera may image points of interest in a direction of interest. These points of interest may be identified. Image recognition software may be used to identify the visualized landscape by its identifying features. Machine learning may be used to train systems using this software to identify specific features of the environment in question.
To create a supplemental topographic part of a model of the environment of a user, laser scanning and/or LiDAR may be performed during the acquisition of imagery for a reference database. A resulting three dimensional shape model may be modelled with the captured imagery to help in the comparison to user data. Three dimensional shape can be used to infer comparative imagery at different angles of acquisition than exist in a database. In another example, a device of a user may have the means of performing laser scanning or LiDAR scanning of the environment as well as obtaining images. The resultant three dimensional data or a composite of the three dimensional data and the imagery may be used to recognize features and determine a direction that the user was facing when they collected the image.
The results of scanning may be stored and presented in different manners. In some examples, the scanned data may be represented by a point cloud representation, in other examples an estimated topographic surface representation may be used to visualize the three dimensional shape data obtained. In some examples, outward facing planes of the surface topography may have the captured imagery superimposed upon them. The resulting image and three dimensional models may be used to calculate a direction of interest or a device field of view in a dynamic sense or alternatively upon user request.
In some examples other methods of capturing spatially accurate information may include the use of drones and optical scanning techniques which may include high resolution imagery obtained from multiple viewpoints. Scanning may be performed with light based methods such as a CCD camera. Other methods may include infrared, ultraviolet, acoustic, and magnetic and electric field mapping techniques.
In other embodiments, a single distance to a point of interest in an image, which may be obtained by a laser, other collimated light source, sound source or the like, may be used with models of the environment of the user. A comparison of the imagery and the measurement of the distance of the user to a prominent feature in the image may allow for an orientation of the user to be determined algorithmically.
Referring now to FIG. 2, in some exemplary embodiments, radio frequencies used for wireless communication include sound waves used to perform one or more of: location determination; movement tracking in interior or exterior areas; and distance calculation. Sound wave transmissions include a number of significant attributes, which may translate into a benefit for a given set of circumstances when used for RF based location determination.
According to the present invention, sonic waves may be deployed independently, or in combination with electromagnetic transmissions and reception of logical communications utilizing other bandwidths, such as bandwidths associated with Ultrawideband, WiFi, Bluetooth, Ultrawideband, ANT, infrared or almost any wavelength in the Industrial, Scientific and Medical bands (sometimes referred to as “ISM Bands”).
For example, sound waves travel through an ambient atmosphere at a significantly slower speed than electromagnetic radiation (6×10{circumflex over ( )}2 m/sec versus 3×10{circumflex over ( )}8 m/sec). Therefore, a relative accuracy for measurements related to travel times may be improved in some environments by orders of magnitude using sonic based location tracking as compared to electromagnetic based measurements. Therefore, using sonic communications may result in increased accuracy of location determination, in some environments.
The present invention also provides for sonic wave emanations may be used to compliment electromagnetic emanations based upon a tendency that sonic waves generally do not efficiently penetrate walls other physical items or structures. Sonic transceivers may be particularly advantageous in a defined area where location can be unambiguously determined to be within a particular room (the use of multiple bandwidth transmitting and receiving for different purposes is further discussed below). Sound wave interaction with a solid surface, such as a wall, may require that for optimal performance, transceiver/transmitters/receivers to be located in each room where location detection is desired. In some embodiments, a reflected sonic transmission may be received and analyzed to determine an effect of the reflected nature of the transmission.
Accordingly, methods may be employed using sonic emanations and reception for location determination. In general, frequencies of effective indoor sonic location detection may be at ultrasonic bandwidths (commonly used bandwidths include frequencies of between 1 MHz and 10 MHz, although frequencies of less than 50 kHz to greater than 200 MHz are possible). The utilized frequencies may be either below or above audible detection by people or other animals in the location; such as at frequencies above 20 kHz.
Sound waves may be used to perform one or more of: location determination, movement tracking in interior or exterior locations, and distance calculation from a position to a Smart Device 101, which may be accomplished based upon transmission and receipt of sonic transmission. Sound wave transmissions include several significant attributes, which may be beneficial for a given set of circumstances when used for radiofrequency-based location determination. According to the present invention, sonic waves may be deployed independently, or in combination with, transmissions and reception of logical communications utilizing other bandwidths, such as bandwidths associated with Wi-Fi, Bluetooth, ANT, infrared or almost any wavelength in the Industrial, Scientific and Medical bands (sometimes referred to as “ISM Bands”). Methods, devices, and apparatus for using sound waves in location determination may be found, for example, in U.S. Pat. No. 10,628,617, the contents of which are incorporated herein.
As illustrated in FIG. 2, in some embodiments, a sonic Transceiver 200 may transmit a sonic transmission 202 and determine a location 201 based upon receiving an echo 203 back from an Agent supported transceiver 204. Walls 205 may also generate reflected sonic emanations 206.
In some examples, as may be used in orienteering herein, an Agent supported device 204 may support receivers, transmitters or transceivers that interact with ultrasonic transceivers fixedly secured to a reference point position, such as via mechanical mounting within a room environment. An ultrasonic positioning system may have indoor positioning accuracy at centimeter, millimeter, and even sub millimeter accuracy. Multiple ultrasonic Transceivers may transceive from an Agent supported device to communicate with fixed reference point transceivers may transmit signals. Arrival of the sound transmissions may be accurately timed and converted to distances. In some embodiments, distance determinations may be improved with knowledge of temperatures in the environment containing the sound transceiving. For example, temperature may be measured at one or more of: an Agent supported smart device, a Reference Point position, and an ambient environment.
In some examples, synced timing apparatus is able to generate a location of a stationary Agent to within centimeter accuracy using sonic wave transmissions and reception and preferably within several millimeters of accuracy. In addition, in some embodiments sensors are able to detect frequency shifts within the sonic emanations which may add information about the relative rate of movement of the Agent, which may then in turn allow for correction to the timing signals.
In some examples, a combination of radio frequency emissions and ultrasonic emissions may be used. For example, a compliment of radio frequency emissions/receptions and ultrasonic of radio frequency emissions and ultrasonic emissions may be reconciled to generate more accurate location determination. In another aspect, a radio frequency signal may be used to transmit syncing signals that establish a time that ultrasonic signals are transmitted. Since, the electromagnetic transmissions may be orders of magnitude faster than sound transmissions, the electromagnetic transmissions relatively small time of travel from the Transceivers to the Agent may be negligible and therefore used as “zero-time” setpoints as received at the Agent supported Transceiver. In such embodiments, a controller determining a location may use not only relative arrival times, but also a delta time between a radiofrequency transmission and ultrasonic transmission to determine a distance from a transmitting Transceiver. An array of such ultrasonic and/or radiofrequency transceivers provide increased accuracy in triangulating a location of the Agent.
In still further examples, RF communications may not only transmit a syncing pulse, but also transmit digital data about various aspects of a defined area, such as the defined area's identification, its relative and/or absolute location in space and other refinements. In some examples, data related to improved distance calculation may also be transmitted by RF communication such as temperature of the environment, humidity and the like which may influence the speed of sound in the environment as a non-limiting example. In some examples such a system may result in millimeter level accuracy of position determination.
In some examples, the process may be iterated to refine the direction of each of the ultrasonic transmitters and maximize signal levels of the transmission which may provide additional information in the calculation of a position. RF and WiFi transmissions may be used for data communications and syncing as have been described. In other examples, as an Agent supported device 204 is moving, an iterative process may be used to track the Agent supported device 204 moves through space. Stationary Agents may be tracked with submillimeter accuracy in some embodiments.
Referring now to FIG. 3, methods and devices for determining a direction that may be referenced for one or both of data capture and data presentation of a particular portion of the virtual representation of surroundings of a user. A User 300 may position a Transceiver 305 in a first position 301 proximate to a portion in a space of interest 325. The first position 301 of the Transceiver 305 may be determined and recorded. The User 300 may then relocate the Transceiver 305 to a second position 302 in a general direction of the portion in space of interest. With associated position information obtained for the first and second positions a controller in an external system or in an associated Transceiver 305 may generate one or both of a ray and a vector towards the portion of a space of interest.
In some embodiments, the vector may have a length determined by a controller that is based upon a distance to a feature of interest in space as represented in a model on the controller in the direction of the generated vector. The vector may represent a distance 303 from the second position 302 to the space of interest 325 along the axis defined by a line between the first position 301 and the second position 302. In contrast, a ray may include just a starting point and a direction.
In still other embodiments, a device with a controller and an accelerometer, such as mobile phone, tablet or other Smart Device that includes a Transceiver 305, may include a user display that allows a direction to be indicated by movement of the device from a determined location acting as a base position towards an point in a direction of interest or representing a center of a field of view of the device. The movement may occur to a second location in an extended position. In some implementations, the Smart Device determines a first position 301 based upon triangulation with the reference points. The process of determination of a position based upon triangulation with the reference points may be accomplished, for example via executable software interacting with a controller in the Smart Device, such as, for example by running an app on the Transceiver 305.
Referring to FIGS. 4A-D a series of exemplary devices employing matrices (or arrays) of antennas for use with Nodes that communicate wirelessly, such as, via exemplary UWB, Sonic, Bluetooth, a Wi-Fi or other modality, is illustrated. Linear antenna arrays 401 are illustrated in FIG. 4A. Rectangular antenna arrays 402 are illustrated in FIG. 4B. Circular antenna arrays 403 are illustrated in FIG. 4C, other shapes for arrays are within the scope of the invention. In addition, an antenna array may be omni-directional or directional.
In some embodiments, a Smart Device 405 may include one or more Nodes 406 internal to the Smart Device 405 or fixedly attached or removably attached to the Smart Device 405. Each Node 406 may include antenna arrays combined with a power source and circuitry to form complete self-contained devices. The Nodes 406 or a controller may determine an RTT, time of arrival, AoA and/or AoD or other related angular determinations based upon values for variables involved in wireless communications. For example, a composite device 404 may be formed when a Node 406 with a configuration of antennas, such as the illustrated exemplary circular configuration of antennas 407, is attached to a Smart Device 405. The Node 406 attached to the Smart Device 405 may communicate information from and to the Smart Device 405 including calculated results received from or about another Node 406, such as a Node 406 fixed as a Reference Point Transceiver or a Node with dynamic locations, wherein the wireless communications are conducive to generation of data useful for determination of a position (e.g. timing data, angles of departure and/or arrival, amplitude, strength, phase change, etc.). A combination of angles from multiple fixed reference points to the antenna array can allow for a location of a user in space. However, with even a single wireless source able to communicate with the antenna array, it may be possible to determine a direction of interest or a device related field of view.
An array of antennas positioned at a reference point may allow for the accurate receipt of orientation information from a transmitter. As discussed earlier a combination devices with arrays of antennas may be used to calculation a position. A single Node with an array of antennas can be used for orienteering and determining a direction of interest. Each of the antennas in such an array receiving a signal from a source will have different phase aspects of the received signal at the antennas due to different distances that the emitted signal passes through. The phase differences can be turned into a computed angle that the source makes with the antenna array.
Referring now to FIG. 5A, in some embodiments, one or both of a Smart Device 501 and a Smart Receptacle 502 may incorporate multiple Transceivers 503-510 and a direction of interest may be ascertained by generating a vector 527 passing through a respective position of each of at least two of the transceivers (as illustrated through transceiver 505 and 507). The respective positions of each of the transceivers 503-510 supported by the Smart Device 501 and/or Smart Receptacle 502 may be ascertained according to the methods presented herein, including for example via triangulation, trilateration, signal strength analysis, RTT, AoD, AoA, topography recognition.
In some embodiments, one or both of the Smart Device 501 and the Smart Receptacle 502 incorporating Transceivers 503-510 may be rotated in a manner (such as, for example in a clockwise or counterclockwise movement 520 522 relative to a display screen) that repositions one or more Transceivers 503-510 from a first position to a second position. A vector 526 may be generated at an angle that is zero degrees 524 with a plane of a display 515 or perpendicular 525 or some other designated angle in relation to the smart device 501 and an associated display screen 515. In some embodiments, an angle in relation to the smart device is perpendicular 525 to a display screen 515 and thereby viewable via a forward looking camera (or other CCD or LIDAR device) on the smart device. In addition, a mirror or other angle altering device may be used in conjunction with a CCD, LIDAR or other energy receiving device.
Movements of a Smart Device 501 equipped with an antenna array can be determined via relative positions of the antenna and/or via operation of an accelerometer 524 within the Smart Device 501 or Smart Receptacle 502. Rough movement sense may be inferred with a single source to the antenna array. However, with multiple sources, the positional movement of each of the antennas can be used to sense many types of movements including translations and rotations.
A user may position the smart device 501 such that an object in a direction of interest is within in the CCD view. The smart device may then be moved to reposition one or more of the transceivers 503-510 from a first position to a second position and thereby capture the direction of interest via a generation of a vector in the direction of interest.
In addition to movement of the Smart Device 501 and/or the Smart Receptacle 502 may include a magnetic force detection device 523, such as a magnetometer. A registration of a magnetic force may be determined in relation to a particular direction of interest 524-525 and a subsequent determination of magnetic force referenced or provide a subsequent orientation of the Smart Device 501 or Smart Receptacle 502.
In some embodiments, the magnetic force detection device 523 may be used combination with, or in place of directional movement of the Smart Device transceivers 503-507 to quantify a direction of interest to a user. Embodiments may include an electronic and/or magnetic sensor to indicate a direction of interest when a Smart Device 501 and/or Smart Receptacle 502 is aligned in a direction of interest. Alignment may include, for example, pointing a specified side of a Smart Device 501 and/or Smart Receptacle 502, or pointing an arrow or other symbol displayed upon a user interface on the Smart Device 501 towards a direction of interest.
A magnetic force detection device 523 may detect a magnetic field particular to a setting that a Smart Device is located. For example, in some embodiments, a particular structure or other setting may have a magnetic force that is primarily subject to the earth's magnetic field or may be primarily subject to electromagnetic forces from equipment, power lines, or some other magnetic influence or disturbance. An initial quantification of a magnetic influence at a first instance in time may be completed and may be compared to a subsequent quantification of magnetic influence at a later instance in time. In this manner an initial direction of interest indicated by a position, orientation, pitch and yaw of a Node, such as a Smart Device may be compared to a subsequent position, orientation, pitch and yaw of the Smart Device.
In some embodiments, an initial position, pitch and yaw of a Smart Device 501 may be described as a relative angle to a presiding magnetic force. Examples of presiding magnetic forces include, magnetic influences of electrical charges, Earth's magnetic field, magnetized materials, permanent magnetic material, strong magnetic fields, ferromagnetism, ferrimagnetism, antiferromagnetism, paramagnetism, and diamagnetism, or electric fields that are generated at reference nodes at known positions which may be locally used to indicate a direction of interest.
Smart devices may include electronic magnetic sensors as part of their device infrastructure. The magnetic sensors may typically include sensing elements deployed along three axis. In some examples, the magnetic sensors may be supplemented with electronic accelerometers, such as MEMS accelerometers.
In some examples the magnetic sensors may measure a sensed magnetic field perpendicular to the body of the sensor through a Hall Effect sensor. In some examples, a Hall Effect sensor may be built into silicon to generate a relatively sensitive sensing capability for magnetic fields. In some hall effect sensors, electrons and holes flowing in a region of the silicon may interact with the regional magnetic field and build up on the fringes of the conduction region, thus generating a measurable voltage potential. In other examples, anisotropic magnetoresistance sensors may sensitively detect the magnetic field at the device as a significant change in resistance of structure elements in the device.
In still further examples, giant magnetoresistance (GMR) sensors may detect the magnetic field. In some of these examples, the GMR sensors may detect a magnetic field with a current-perpendicular-to-plane (CPP) GMR configuration. In other examples a current in plane (CIP) GMR sensor configuration may be used. The resulting three axis magnetic sensors may perform a sensitive compass function to determine a direction of a specified portion of the smart device and/or an edge of the smart device relative to the local magnetic field environment. A specified portion of the Smart Device may be indicated via a user interface presented on a screen of the Smart Device.
Referring now to FIG. 5B, as illustrated, a vector in a direction of interest 525 may be based upon a rocking motion 533-534 of the smart device 501, such as a movement of an upper edge 518 in a forward arcuate movement 533. The lower edge 519 may also be moved in a complementary arcuate movement 534 or remain stationary. The movement of one or both the upper edge 518-519 also results in movement of one or more transceivers 503-510 and/or registration in an onboard accelerometer 534. The movement of the transceivers 503-510 will preferably be a sufficient distance to register disparate geospatial positions based upon wireless transmissions and/or sufficient to register movement via the accelerometer 534.
As presented herein, a direction dimension may be based upon one or more of: a wireless position of two or more transceivers, a movement of a device, a magnetic force determination, a LIDAR transmission and receiving, CCD energy determinations and other assessments of an environment containing the Smart Device and/or Smart Receptacle. For example, a device with a controller and an accelerometer, such as mobile Smart Device, may include a user display that allows a direction to be indicated by movement of the device from a determined location acting as a base position towards a feature in the intended direction where the movement results in an extended position. In some implementations, the Smart Device may first determine a first position based upon triangulation with the reference points and a second position (extended position) also based upon triangulation with the reference points.
As described above, facing a mobile device towards an area in a Structure and movement of the mobile device in a particular pattern may be used to ascertain a specific area in space to be interpreted by various methods.
Referring to FIG. 5C, an illustration of an Agent 550 utilizing an oriented stereoscopic camera system 5551 to orient a direction of interest is shown. The stereoscopic camera system 551 may obtain two different images from different viewpoints 552-553 which may be used to create topographical shape profiles algorithmically. A controller may obtain the image and topographic data and algorithmically compare them to previously stored images and topographic data in the general environment of the user. The resulting comparison of the imagery and topography may determine an orientation in space of the Agent 501 and in some examples determine a device field of view. The controller may utilize this determined field of view for various functionality as described herein.
Referring now to FIG. 6 an automated controller is illustrated that may be used to implement various aspects of the present invention, in various embodiments, and for various aspects of the present invention, controller 600 may be included in one or more of: a wireless tablet or handheld device, a server, a rack mounted processor unit. The controller may be included in one or more of the apparatus described above, such as a Smart Device, Smart Tablet, Headgear, Smart Watch Smart Ring or other Smart Device. The controller 600 includes a processor unit 620, such as one or more semiconductor based processors, coupled to a communication device 610 configured to communicate via a communication network (not shown in FIG. 6). The communication device 610 may be used to communicate, for example, via a distributed network such as a cellular network, an IP Protocol network, the Internet or other distributed logic communication network.
The processor 620 is also in communication with a storage device 630. The storage device 630 may comprise any appropriate information storage device, including combinations of digital data storage devices (e.g., Solid State Drives and hard disk drives), optical storage devices, and/or semiconductor memory devices such as Random Access Memory (RAM) devices and Read Only Memory (ROM) devices.
The storage device 630 can store a software program 640 with executable logic for controlling the processor 620. The processor 620 performs instructions of the software program 640, and thereby operates in accordance with the present invention. The processor 620 may also cause the communication device 610 to transmit information, including, in some instances, control commands to operate apparatus to implement the processes described above. The storage device 630 can additionally store related data in a database 650 and database 660, as needed.
Referring now to FIG. 7, a block diagram of an exemplary smart device 702. The smart device 702 comprises an optical capture device 708 to capture an image and convert it to machine-compatible data, and an optical path 706, typically a lens, an aperture or an image conduit to convey the image from the rendered document to the optical capture device 708. The optical capture device 708 may incorporate a Charge-Coupled Device (CCD), a Complementary Metal Oxide Semiconductor (CMOS) imaging device, or an optical Sensor 724 of another type.
A microphone 710 and associated circuitry may convert the sound of the environment, including spoken words, into machine-compatible signals. Input facilities may exist in the form of buttons, scroll wheels, or other tactile Sensors such as touch-pads. In some embodiments, input facilities may include a touchscreen display.
Visual feedback to the user is possible through a visual display, touchscreen display, or indicator lights. Audible feedback 734 may come from a loudspeaker or other audio transducer. Tactile feedback may come from a vibrate module 736.
A magnetic force sensor 737, such as a Hall Effect Sensor, solid state device, MEMS device or other silicon based or micro-electronic apparatus.
A motion Sensor 738 and associated circuitry converts motion of the smart device 702 into a digital value or other machine-compatible signals. The motion Sensor 738 may comprise an accelerometer that may be used to sense measurable physical acceleration, orientation, vibration, and other movements. In some embodiments, motion Sensor 738 may include a gyroscope or other device to sense different motions.
A location Sensor 740 and associated circuitry may be used to determine the location of the device. The location Sensor 740 may detect Global Position System (GPS) radio signals from satellites or may also use assisted GPS where the mobile device may use a cellular network to decrease the time necessary to determine location. In some embodiments, the location Sensor 740 may use radio waves to determine the distance from known radio sources such as cellular towers to determine the location of the smart device 702. In some embodiments these radio signals may be used in addition to GPS.
The smart device 702 comprises logic 726 to interact with the various other components, possibly processing the received signals into different formats and/or interpretations. Logic 726 may be operable to read and write data and program instructions stored in associated storage or memory 730 such as RAM, ROM, flash, SSD, or other suitable memory. It may read a time signal from the clock unit 728. In some embodiments, the smart device 702 may have an on-board power supply 732. In other embodiments, the smart device 702 may be powered from a tethered connection to another device or power source.
The smart device 702 also includes a network interface 716 to communicate data to a network and/or an associated computing device. Network interface 716 may provide two-way data communication. For example, network interface 716 may operate according to the internet protocol. As another example, network interface 716 may be a local area network (LAN) card allowing a data communication connection to a compatible LAN. As another example, network interface 716 may be a cellular antenna and associated circuitry which may allow the mobile device to communicate over standard wireless data communication networks. In some implementations, network interface 716 may include a Universal Serial Bus (USB) to supply power or transmit data. In some embodiments other wireless links may also be implemented.
As an example of one use of smart device 702, a reader may scan some coded information from a location marker in a facility with the smart device 702. The coded information may include for example, a hash code, bar code, RFID or other data storage device. In some embodiments, the scan may include a bit-mapped image via the optical capture device 708. Logic 726 causes the bit-mapped image to be stored in memory 730 with an associated time-stamp read from the clock unit 728. Logic 726 may also perform optical character recognition (OCR) or other post-scan processing on the bit-mapped image to convert it to text. Logic 726 may optionally extract a signature from the image, for example by performing a convolution-like process to locate repeating occurrences of characters, symbols or objects, and determine the distance or number of other characters, symbols, or objects between these repeated elements. The reader may then upload the bit-mapped image (or text or other signature, if post-scan processing has been performed by logic 726) to an associated computer via network interface 716.
As an example of another use of smart device 702, a reader may recite words to create an audio file by using microphone 710 as an acoustic capture port. Logic 726 causes audio file to be stored in memory 730. Logic 726 may also perform voice recognition or other post-scan processing on the audio file to convert it to text. As above, the reader may then upload the audio file (or text produced by post-scan processing performed by logic 726) to an associated computer via network interface 716.
A directional sensor 741 may also be incorporated into the smart device 702. The directional device may be a compass and be based upon a magnetic reading, or based upon network settings. The magnetic sensor may include three axes of magnetic sensitive elements and may also be coupled with an accelerometer in the directional sensor 741.
A LiDAR sensing system 751 may also be incorporated into the smart device 702. The LiDAR system may include a scannable laser light (or other collimated) light source which may operate at nonvisible wavelengths such as in the infrared. An associated sensor device, sensitive to the light of emission may be included in the system to record time and strength of returned signal that is reflected off of surfaces in the environment of the smart device 702.
In the following sections, detailed descriptions of examples and methods of the invention will be given. The description of both preferred and alternative examples though through are exemplary only, and it is understood that to those skilled in the art that variations, modifications and alterations may be apparent. It is therefore to be understood that the examples do not limit the broadness of the aspects of the underlying invention as defined by the claims.
Referring now to FIGS. 8A-8C, a support 800 for a smart device 801 is illustrated. The support remains stationary in relation to a ground plane. One or more position devices 803-808 are shown located within, on or proximate to the smart device 801. In FIG. 8A, generally linear movement 814-815 from a first position to a second position is illustrated. In some embodiments, a cessation of movement in a general direction is determined via an accelerometer included in or operated by the smart device 801. In other embodiments (show here as the support 800) may activate a user interactive device, such as a button on a touch screen, or a switch to indicate one or both of the first position and the second position.
The transceivers 803-808 enter into logical communication with multiple wireless positional reference transceivers 810-813.
In some embodiments, a direction of interest will include an item of interest 809, such as an apparatus or other piece of equipment. A direction of interest 814 may include a vector with a direction pointing towards the item of interest 809. The vector length will be sufficient to reach the item of interest 809.
In some embodiments, a vector indicating a direction of interest 814 may be used to reference stored data, such as, for example, digital data stored in a database, an AVM, URL, cloud storage or other storage medium. A user interface may include a drop down menu that includes potential items of interest 809 in a general direction of the vector.
In some embodiments, a selection of an item of interest may be used to facilitate determination of a length of the vector 814.
Referring now to FIG. 8C, a movement of a smart device 801 may be arcuate in nature 814C so long as arcuate movement 814C results in sufficient distance of movement of one or more position devices 803-808.
Referring now to FIG. 9, method steps that may be implemented in some embodiments of the present invention are illustrated. At method step 900, geospatial location services are used to determine geospatial location such as a location of the structure with a position and direction of interest. Geospatial services may be used to determine a user's location relative to the structure and directions thereto. The methods used may include, by way of non-limiting example, one or more of: satellite-based global positioning systems (GPS), cell tower triangulation, radio signal triangulation, Wi-Fi signal location services, infrared transmitters and the like.
Geospatial location services will be cross-referenced with database registry of as built virtually modeled facilities and may be used in conjunction with a network of registered service technicians to route the nearest available service technician to the structure experiencing equipment area of interest. Service technician may register with the system to accept geospatial location tracking services by the system.
At method step 901, the service technician's entry into the structure will be registered. Registration of entry into the structure may be achieved through multiple methods, which may include, by way of non-limiting example, on or more of: WiFi gateway detection, infrared detection, magnetic door locking systems, Bluetooth services, and the like. Upon entry into the structure requesting the service call, system will register the service technician's entry into the structure.
At method step 902, a support unit for a smart device, such as service technician or an unmanned vehicle may be tacked via a change in triangulation values and/or an accelerometer and a position and direction within the structure is tracked. The methods used may be, by means of non-limiting example, one or more of: use of data gleaned from accelerometers located on or in possession of service technicians, Wifi services, radio frequency (RF) triangulation, Bluetooth technology, infrared detection, RFID badges, and the like.
At method step 903, a smart device will be registered as entering within structure. Following the smart device entry into structure.
At method step 904, a smart device may be associated with one or both of a person and an entity.
At method step 905, the smart device is pre-registered by the system with detailed instructions regarding a reason for the device to be at a particular location. The reason may be, for example, one or more of: a service call placed from structure to system detailing current equipment area of interest, service calls from structure detailing non-specific area of interests and symptomatic data indicating equipment area of interest, a service call placed by self-assessing equipment utilizing internet of things (IoT) and machine learning functionality to ascertain area of interests and predictive analytics to anticipate area of interests, and the like. The system may integrate data reports into the AVM and relay as much to the smart device in the field.
Alternatively, at method step 905A, the smart device may arrive at the structure without prior knowledge of a purpose. Upon entry into the structure and registration of the smart device as described in method steps 901 through 904, system will relay data gleaned from the AVM, operational data uploaded to the system through IoT processes, and other experiential data reported to the system and thereby relayed to the smart device on site. Methods for relation of such data to the onsite smart device may include, by means of non-limiting example, referential data based on proprietary orienteering processes to determine smart device location within structure, which location will be cross-referenced with AVM data.
At method step 906, a position within or proximate to the structure may be determined via positioning identifiers. The position within or proximate to the structure is determined and detailed instructions directing smart device to the source of an area of interest is relayed by the system to the smart device directly or by means of smart device application. The methods used may be, by means of non-limiting example, one or more of: augmented reality overlays displayed on heads-up displays or other wearable technologies, augmented reality overlays displayed on smart devices, direct instructional vectoring provided to the smart device by the system over Wifi internet connection or LTE signal, virtual reality walkthrough instructions provided to smart device on site or prior to arrival at the structure, updated map/schematic displays detailing the structure and directing the smart device to the source of the subject area of interest by means of vectoring and orienteering processes.
At method step 907, a smart device's location within the structure along an XY axis will be tracked and recorded by the system by means of fixed or adaptive orienteering apparatus within the structure. Such orienteering apparatus may include, by means of non-limiting example, on or more of: radio frequency (e.g. UWB, WiFi, sub GHz, etc.) triangulation, infrared position detection, radio frequency (RF) detection, RF ID tracking, onboard accelerometers located on the smart device or carried smart devices, quantification of magnetic fields, topographical pattern recognition, and the like.
At method step 908, the smart device's location within the structure along the Z axis will be determined. The methods used may be, by means of non-limiting example, one or more of: Smart Device integrated magnetometers, Smart Device integrated barometers, Smart Device integrated accelerometers, Smart Device integrated tilt sensors, used in conjunction with in-structure XY axis position processes described in method step 097 above, along with data detailed in a database or model of the structure.
At method step 909, the smart device's direction of interest will be determined. Method steps 901 through 909 work in conjunction to track and direct the smart device to an area of interest; as the smart device approaches an area of interest, the smart device will be oriented to the area of interest. The system will determine the smart device's direction of interest using, by means of non-limiting example, one or more of the following methods: generation of a vector based upon multiple locations of transceiver(s), wherein the transceiver locations may be associated with a single transceiver at two different locations, or disparate transceivers located at respective locations; a transceiver location and an AoA or AoD combined with a distance; infrared pointers, laser direction finding devices, Smart Device integrated CCDs; LIDAR Wifi triangulation, and the like.
At method step 910, the smart device's distance to the subject area of interest will be determined. The methods used may be, by means of non-limiting example, one or more of: time of flight of a transmitted signal, RTT, phase differentiation, depth orientation of the smart device 1001. One or more position devices 1003-1008 are shown located within, on or proximate to the smart device 1001. The smart device 1001 may include an electronic magnetic sensor 1050. The electronic magnetic sensor 1050 may quantify an ambient magnetic field and generate an orientation of the smart device 1001 along three axis of the ambient magnetic field that correlates with the quantified ambient magnetic field. In some embodiments, a direction of interest 1014 may be quantified in terms of an angle relative to the three axis of the ambient magnetic field.
The transceivers 1003-1008 enter into logical communication with multiple wireless positional reference transceivers 1010-1013 to generate data sufficient to quantify a physical position of the transceivers 1010-1013 and an orientation of the smart device.
In some embodiments, a direction of interest will permit identification of an item of interest 1009, such as an apparatus or other piece of equipment or a direction into the environment. A direction of interest 1014 may include a vector with a direction pointing towards the area of interest 1009. The vector length may be sufficient to reach the item of interest 1009.
Referring to FIG. 11 in some cases, it may be prohibitively inefficient or expensive to “tag” or associate a Node or Smart Device with every item to be monitored. For example, it may be cost and time prohibitive to assign each product being stored in the Facility with a Node or Smart Device. As such, other methods of monitoring the products and other items may be used. In some embodiments of the invention, an item may be assigned a “Virtual Tag,” in which a Smart Device can identify an object in its field of view and capture its location coordinates based on the position of the Smart Device. A “Virtual Tag” or “icon” may be generated in the Smart Device's field of view at those coordinates and saved in the digital memory. Information regarding the object may also be saved with the spatial coordinates. This information can be transmitted to the monitoring system so that others may find the object by routing to the particular coordinates. In addition, when that Smart Device, or another Smart Device in logical communication with the monitoring system, views the particular location, the Smart Device will identify that a virtual tag exists at that location with, for example, an icon. The icon may also be interactive so that information associated with the object may be known. Thus, the monitoring system may be in logical communication with both physical tags such as Smart Devices or Nodes and with Virtual Tags created by a Smart Device. This will be described in further detail below.
Creating of Virtual Tags may, in some embodiments, performed using the following steps: in a first step 1101, an orientation home position is established. At step 1102, the Smart Device initiates transceiving and at step 1103, a coordinate system is selected. If Cartesian coordinate system is selected, at step 1104, the Smart Device may triangulate with Reference Point Transceivers/Nodes to determine the position of the Image Capture Device in the Smart Device (step 1106). If a polar coordinate system is selected, at Step 1505, an angle and distance involved in transceiving is measured, and the camera position is determined (step 1106). In some embodiments, the Image Capture Device may be a camera, including a charge-coupled device (CCD), or a LIDAR apparatus, including a solid-state LIDAR or a MEMS-based LIDAR.
At step 1107, the position of any Nodes or Reference Point Transceivers is verified, and in step 1108, the direction of the Image Capture Device is determined based on the methods described herein, using the Smart Device and any Nodes or Reference Point Transceivers nearby.
At step 1109, the Image Capture Device captures an image. This may be done by capture of wireless energy of a first wavelength is received into a wireless receiver in the Smart Device. In exemplary embodiments, this step may comprise receiving image data based on visible light into a camera of the smart device. The wireless energy may be dispersed over a one-, two-, or three-dimensional space in a defined physical area, and may be received into a one-, two-, or three-dimensional array in the receiver. The wireless energy may take the form of electromagnetic radiation, such as light in the human-visible light spectrum (generally having a wavelength between 380 nm-740 nm), ultraviolet light (generally having a wavelength between 10.0 nm-400 nm), or infrared light (generally having a wavelength between 740 nm-2.00 mm). The set of wireless energy available to the wireless receiver is the smart device's Field of View. In step 1110, the Field of View may change as the angle or position of the Smart Device changes.
In step 1111, the Field of View is quantified. In some embodiments, a pattern of digital values is generated based upon receipt of wireless energy into the wireless receiver. This pattern of digital values may be based on one or more qualities of the received wireless energy, including its intensity, spatial dispersion, wavelength, or angle of arrival. The pattern may be placed into an appropriate array. For example, if the display of the smart device is a two-dimensional display, then the pattern of digital values may comprise a two-dimensional representation of the image data received. In some embodiments, the pattern of digital values may be based on an aggregated set of values from an array of receivers. For example, if the basis of the digital values is the intensity of the wireless energy received into the receiver, then the digital value assigned to a given entry in the array may be based on a weighted average of intensity of wireless energy received at a plurality of the receivers in the array. The set of digital values within the Field of View is the Digital Field of View. In step 1112, optionally, the coordinates of the space in the Digital Field of View may also be determined. Any suitable method may be used, but in some embodiments, LIDAR may be used to determine the presence and distance from the Smart Device of various items in the Field of View and therefore determine the position of items. In step 1113, items or space in the Field of View may be “tagged” such as via a touch screen or cursor. Using similar steps for step 1112, the location coordinates for the Virtual Tags may be known and transmitted, for example, to a monitoring system or controller.
Referring to step 1111, when a Smart Device that is part of the monitoring system approaches the location of the Virtual Tag, the Smart Device will identify that the Virtual Tag is within the Smart Device's Field of View. In step 1115, when this occurs, the Smart Device will generate an icon in the position of the Virtual Tag. This icon may be interactive so that information regarding the location or Virtual Tag may be ascertained. For example, the Virtual Tag may provide a location and environmental history of the product at the coordinates, as will be discussed in further detail below.
Glossary
“Agent” as used herein refers to a person or automation capable of supporting a Smart Device at a geospatial location relative to a Ground Plane.
“Ambient Data” as used herein refers to data and data streams captured in an environment proximate to a Vantage Point and/or an equipment item that are not audio data or video data. Examples of Ambient Data include, but are not limited to Sensor perception of: temperature, humidity, particulate, chemical presence, gas presence, light, electromagnetic radiation, electrical power, moisture and mineral presence.
“Analog Sensor” and “Digital Sensor” as used herein include a Sensor operative to quantify a state in the physical world in an analog representation.
“As Built” as used herein refers to details of a physical structure associated with a specific location within the physical structure or parcel and empirical data captured in relation to the specific location.
“As Built Features” as used herein refers to a feature in a virtual model or AVM that is based at least in part upon empirical data captured at or proximate to a correlating physical location of the feature. Examples of As Built Features include placement of structural components such as a wall, doorway, window, plumbing, electrical utility, machinery and/or improvements to a parcel, such as a well, septic, electric or water utility line, easement, berm, pond, wet land, retaining wall, driveway, right of way and the like.
“Augmented Virtual Model” (sometimes referred to herein as “AVM”): as used herein is a digital representation of a real property parcel including one or more three dimensional representations of physical structures suitable for use and As Built data captured descriptive of the real property parcel. An Augmented Virtual Model includes As Built Features of the structure and may include improvements and features contained within a Processing Facility.
“Bluetooth” as used herein means the Wireless Personal Area Network (WPAN) standards managed and maintained by Bluetooth SIG. Unless otherwise specifically limited to a subset of all Bluetooth standards, the Bluetooth will encompass all Bluetooth standards (e.g. Bluetooth 4.0; 5.0; 5.1 and BLE versions).
“Directional Indicator” as used herein shall mean a quantification of a direction generated via one or both of: analogue and digital indications.
“Directional Image Data” as used herein refers to image data captured from a Vantage Point with reference to a direction. Image data may include video data.
“Directional Audio” as used herein refers to audio data captured from a Vantage Point within or proximate to a property and from a direction.
“Digital Sensor” as used herein includes a Sensor operative to quantify a state in the physical world in a digital representation.
“Ground Plane” as used herein refers to horizontal plane from which a direction of interest may be projected.
“Image Capture Device” or “Scanner” as used herein refers to apparatus for capturing digital or analog image data, an Image capture device may be one or both of: a two dimensional camera (sometimes referred to as “2D”) or a three dimensional camera (sometimes referred to as “3D”). In some examples an Image Capture Device includes a charged coupled device (“CCD”) camera. “Intelligent Automation” as used herein refers to a logical processing by a device, system, machine or equipment item (such as data gathering, analysis, artificial intelligence, and functional operation) and communication capabilities.
“MultiModal” as used herein refers to the ability of a device to communication using multiple protocols and/or bandwidths. Examples of multimodal may include being capable of communication using two to more of: Ultra Wideband, Bluetooth; Bluetooth Low Energy; WiFi; WiFi RT; GPS; ultrasonic; infrared protocols and/or mediums.
“Node” as used herein means a device including at least a processor, a digital storage and a wireless transceiver.
“Proximate” as used herein means physically located within range of a relevant modality of wireless communication.
“Ray” as used herein refers to a straight line including a starting point and extending indefinitely in a direction.
“Sensor” as used herein refers to one or more of a solid state, electro-mechanical, and mechanical device capable of transducing a physical condition or property into an analogue or digital representation and/or metric.
“Smart Device” as used herein includes an electronic device including, or in logical communication with, a processor and digital storage and capable of executing logical commands.
“Smart Receptacle” as used herein includes a case or other receiver of a smart device with components capable of receiving wireless transmissions from multiple wireless positional reference transceivers. In some embodiments, the smart receptacle will include a wireless transmitter and/or a physical connector for creating an electrical path for carrying one or both of electrical power and logic signals between an associated Smart Device and the Smart Receptacle.
“Structure” as used herein refers to a manmade assembly of parts connected in an ordered way. Examples of a Structure in this disclosure include a building; a sub-assembly of a building; a bridge, a roadway, a train track, a train trestle, an aqueduct; a tunnel a dam, and a retainer berm.
“Transceive” as used herein refers to an act of at least one of: transmitting and receiving data.
“Transceiver” as used herein refers to an electronic device capable of one or both of wirelessly transmitting and receiving data.
“Ultra Wideband” sometimes referred to herein as “UWB” means a wireless communication modality which may engage a bandwidth generally greater than 500 MHz and transmit wireless energy in pulses.
“Vantage Point” as used herein refers to a specified location which may be an actual location within a physical environment or a virtual representation of the actual location within a physical environment.
“Vector” as used herein refers to a magnitude and a direction as may be represented and/or modeled by a directed line segment with a length that represents the magnitude and an orientation in space that represents the direction.
“Virtual Tag” as used here shall mean a reference point identified with a location in a virtual model of a space. It may have common aspects with an artificial reality anchor, reference anchor and the like. The tag may have numerous features and content associated with it including data values, user functionality, image data, video data and the like.
INCORPORATION BY REFERENCE TO RELATED APPLICATIONS
This application references the Non Provisional U.S. patent application Ser. No. 16/504,919, filed Jul. 8, 2019 and entitled METHOD AND APPARATUS FOR POSITION BASED QUERY WITH AUGMENTED REALITY HEADGEAR, the entire contents of which are hereby incorporated by reference. This application references and the Non Provisional patent application Ser. No. 16/688,775, filed Nov. 19, 2019 and entitled METHOD AND APPARATUS FOR WIRELESS DETERMINATION OF POSITION AND ORIENTATION OF A SMART DEVICE the entire contents of which are hereby incorporated by reference. This application references the Non Provisional patent application Ser. No. 16/503,878, filed Jul. 5, 2019 and entitled METHOD AND APPARATUS FOR ENHANCED AUTOMATED WIRELESS ORIENTEERING, the entire contents of which are hereby incorporated by reference. This application references the Non Provisional patent application Ser. No. 16/297,383, filed Jul. 5, 2019 and entitled SYSTEM FOR CONDUCTING A SERVICE CALL WITH ORIENTEERING, the entire contents of which are hereby incorporated by reference. This application references the Non Provisional patent application Ser. No. 16/249,574, filed Jan. 16, 2019 and entitled ORIENTEERING SYSTEM FOR RESPONDING TO AN EMERGENCY IN A STRUCTURE, the entire contents of which are hereby incorporated by reference. This application references the Non Provisional patent application Ser. No. 16/176,002, filed Oct. 31, 2018 and entitled SYSTEM FOR CONDUCTING A SERVICE CALL WITH ORIENTEERING, the entire contents of which are hereby incorporated by reference. This application references the Non Provisional patent application Ser. No. 16/171,593, filed Oct. 26, 2018 and entitled SYSTEM FOR HIERARCHICAL ACTIONS BASED UPON MONITORED BUILDING CONDITIONS, the entire contents of which are hereby incorporated by reference. This application references the Non Provisional patent application Ser. No. 16/165,517, filed Oct. 19, 2018 and entitled BUILDING VITAL CONDITIONS MONITORING, the entire contents of which are hereby incorporated by reference. This application references the Non Provisional patent application Ser. No. 16/161,823, filed Oct. 16, 2018 and entitled BUILDING MODEL WITH CAPTURE OF AS BUILT FEATURES AND EXPERIENTIAL DATA, the entire contents of which are hereby incorporated by reference. This application references the Non Provisional patent application Ser. No. 16/142,275, filed Sep. 26, 2018 and entitled METHODS AND APPARATUS FOR ORIENTEERING, the entire contents of which are hereby incorporated by reference. This application references the Non Provisional patent application Ser. No. 15/887,637, filed Feb. 2, 2018 and entitled BUILDING MODEL WITH CAPTURE OF AS BUILT FEATURES AND EXPERIENTIAL DATA, the entire contents of which are hereby incorporated by reference. This application references the Non Provisional patent application Ser. No. 15/716,53, filed Sep. 26, 2017 and entitled BUILDING MODEL WITH VIRTUAL CAPTURE OF AS BUILT FEATURES AND OBJECTIVE PERFORMANCE TRACKING, the entire contents of which are hereby incorporated by reference. This application references the Non Provisional patent application Ser. No. 15/703,310, filed Sep. 5, 2017 and entitled BUILDING MODEL WITH VIRTUAL CAPTURE OF AS BUILT FEATURES AND OBJECTIVE PERFORMANCE TRACKING, the entire contents of which are hereby incorporated by reference. This application references the Non Provisional patent application Ser. No. 16/528,104, filed Jul. 31, 2019 and entitled SMART CONSTRUCTION WITH AUTOMATED DETECTION OF ADVERSE STRUCTURE CONDITIONS AND REMEDIATION, the entire contents of which are hereby incorporated by reference. This application references the Non-Provisional U.S. patent application Ser. No. 16/657,660, filed Oct. 18, 2019 and entitled METHOD AND APPARATUS FOR CONSTRUCTION AND OPERATION OF CONNECTED INFRASTRUCTURE, the entire contents of which are hereby incorporated by reference. This application references the Non-Provisional U.S. patent application Ser. No. 16/721,906, filed Dec. 19, 2019 and entitled METHOD AND APPARATUS FOR WIRELESS DETERMINATION OF POSITION AND ORIENTATION OF A SMART DEVICE, the entire contents of which are hereby incorporated by reference. This application references the Non Provisional patent application Ser. No. 16/549,503, filed Aug. 23, 2019 and entitled METHOD AND APPARATUS FOR AUGMENTED VIRTUAL MODELS AND ORIENTEERING, the entire contents of which are hereby incorporated by reference. This application references the Non Provisional patent application Ser. No. 16/775,223, filed Jan. 28, 2020 and entitled SPATIAL SELF-VERIFYING ARRAY OF NODES, the entire contents of which are hereby incorporated by reference.
Particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order show, or sequential order, to achieve desirable results. In certain implementations, multitasking and parallel processing may be advantageous. Nevertheless, it will be understood that various modifications may be made without departing from the spirit and scope of the claimed invention.

Claims (20)

What is claimed is:
1. Method for displaying a virtual tag on a smart device, the method comprising the steps of:
a) supporting a first transceiver and a magnetic field sensor via an agent at a first geospatial position;
b) generating respective logical communications between the first transceiver and each of a first set of three reference point transceivers, the three reference point transceivers fixedly positioned at disparate points in a structure;
c) with a controller, determining X, Y and Z coordinates of the first transceiver based upon the respective logical communications between the first transceiver and each of the first set of three reference point transceivers;
d) with the controller, determining a direction of interest based upon a quantification of a magnetic field with the magnetic field sensor;
e) capturing an image with a camera located within the smart device;
f) calculating a field of view, based upon an algorithmic calculation comprising input parameters of a set of camera parameters, the generated location and the first direction of interest;
g) accessing data associated with the field of view and determining a presence of at least a first virtual tag within a digital representation of the field of view; and
h) generating a user interface on the smart device, the user interface comprising a display of at least a first data value associated with the first virtual tag.
2. The method of claim 1 wherein the capturing of the image of step h) further includes an acquisition of a LiDAR scan of the field of view.
3. The method of claim 2 additionally comprising the steps of:
reorienting the smart device;
generating a topographical representation of at least a portion of an environment within an area capable of wireless communication with the smart device; and
generating a second direction of interest based upon the topographical representation.
4. The method of claim 1 wherein the capturing of the image of step h) detects infrared energy.
5. The method of claim 1 wherein the capturing of the image of step h) detects ultraviolet energy.
6. The method of claim 1 further comprising displaying an icon on the user interface, wherein the icon is interactive.
7. The method of claim 1 wherein one of the first transceiver and the second transceiver comprises a smart watch.
8. The method of claim 1 wherein the smart device comprises both the first transceiver and the magnetic field sensor.
9. The method of claim 1 wherein at least one of the first set of three reference point transceivers comprises a multi-modality transceiver capable of transceiving in multiple frequency bandwidths.
10. The method of claim 9 wherein the multiple frequency bandwidths comprise bandwidths associated with two or more of: WiFi, Bluetooth, Ultra Wideband, infrared and ultrasonic modalities.
11. Apparatus for displaying a virtual tag on a smart device, the apparatus comprising:
l) a controller comprising a processor and a digital storage, said controller in logical communication with a user interface display;
m) a first transceiver supportable by an Agent at a first geospatial position and in logical communication with the controller;
n) and magnetic field sensor supportable by the Agent at the first geospatial position and in logical communication with the controller;
o) the smart device with at least a first camera device in logical communication with the controller; and
p) executable code stored on the digital storage and executable by the processor to cause the controller to:
I) determine X, Y and Z coordinates of the first transceiver based upon the respective logical communications between the first transceiver and each of a first set of three reference point transceivers;
II) generate a first direction of interest based upon an output of the magnetic field sensor;
V) receive and process a captured image from a camera located within the smart device;
VI) calculate a field of view comprising input parameters of a set of camera parameters, the generated location and the first direction of interest;
VII) access data associated with the field of view and determining a presence of at least a first virtual tag within a digital representation of the field of view; and
VIII) generate a user interface on the smart device, the user interface comprising a display of at least a first data value associated with the first virtual tag.
12. The apparatus of claim 11 wherein the executable code stored on the digital storage and executable by the processor additionally causes the controller to generate an icon for the first virtual tag.
13. The apparatus of claim 12 wherein a touch screen of the smart device senses a user selection of the icon and the executable code stored on the digital storage and executable by the processor causes a display of data associated with the icon for the first virtual tag.
14. The apparatus of claim 12 wherein the smart device additionally comprises a LiDAR camera.
15. The apparatus of claim 11 wherein one of the first transceiver and the second transceiver comprises a smart watch.
16. The apparatus of claim 11 wherein one of the first transceiver and the second transceiver comprises a smart ring.
17. The apparatus of claim 14 wherein the first camera device detects infrared energy.
18. The apparatus of claim 14 wherein the first camera device detects ultraviolet energy.
19. The apparatus of claim 11 wherein at least one of the first set of three reference point transceivers comprises a multi-modality transceiver capable of transceiving in multiple frequency bandwidths.
20. The apparatus of claim 19 wherein the multiple frequency bandwidths comprise bandwidths associated with two or more of: WiFi, Bluetooth, Ultra Wideband, infrared and ultrasonic modalities.
US17/398,804 2017-02-22 2021-08-10 Method and apparatus for improved position and orientation based information display Active US11475177B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/398,804 US11475177B2 (en) 2017-02-22 2021-08-10 Method and apparatus for improved position and orientation based information display

Applications Claiming Priority (31)

Application Number Priority Date Filing Date Title
US201762462347P 2017-02-22 2017-02-22
US201762531955P 2017-07-13 2017-07-13
US201762531975P 2017-07-13 2017-07-13
US15/703,310 US20180239313A1 (en) 2017-02-22 2017-09-13 Building model with virtual capture of as built features and objective performance tracking
US15/716,133 US10025887B1 (en) 2017-02-22 2017-09-26 Building model with virtual capture of as built features and objective performance tracking
US15/887,637 US20180239840A1 (en) 2017-02-22 2018-02-02 Building model with capture of as built features and experiential data
US201862712714P 2018-07-31 2018-07-31
US16/142,275 US10433112B2 (en) 2017-02-22 2018-09-26 Methods and apparatus for orienteering
US16/161,823 US10467353B2 (en) 2017-02-22 2018-10-16 Building model with capture of as built features and experiential data
US16/165,517 US10733334B2 (en) 2017-02-22 2018-10-19 Building vital conditions monitoring
US16/171,593 US10620084B2 (en) 2017-02-22 2018-10-26 System for hierarchical actions based upon monitored building conditions
US16/176,002 US10268782B1 (en) 2017-02-22 2018-10-31 System for conducting a service call with orienteering
US201862769133P 2018-11-19 2018-11-19
US16/249,574 US10831943B2 (en) 2017-02-22 2019-01-16 Orienteering system for responding to an emergency in a structure
US201962793714P 2019-01-17 2019-01-17
US16/297,383 US10762251B2 (en) 2017-02-22 2019-03-08 System for conducting a service call with orienteering
PCT/US2019/033134 WO2020068177A1 (en) 2018-09-26 2019-05-20 Method and apparatus for augmented virtual models and orienteering
US16/503,878 US10776529B2 (en) 2017-02-22 2019-07-05 Method and apparatus for enhanced automated wireless orienteering
US201962871499P 2019-07-08 2019-07-08
US16/504,919 US10740502B2 (en) 2017-02-22 2019-07-08 Method and apparatus for position based query with augmented reality headgear
US16/528,104 US10671767B2 (en) 2017-02-22 2019-07-31 Smart construction with automated detection of adverse structure conditions and remediation
US201962909061P 2019-10-01 2019-10-01
US16/597,271 US20200050712A1 (en) 2017-02-22 2019-10-09 Building model with capture of as built features and experiential data
US16/657,660 US20200065433A1 (en) 2017-02-22 2019-10-18 Method and apparatus for construction and operation of connected infrastructure
US16/688,775 US10628617B1 (en) 2017-02-22 2019-11-19 Method and apparatus for wireless determination of position and orientation of a smart device
US16/721,906 US10726167B2 (en) 2017-02-22 2019-12-19 Method and apparatus for determining a direction of interest
US16/775,233 US20210041440A1 (en) 2006-05-01 2020-01-28 Methods and apparatus for identifying disease status using biomarkers
US16/817,926 US10831945B2 (en) 2017-02-22 2020-03-13 Apparatus for operation of connected infrastructure
US16/898,602 US10949579B2 (en) 2017-02-22 2020-06-11 Method and apparatus for enhanced position and orientation determination
US17/183,062 US11106837B2 (en) 2017-02-22 2021-02-23 Method and apparatus for enhanced position and orientation based information display
US17/398,804 US11475177B2 (en) 2017-02-22 2021-08-10 Method and apparatus for improved position and orientation based information display

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US17/183,062 Continuation US11106837B2 (en) 2017-02-22 2021-02-23 Method and apparatus for enhanced position and orientation based information display

Publications (2)

Publication Number Publication Date
US20210374300A1 US20210374300A1 (en) 2021-12-02
US11475177B2 true US11475177B2 (en) 2022-10-18

Family

ID=83594592

Family Applications (1)

Application Number Title Priority Date Filing Date
US17/398,804 Active US11475177B2 (en) 2017-02-22 2021-08-10 Method and apparatus for improved position and orientation based information display

Country Status (1)

Country Link
US (1) US11475177B2 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11946761B2 (en) 2018-06-04 2024-04-02 The Research Foundation For The State University Of New York System and method associated with expedient determination of location of one or more object(s) within a bounded perimeter of 3D space based on mapping and navigation to a precise POI destination using a smart laser pointer device

Citations (227)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5884202A (en) 1995-07-20 1999-03-16 Hewlett-Packard Company Modular wireless diagnostic test and information system
US5933479A (en) 1998-10-22 1999-08-03 Toyoda Machinery Usa Corp. Remote service system
US6006021A (en) 1996-07-01 1999-12-21 Sun Microsystems, Inc. Device for mapping dwellings and other structures in 3D
US6292108B1 (en) 1997-09-04 2001-09-18 The Board Of Trustees Of The Leland Standford Junior University Modular, wireless damage monitoring system for structures
US6321158B1 (en) 1994-06-24 2001-11-20 Delorme Publishing Company Integrated routing/mapping information
US20020095269A1 (en) 2001-01-17 2002-07-18 Francesco Natalini System for monitoring and servicing appliances
US20020181405A1 (en) 2000-04-10 2002-12-05 I/O Controls Corporation System for providing remote access to diagnostic information over a wide area network
US20030110001A1 (en) 2001-12-12 2003-06-12 Chassin David P. Rooftop package unit diagnostician
US20030135324A1 (en) 2001-11-20 2003-07-17 Nassir Navab System and method for tracking, locating, and guiding within buildings
US20030163440A1 (en) 2002-02-22 2003-08-28 First Data Corporation Maintenance request systems and methods
US20030195008A1 (en) 2000-07-14 2003-10-16 Norman Mohi Locating system and method
US20040002786A1 (en) 2002-06-27 2004-01-01 Takashi Sasaki Method of predicting dryer steam pressure in paper machine and apparatus for the method
US20040122628A1 (en) 2002-12-03 2004-06-24 Laurie Michael Stuart Method and device for generating two-dimensional floor plans
US20040119662A1 (en) 2002-12-19 2004-06-24 Accenture Global Services Gmbh Arbitrary object tracking in augmented reality applications
US6853958B1 (en) 2000-06-30 2005-02-08 Integrex System and method for collecting and disseminating household information and for coordinating repair and maintenance services
US20050165576A1 (en) 2004-01-26 2005-07-28 Jesmonth Richard E. System and method for generating three-dimensional density-based defect map
US20050208951A1 (en) 2002-05-31 2005-09-22 Armando Annunziato Method for locating mobile terminals, system and components therefor
US20050275525A1 (en) 2004-03-25 2005-12-15 Siemens Building Technologies, Inc. Method and apparatus for graphical display of a condition in a building system with a mobile display unit
US20060028345A1 (en) 2002-10-09 2006-02-09 Peng Lee Termite acoustic detection
US20060084436A1 (en) 2004-06-29 2006-04-20 Michael Green Methods and apparatus for inter-BSC soft handoff
US20060084463A1 (en) 2004-10-04 2006-04-20 Curitel Communications, Inc. Mobile communication terminal having emergency call function and method of processing emergency call request in the mobile communication terminal
US7057557B2 (en) 2001-04-20 2006-06-06 Lg Electronics Inc. Apparatus and method for estimating position of mobile communication terminal
US20060164382A1 (en) 2005-01-25 2006-07-27 Technology Licensing Company, Inc. Image manipulation in response to a movement of a display
US20070004449A1 (en) 2005-06-29 2007-01-04 Sham John C Mobile communication device with environmental sensors
US20070266395A1 (en) 2004-09-27 2007-11-15 Morris Lee Methods and apparatus for using location information to manage spillover in an audience monitoring system
US20080103835A1 (en) 2006-10-31 2008-05-01 Caterpillar Inc. Systems and methods for providing road insurance
US20080180246A1 (en) 2007-01-31 2008-07-31 Symbol Technologies, Inc. Methods and apparatus for opportunistic locationing of rf tags using location triggers
US20080246734A1 (en) 2007-04-04 2008-10-09 The Hong Kong University Of Science And Technology Body movement based usage of mobile device
WO2008124713A2 (en) 2007-04-09 2008-10-16 Sam Stathis A system and method capable of navigating and/or mapping any multi-dimensional space
US20080319654A1 (en) 2007-06-22 2008-12-25 Sony Corporation System and method for effectively implementing an electronic navigation device
US20090097710A1 (en) 2006-05-22 2009-04-16 Rafael Advanced Defense Systems Ltd. Methods and system for communication and displaying points-of-interest
US20090189810A1 (en) 2008-01-24 2009-07-30 Broadcom Corporation Weighted aiding for positioning systems
US20090216438A1 (en) 2008-02-21 2009-08-27 Microsoft Corporation Facility map framework
US20090293051A1 (en) 2008-05-22 2009-11-26 Fortinet, Inc., A Delaware Corporation Monitoring and dynamic tuning of target system performance
US20100103036A1 (en) 2008-10-28 2010-04-29 Audiovox Corporation Portable transceiver with vehicle security control and locate features
US20100107635A1 (en) 2007-03-02 2010-05-06 Kenji Tsubone Thermal storage device
US20100271263A1 (en) 2008-03-31 2010-10-28 Mehran Moshfeghi Method and System for Determining the Position of a Mobile Station
US20100296075A1 (en) 2007-10-26 2010-11-25 Leica Geosystems Ag Distance-measuring method for a device projecting a reference line, and such a device
US20100309044A1 (en) 2009-06-05 2010-12-09 Qualcomm Incorporated On Demand Positioning
WO2011019810A1 (en) 2009-08-11 2011-02-17 Certusview Technologies, Llc Locating equipment equipped with a mobile/portable device
US20110047516A1 (en) 2009-08-18 2011-02-24 Honeywell Asca, Inc. Rapid manipulation of flowsheet configurations
US20110068906A1 (en) 2009-09-21 2011-03-24 Checkpoint Systems, Inc. Systems, methods, and apparatuses for managing configurable monitoring devices
US20110075835A1 (en) 2009-09-30 2011-03-31 Apple Inc. Self adapting haptic device
US20110115816A1 (en) 2009-11-16 2011-05-19 Alliance For Sustainable Energy, Llc. Augmented reality building operations tool
US20110153101A1 (en) 2009-12-22 2011-06-23 General Electric Company Household energy management system and method for one or more appliances
US20110182202A1 (en) 2008-09-26 2011-07-28 Huawei Technologies Co., Ltd. Wireless service provision
US7994981B1 (en) 2010-09-07 2011-08-09 etherwhere Corporation System framework for mobile device location
CN102147597A (en) 2010-02-10 2011-08-10 广州大学 Health monitoring system for structures of great building and bridge
US20110199479A1 (en) * 2010-02-12 2011-08-18 Apple Inc. Augmented reality maps
US20110251787A1 (en) 2010-04-07 2011-10-13 Qualcomm Incorporated Navigational Coordinate Systems in Conjunction with Transports and/or Mobile Devices
US20120087212A1 (en) 2010-10-08 2012-04-12 Harry Vartanian Apparatus and method for providing indoor location or position determination of object devices using building information and/or powerlines
US20120127976A1 (en) 2010-11-23 2012-05-24 Symbol Technologies, Inc. Radio frequency identification system and related operating methods
US20120127306A1 (en) 2010-11-19 2012-05-24 Honeywell International Inc. Security video detection of personal distress and gesture commands
US20120173456A1 (en) 2010-11-24 2012-07-05 Hirl Joseph P Decision support system for the management of energy use, contracting and capital investments for facilities
US20120188847A1 (en) 2009-07-31 2012-07-26 Nec Corporation Position detection apparatus, position detection method, mobile, and receiver
US20120204646A1 (en) 2011-02-10 2012-08-16 National Taiwan University Of Science And Technology Method for analyzing structure safety
US8280404B1 (en) * 2011-03-31 2012-10-02 Google Inc. Directional feedback
US20120296610A1 (en) 2011-05-17 2012-11-22 Ebenezer Hailemariam Occupant centric capture and visualization of building performance data
US20130010103A1 (en) 2010-03-14 2013-01-10 Ns Solutions Corporation Information processing system, information processing method and program, information processing apparatus, vacant space guidance system, vacant space guidance method and program, image display system, image display method and program
US20130073059A1 (en) 2011-09-20 2013-03-21 Joseph Mark Brian User interface for determining resource consumption
US20130084886A1 (en) 2011-10-03 2013-04-04 Cellular Specialties, Inc. Pilot beacon system for indoor positioning
US20130120630A1 (en) 2011-11-10 2013-05-16 Samsung Electronics Co., Ltd. System and method for controlling temperature in mobile device
US20130169801A1 (en) 2011-12-28 2013-07-04 Pelco, Inc. Visual Command Processing
US20130169681A1 (en) 2011-06-29 2013-07-04 Honeywell International Inc. Systems and methods for presenting building information
GB2498177A (en) 2011-12-21 2013-07-10 Max Christian Apparatus for determining a floor plan of a building
US20130197685A1 (en) 2012-01-27 2013-08-01 Fujitsu Limited Medium storing diagram generation program, diagram generation method, and diagram generation apparatus
US20130201311A1 (en) 2011-08-01 2013-08-08 Olympus Medical Systems Corp. Apparatus for estimating shape of insertion portion
US20130223261A1 (en) 2008-11-21 2013-08-29 Qualcomm Incorporated Processing time determination for wireless position determination
US20130288719A1 (en) 2012-04-27 2013-10-31 Oracle International Corporation Augmented reality for maintenance management, asset management, or real estate management
US20130283529A1 (en) 2005-03-29 2013-10-31 Stryker Corporation Patient support apparatus communication systems
US20130297555A1 (en) 2010-09-14 2013-11-07 Nest Labs, Inc. Occupancy pattern detection, estimation and prediction
US20130345975A1 (en) 2012-06-05 2013-12-26 Apple Inc. Navigation application with adaptive display of graphical directional indicators
KR101354688B1 (en) 2013-07-05 2014-01-27 한국건설기술연구원 System and method for supervision of construction site
US20140058572A1 (en) 2012-08-27 2014-02-27 Gridium, Inc. Systems and methods for energy consumption and energy demand management
US20140084909A1 (en) 2012-09-25 2014-03-27 Stmicroelectronics S.R.L. Block made of a building material
US20140107828A1 (en) 2012-10-15 2014-04-17 International Business Machines Corporation Method and System for Wafer Quality Predictive Modeling based on Multi-Source Information with Heterogeneous Relatedness
US20140129599A1 (en) 2012-11-07 2014-05-08 Intertrust Technologies Corporation Personalized data management systems and methods
US20140146038A1 (en) 2012-11-28 2014-05-29 International Business Machines Corporation Augmented display of internal system components
US20140156455A1 (en) 2012-11-30 2014-06-05 Polar Technology, LLC Refrigerant Exchange, Recycling and Banking System and Computer Program Product
US20140188394A1 (en) 2012-12-27 2014-07-03 International Business Machines Corporation Determining seismic response characteristics of structures
US20140210856A1 (en) 2013-01-30 2014-07-31 F3 & Associates, Inc. Coordinate Geometry Augmented Reality Process for Internal Elements Concealed Behind an External Element
US8818758B1 (en) 2010-03-01 2014-08-26 Wegowise, Inc. Methods and apparatus to track, visualize and understand energy and utilities usage
US20140244160A1 (en) 2013-01-04 2014-08-28 MoneyDesktop, Inc. a Delaware Corporation Augmented reality financial institution branch locator
US20140267420A1 (en) 2013-03-15 2014-09-18 Magic Leap, Inc. Display system and method
US20140266755A1 (en) 2013-03-15 2014-09-18 Emerson Electric Co. Hvac system remote monitoring and diagnosis
US20140277594A1 (en) 2013-03-15 2014-09-18 Fisher-Rosemount Systems, Inc. Method and apparatus for seamless state transfer between user interface devices in a mobile control room
US20140274151A1 (en) 2013-03-15 2014-09-18 Nextnav, Llc Systems and Methods for Using Three-Dimensional Location Information to improve Location Services
US20140278065A1 (en) 2013-03-14 2014-09-18 Robert Bosch Gmbh System and Method for Distortion Correction in Three-Dimensional Environment Visualization
US8843350B2 (en) 2011-06-03 2014-09-23 Walter P. Moore and Associates, Inc. Facilities management system
US8862415B1 (en) 2009-01-29 2014-10-14 Comverge Inc. Method and system for calculating energy metrics of a building and one or more zones within the building
US20140368373A1 (en) 2011-12-20 2014-12-18 Sadar 3D, Inc. Scanners, targets, and methods for surveying
WO2014204753A1 (en) 2013-06-21 2014-12-24 Qualcomm Incorporated Determination of positioning information of a mobile device using modulated light signals
US20150005903A1 (en) 2013-06-28 2015-01-01 Fisher-Rosemount Systems, Inc. Non-Intrusive Data Analytics in a Process Control System
US8965741B2 (en) 2012-04-24 2015-02-24 Microsoft Corporation Context aware surface scanning and reconstruction
US8996156B2 (en) 2009-08-20 2015-03-31 Siemens Aktiengesellschaft Automated repair method and system
US20150094865A1 (en) 2013-10-01 2015-04-02 Yonghyun Choi Heating, ventilation, and/or air conditioning controller
US20150094081A1 (en) 2013-09-30 2015-04-02 Qualcomm Incorporated Determining coordinates of access points in an indoor position location system
US9008979B2 (en) 2010-10-14 2015-04-14 Azbil Corporation Total energy quantity controlling device and method
US20150116132A1 (en) 2013-10-31 2015-04-30 Delta Systems, Inc. Smart monitoring system and method of operation
US20150121222A1 (en) 2012-09-06 2015-04-30 Alberto Daniel Lacaze Method and System for Visualization Enhancement for Situational Awareness
US9037124B1 (en) 2013-03-27 2015-05-19 Open Invention Network, Llc Wireless device application interaction via external control detection
US20150137967A1 (en) 2013-07-15 2015-05-21 Oneevent Technologies, Inc. Owner controlled evacuation system
US20150142179A1 (en) 2012-06-15 2015-05-21 Mitsubishi Electric Corporation Air-conditioning management device, air-conditioning management method, and program
US20150154803A1 (en) 2009-10-12 2015-06-04 Metaio Gmbh Method for representing virtual information in a view of a real environment
US20150156423A1 (en) 2013-11-29 2015-06-04 Axis Ab System for following an object marked by a tag device with a camera
US20150168154A1 (en) 2012-01-03 2015-06-18 Google Inc. Systems, computer-implemented methods, and computer-readable media to provide building information to on-site mobile-devices
US9064219B2 (en) 2010-05-05 2015-06-23 J. E. Dunn Construction Group, Inc. System and method for managing facility content and equipment information
US20150178633A1 (en) 2013-12-20 2015-06-25 Johnson Controls Technology Company Systems and methods for determining the uncertainty in parameters of an energy use model
US20150177718A1 (en) 2012-07-11 2015-06-25 Abb Research Ltd Presenting process data of a process control object on a mobile terminal
US20150206348A1 (en) 2012-09-07 2015-07-23 Hitachi Maxell, Ltd. Reception device
US20150207316A1 (en) 2012-08-16 2015-07-23 Robert Bosch Gmbh Dc building system with energy storage and control system
US20150227123A1 (en) 2014-02-11 2015-08-13 Honeywell International Inc. Mobile extension for industrial operator consoles
US20150262421A1 (en) 2014-03-14 2015-09-17 Matterport, Inc. Processing and/or transmitting 3d data
US20150287318A1 (en) 2014-04-04 2015-10-08 Rockwell Automation Technologies, Inc. Industrial-enabled mobile device
US20150294506A1 (en) 2014-04-15 2015-10-15 Huntington Ingalls, Inc. System and Method for Augmented Reality Display of Dynamic Environment Information
US20150327010A1 (en) 2014-05-07 2015-11-12 Johnson Controls Technology Company Systems and methods for detecting and using equipment location in a building management system
US20150347854A1 (en) 2014-04-25 2015-12-03 Huntington Ingalls Incorporated System and Method for Using Augmented Reality Display in Surface Treatment Procedures
US20150356786A1 (en) 2014-06-09 2015-12-10 Huntington Ingalls, Inc. System and Method for Augmented Reality Display of Electrical System Information
US20150356789A1 (en) 2013-02-21 2015-12-10 Fujitsu Limited Display device and display method
US20160004805A1 (en) 2012-11-16 2016-01-07 Johnson Controls Technology Company Systems and methods for implementing energy conservation measures in a building
US20160019721A1 (en) 2014-07-16 2016-01-21 Huntington Ingalls Incorporated System and Method for Augmented Reality Display of Hoisting and Rigging Information
US20160026729A1 (en) 2014-05-30 2016-01-28 Reylabs Inc Systems and methods involving mobile indoor energy efficiency exploration, monitoring and/or display aspects
US20160026253A1 (en) 2014-03-11 2016-01-28 Magic Leap, Inc. Methods and systems for creating virtual and augmented reality
US20160048200A1 (en) 2013-12-06 2016-02-18 Lg Electronics Inc. Smart watch and control method thereof
US20160066068A1 (en) 2014-09-03 2016-03-03 Oberon, Inc. Environmental Monitor Device with Database
US20160063671A1 (en) 2012-08-30 2016-03-03 Nokia Corporation A method and apparatus for updating a field of view in a user interface
US20160071196A1 (en) 2014-09-08 2016-03-10 Leeo, Inc. Systems and methods for transferring data and revenue
US20160069675A1 (en) 2014-09-04 2016-03-10 Hitachi Construction Machinery Co., Ltd. Position calculating system and haulage vehicle
US20160077532A1 (en) 2012-07-27 2016-03-17 Assa Abloy Ab Setback controls based on out-of-room presence information
US20160084936A1 (en) 2011-08-18 2016-03-24 Rivada Research, Llc. Method and System for Providing Enhanced Location Based Service (ELBS) Trilateration using a Single Device
US20160091217A1 (en) 2014-09-29 2016-03-31 Koninklijke Philips N.V. Systems and methods for managing environmental conditions
US20160095188A1 (en) 2014-09-29 2016-03-31 Koninklijke Philips N.V. Systems and methods for managing environmental conditions
US20160135006A1 (en) * 2014-11-07 2016-05-12 Uber Technologies, Inc. Dynamically controlling the presentation of a graphic feature based on location information
US20160132046A1 (en) 2013-03-15 2016-05-12 Fisher-Rosemount Systems, Inc. Method and apparatus for controlling a process plant with wearable mobile control devices
US20160178383A1 (en) 2014-12-19 2016-06-23 Here Global B.V. User Interface for Displaying Navigation Information in a Small Display
US20160183057A1 (en) 2014-12-18 2016-06-23 Intel Corporation Method and system for hybrid location detection
US20160202678A1 (en) 2013-11-11 2016-07-14 Osram Sylvania Inc. Human presence detection commissioning techniques
US20160216879A1 (en) 2013-09-24 2016-07-28 Lg Electronics Inc. Mobile terminal and method for controlling same
WO2016130571A1 (en) 2015-02-09 2016-08-18 Carnegie Mellon University Indoor identification of individuals through footstep induced structural vibration
US20160238692A1 (en) 2015-02-13 2016-08-18 Position Imaging, Inc. Accurate geographic tracking of mobile devices
US20160258760A1 (en) 2015-03-04 2016-09-08 Lenovo (Singapore) Pte. Ltd. Providing directions to a location in a facility
US20160285416A1 (en) 2015-03-24 2016-09-29 Carrier Corporation System and method for energy harvesting system planning and performance
US20160284128A1 (en) 2015-03-27 2016-09-29 Rockwell Automation Technologies, Inc. Systems and methods for presenting an augmented reality
US20160284075A1 (en) 2013-03-14 2016-09-29 Essess, Inc. Methods, apparatus, and systems for structural analysis using thermal imaging
US20160323149A1 (en) 2015-04-30 2016-11-03 International Business Machines Corporation Automatic Measurement Expression Update
US20160335731A1 (en) 2010-05-05 2016-11-17 Site 10.01, Inc. System and method for monitoring and managing information
US20160345137A1 (en) 2015-05-21 2016-11-24 Toshiba America Business Solutions, Inc. Indoor navigation systems and methods
US20160343093A1 (en) 2014-01-24 2016-11-24 Telvent Usa Llc Utility resource asset management system
US20160343243A1 (en) 2015-05-21 2016-11-24 Google Inc. Method and system for consolidating events across sensors
US20160360429A1 (en) 2014-02-19 2016-12-08 Huawei Technologies Co., Ltd. Method and Device for Wirelessly Accessing Service
WO2016192916A1 (en) 2015-06-02 2016-12-08 Qualcomm Technologies International, Ltd. Intuitive way to point, access and control appliances & other objects in building interiors
US9529072B2 (en) 2013-01-04 2016-12-27 Ortovox Sportartikel Gmbh Search system and method for searching for a buried person
US20160379083A1 (en) 2015-06-24 2016-12-29 Microsoft Technology Licensing, Llc Real-time, model-based object detection and pose estimation
US20170006487A1 (en) 2013-11-27 2017-01-05 At&T Intellectual Property I, L.P. Direct interaction between a user and a communication network
US20170023659A1 (en) 2015-05-08 2017-01-26 5D Robotics, Inc. Adaptive positioning system
US20170055126A1 (en) 2014-09-24 2017-02-23 James Thomas O'Keeffe System and method for user profile enabled smart building control
US20170079001A1 (en) 2015-09-15 2017-03-16 Google Inc. Radio Beacon for Direction Detection
US20170115022A1 (en) 2015-10-21 2017-04-27 Setra Systems, Inc. Room condition monitoring system
US20170131426A1 (en) 2014-06-25 2017-05-11 Robert Bosch Gmbh Location Device and Method for Operating a Location Device
US9668106B2 (en) 2015-05-28 2017-05-30 Hon Hai Precision Industry Co., Ltd. Indoor positioning system and method for indoor position
US20170169683A1 (en) 2015-12-09 2017-06-15 Fire & Risk Alliance, LLC System and methods for detecting, confirming, classifying, and monitoring a fire
US9703517B2 (en) 2014-09-02 2017-07-11 Adobe Systems Incorporated External device screen targeting
US20170200312A1 (en) 2016-01-11 2017-07-13 Jeff Smith Updating mixed reality thumbnails
US20170212668A1 (en) 2016-01-22 2017-07-27 Johnson Controls Technology Company Building energy management system with ad hoc dashboard
US20170234962A1 (en) 2014-08-12 2017-08-17 Philips Lighting Holding B.V. Method and apparatus for locating of a mobile device
US20170237892A1 (en) 2016-02-16 2017-08-17 Fujitsu Limited Non-transitory computer-readable storage medium, control method, and computer
US20170235290A1 (en) 2014-10-14 2017-08-17 Yotta Green Ltd Smart home system and method
WO2017149526A2 (en) 2016-03-04 2017-09-08 May Patents Ltd. A method and apparatus for cooperative usage of multiple distance meters
US9772396B2 (en) 2015-02-09 2017-09-26 Ricoh Company, Ltd. Relative orientation angle calculation method and device as well as relative positioning method
US20170286568A1 (en) 2016-03-31 2017-10-05 M.C. Dean Inc. System and method for testing the validity of bim-designed electrical wiring pathways
US20170289344A1 (en) 2016-03-31 2017-10-05 International Business Machines Corporation Contextually Configuring Consistent Proximity Beacon Identifiers
US9792020B1 (en) 2015-12-30 2017-10-17 Palantir Technologies Inc. Systems for collecting, aggregating, and storing data, generating interactive user interfaces for analyzing data, and generating alerts based upon collected data
US20170363504A1 (en) 2016-06-21 2017-12-21 Thomas Arthur Winant System and method for determining the risk of failure of a structure
WO2017223008A1 (en) 2016-06-21 2017-12-28 Osram Sylvania Inc. Wireless beacon-enabled luminaire identification system and method for determining the position of a portable device
US20180018826A1 (en) 2016-07-15 2018-01-18 Beckhoff Automation Gmbh Method for controlling an object
US20180031618A1 (en) 2015-02-11 2018-02-01 Yehudah FRIEDLANDER System for analyzing electricity consumption
US20180035263A1 (en) 2013-10-08 2018-02-01 Gozio, Inc. Use of rf-based fingerprinting for indoor positioning by mobile technology platforms
US9892559B2 (en) 2014-06-25 2018-02-13 Digital Electronics Corporation Portable terminal device, and portable control device
US20180048693A1 (en) 2016-08-09 2018-02-15 The Joan and Irwin Jacobs Technion-Cornell Institute Techniques for secure data management
US20180075168A1 (en) 2015-03-24 2018-03-15 Carrier Corporation System and method for capturing and analyzing multidimensional building information
US20180084623A1 (en) 2015-04-30 2018-03-22 Zumtobel Lighting Gmbh Method and System for Improving Lighting Control and Method and System for Controlling a Lighting Device
US9934335B2 (en) 2012-10-08 2018-04-03 Hexagon Technology Center Gmbh Method and system for virtual assembly of a structure
US20180101803A1 (en) 2015-03-24 2018-04-12 Carrier Corporation Integrated system for sales, installation, and maintenance of building systems
US20180102858A1 (en) 2015-03-24 2018-04-12 Carrier Corporation System and method for determining rf sensor performance relative to a floor plan
US20180108079A1 (en) 2016-10-13 2018-04-19 River Ventures Spa Augmented Reality E-Commerce Platform
US9952569B2 (en) 2012-04-19 2018-04-24 Schneider Electric Buildings Llc Association of a portable sensor device in a building management system
US20180113506A1 (en) 2016-10-25 2018-04-26 Oculus Vr, Llc Position tracking system that exploits arbitrary configurations to determine loop closure
US20180130260A1 (en) 2016-11-08 2018-05-10 Rockwell Automation Technologies, Inc. Virtual reality and augmented reality for industrial automation
US20180131907A1 (en) 2016-11-08 2018-05-10 Rockwell Automation Technologies, Inc. Virtual reality and augmented reality for industrial automation
US20180159904A1 (en) 2016-10-06 2018-06-07 Abb Technology Oy Remote live support tool
US20180206096A1 (en) 2017-01-18 2018-07-19 Johnson Controls Technology Company Systems and methods for enhancing building management system interaction and visualization
US10042341B1 (en) 2015-02-19 2018-08-07 State Farm Mutual Automobile Insurance Company Systems and methods for monitoring building health
US20180242907A1 (en) 2015-08-10 2018-08-30 Koninklijke Philips N.V. Determining metabolic parameters using wearables
EP2726817B1 (en) 2011-06-30 2018-09-12 Qualcomm Incorporated Navigation in buildings with rectangular floor plan
US20180285482A1 (en) 2017-03-28 2018-10-04 Faro Technologies, Inc. System and method of scanning an environment and generating two dimensional images of the environment
US20180295526A1 (en) 2017-04-07 2018-10-11 Wireless Applications Corp. Radio signal path design tool with graphical display to facilitate selection of alternative radio antenna sites
US20180328753A1 (en) 2017-05-09 2018-11-15 Raven Telemetry Inc. Local location mapping method and system
US10130285B1 (en) 2017-09-24 2018-11-20 Wisilica Inc Wearable apparatus for patient tracking
US10149141B1 (en) 2014-05-13 2018-12-04 Senseware, Inc. System, method and apparatus for building operations management
US20180357823A1 (en) 2017-06-08 2018-12-13 Honeywell International Inc. Apparatus and method for visual-assisted training, collaboration, and monitoring in augmented/virtual reality in industrial automation systems and other systems
US20180374269A1 (en) 2017-06-21 2018-12-27 INVRSE Reality Limited Augmented reality and virtual reality mobile device user interface automatic calibration
US20190025905A1 (en) 2016-10-18 2019-01-24 Raytheon Company Avionics maintenance training
US20190057169A1 (en) 2017-02-22 2019-02-21 Middle Chart, LLC Building Vital Conditions Monitoring
US10222301B2 (en) 2016-05-04 2019-03-05 Embraer S.A. Structural health monitoring system with the identification of the damage through a device based in augmented reality technology
US20190096232A1 (en) 2017-06-27 2019-03-28 Oneevent Technologies, Inc. Augmented reality of a building
US10278016B2 (en) 2017-02-03 2019-04-30 Qualcomm Incorporated Systems and methods for position estimation using proximity devices
US20190145648A1 (en) 2015-10-28 2019-05-16 Johnson Controls Technology Company Thermostat with halo light system and emergency directions
US20190156576A1 (en) 2017-11-20 2019-05-23 Bernard Ndolo Method and system for curating, accessing, and displaying a plurality of data records pertaining to premises, and a plurality of devices installed in the premises
US10355351B2 (en) 2014-04-21 2019-07-16 Maxtena, Inc. Antenna array pointing direction estimation and control
US20190216333A1 (en) 2018-01-12 2019-07-18 Futurewei Technologies, Inc. Thermal face image use for health estimation
US20190228370A1 (en) 2018-01-24 2019-07-25 Andersen Corporation Project management system with client interaction
US20190268062A1 (en) 2018-02-23 2019-08-29 Precision Integrated Photonics, Inc. Disadvantaged node discovery
US20190281573A1 (en) 2018-03-06 2019-09-12 Motorola Mobility Llc Location Correction Apparatus and Method in a Real-Time Locating System
US20190294834A1 (en) 2017-12-07 2019-09-26 Kacchip, LLC Virtual Mapping of an Indoor Space Using Indoor Position and Vector Tracking
US10444324B2 (en) 2016-02-22 2019-10-15 Nida Tech Sweden Ab Single node location system and method
US20190355177A1 (en) 2018-05-15 2019-11-21 Honeywell International Inc. Building system maintenance using mixed reality
US20190392088A1 (en) 2017-02-22 2019-12-26 Middle Chart, LLC Smart construction with automated detection of adverse structure conditions and remediation
US20190392640A1 (en) 2018-06-26 2019-12-26 Lenovo (Singapore) Pte. Ltd. Presentation of augmented reality images at display locations that do not obstruct user's view
US20200057825A1 (en) 2016-10-24 2020-02-20 Wemarathon System for improving the design, construction and operation of a structure
US20200072538A1 (en) 2018-05-30 2020-03-05 Lineage Logistics, LLC Thermal control system
US20200108926A1 (en) 2018-10-03 2020-04-09 Sarcos Corp. Aerial Vehicles Having Countermeasures Deployed From a Platform for Neutralizing Target Aerial Vehicles
US20200151450A1 (en) 2018-11-13 2020-05-14 Kabushiki Kaisha Toshiba Maintenance work support system and maintenance work support method
US20200286300A1 (en) 2019-03-06 2020-09-10 Quanta Computer Inc. Electronic device and method for adjusting size of three-dimensional object in augmented reality
US10825247B1 (en) 2019-11-12 2020-11-03 Zillow Group, Inc. Presenting integrated building information using three-dimensional building models
US10824774B2 (en) 2019-01-17 2020-11-03 Middle Chart, LLC Methods and apparatus for healthcare facility optimization
US10872179B2 (en) 2017-02-22 2020-12-22 Middle Chart, LLC Method and apparatus for automated site augmentation
US11106837B2 (en) * 2017-02-22 2021-08-31 Middle Chart, LLC Method and apparatus for enhanced position and orientation based information display

Patent Citations (230)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6321158B1 (en) 1994-06-24 2001-11-20 Delorme Publishing Company Integrated routing/mapping information
US5884202A (en) 1995-07-20 1999-03-16 Hewlett-Packard Company Modular wireless diagnostic test and information system
US6006021A (en) 1996-07-01 1999-12-21 Sun Microsystems, Inc. Device for mapping dwellings and other structures in 3D
US6292108B1 (en) 1997-09-04 2001-09-18 The Board Of Trustees Of The Leland Standford Junior University Modular, wireless damage monitoring system for structures
US5933479A (en) 1998-10-22 1999-08-03 Toyoda Machinery Usa Corp. Remote service system
US20020181405A1 (en) 2000-04-10 2002-12-05 I/O Controls Corporation System for providing remote access to diagnostic information over a wide area network
US6853958B1 (en) 2000-06-30 2005-02-08 Integrex System and method for collecting and disseminating household information and for coordinating repair and maintenance services
US20030195008A1 (en) 2000-07-14 2003-10-16 Norman Mohi Locating system and method
US20020095269A1 (en) 2001-01-17 2002-07-18 Francesco Natalini System for monitoring and servicing appliances
US7057557B2 (en) 2001-04-20 2006-06-06 Lg Electronics Inc. Apparatus and method for estimating position of mobile communication terminal
US20030135324A1 (en) 2001-11-20 2003-07-17 Nassir Navab System and method for tracking, locating, and guiding within buildings
US20030110001A1 (en) 2001-12-12 2003-06-12 Chassin David P. Rooftop package unit diagnostician
US20030163440A1 (en) 2002-02-22 2003-08-28 First Data Corporation Maintenance request systems and methods
US20050208951A1 (en) 2002-05-31 2005-09-22 Armando Annunziato Method for locating mobile terminals, system and components therefor
US20040002786A1 (en) 2002-06-27 2004-01-01 Takashi Sasaki Method of predicting dryer steam pressure in paper machine and apparatus for the method
US20060028345A1 (en) 2002-10-09 2006-02-09 Peng Lee Termite acoustic detection
US20040122628A1 (en) 2002-12-03 2004-06-24 Laurie Michael Stuart Method and device for generating two-dimensional floor plans
US20040119662A1 (en) 2002-12-19 2004-06-24 Accenture Global Services Gmbh Arbitrary object tracking in augmented reality applications
US20050165576A1 (en) 2004-01-26 2005-07-28 Jesmonth Richard E. System and method for generating three-dimensional density-based defect map
US20050275525A1 (en) 2004-03-25 2005-12-15 Siemens Building Technologies, Inc. Method and apparatus for graphical display of a condition in a building system with a mobile display unit
US20060084436A1 (en) 2004-06-29 2006-04-20 Michael Green Methods and apparatus for inter-BSC soft handoff
US20070266395A1 (en) 2004-09-27 2007-11-15 Morris Lee Methods and apparatus for using location information to manage spillover in an audience monitoring system
US20060084463A1 (en) 2004-10-04 2006-04-20 Curitel Communications, Inc. Mobile communication terminal having emergency call function and method of processing emergency call request in the mobile communication terminal
US20060164382A1 (en) 2005-01-25 2006-07-27 Technology Licensing Company, Inc. Image manipulation in response to a movement of a display
US20130283529A1 (en) 2005-03-29 2013-10-31 Stryker Corporation Patient support apparatus communication systems
US20070004449A1 (en) 2005-06-29 2007-01-04 Sham John C Mobile communication device with environmental sensors
US20090097710A1 (en) 2006-05-22 2009-04-16 Rafael Advanced Defense Systems Ltd. Methods and system for communication and displaying points-of-interest
US20080103835A1 (en) 2006-10-31 2008-05-01 Caterpillar Inc. Systems and methods for providing road insurance
US20080180246A1 (en) 2007-01-31 2008-07-31 Symbol Technologies, Inc. Methods and apparatus for opportunistic locationing of rf tags using location triggers
US20100107635A1 (en) 2007-03-02 2010-05-06 Kenji Tsubone Thermal storage device
US20080246734A1 (en) 2007-04-04 2008-10-09 The Hong Kong University Of Science And Technology Body movement based usage of mobile device
WO2008124713A2 (en) 2007-04-09 2008-10-16 Sam Stathis A system and method capable of navigating and/or mapping any multi-dimensional space
US20080319654A1 (en) 2007-06-22 2008-12-25 Sony Corporation System and method for effectively implementing an electronic navigation device
US20100296075A1 (en) 2007-10-26 2010-11-25 Leica Geosystems Ag Distance-measuring method for a device projecting a reference line, and such a device
US20090189810A1 (en) 2008-01-24 2009-07-30 Broadcom Corporation Weighted aiding for positioning systems
US20090216438A1 (en) 2008-02-21 2009-08-27 Microsoft Corporation Facility map framework
US20100271263A1 (en) 2008-03-31 2010-10-28 Mehran Moshfeghi Method and System for Determining the Position of a Mobile Station
US20090293051A1 (en) 2008-05-22 2009-11-26 Fortinet, Inc., A Delaware Corporation Monitoring and dynamic tuning of target system performance
US20110182202A1 (en) 2008-09-26 2011-07-28 Huawei Technologies Co., Ltd. Wireless service provision
US20100103036A1 (en) 2008-10-28 2010-04-29 Audiovox Corporation Portable transceiver with vehicle security control and locate features
US20130223261A1 (en) 2008-11-21 2013-08-29 Qualcomm Incorporated Processing time determination for wireless position determination
US8862415B1 (en) 2009-01-29 2014-10-14 Comverge Inc. Method and system for calculating energy metrics of a building and one or more zones within the building
US20100309044A1 (en) 2009-06-05 2010-12-09 Qualcomm Incorporated On Demand Positioning
US20120188847A1 (en) 2009-07-31 2012-07-26 Nec Corporation Position detection apparatus, position detection method, mobile, and receiver
WO2011019810A1 (en) 2009-08-11 2011-02-17 Certusview Technologies, Llc Locating equipment equipped with a mobile/portable device
US20110047516A1 (en) 2009-08-18 2011-02-24 Honeywell Asca, Inc. Rapid manipulation of flowsheet configurations
US8996156B2 (en) 2009-08-20 2015-03-31 Siemens Aktiengesellschaft Automated repair method and system
US20110068906A1 (en) 2009-09-21 2011-03-24 Checkpoint Systems, Inc. Systems, methods, and apparatuses for managing configurable monitoring devices
US20110075835A1 (en) 2009-09-30 2011-03-31 Apple Inc. Self adapting haptic device
US20150154803A1 (en) 2009-10-12 2015-06-04 Metaio Gmbh Method for representing virtual information in a view of a real environment
US20110115816A1 (en) 2009-11-16 2011-05-19 Alliance For Sustainable Energy, Llc. Augmented reality building operations tool
US20110153101A1 (en) 2009-12-22 2011-06-23 General Electric Company Household energy management system and method for one or more appliances
CN102147597A (en) 2010-02-10 2011-08-10 广州大学 Health monitoring system for structures of great building and bridge
US20110199479A1 (en) * 2010-02-12 2011-08-18 Apple Inc. Augmented reality maps
US8818758B1 (en) 2010-03-01 2014-08-26 Wegowise, Inc. Methods and apparatus to track, visualize and understand energy and utilities usage
US20130010103A1 (en) 2010-03-14 2013-01-10 Ns Solutions Corporation Information processing system, information processing method and program, information processing apparatus, vacant space guidance system, vacant space guidance method and program, image display system, image display method and program
US20110251787A1 (en) 2010-04-07 2011-10-13 Qualcomm Incorporated Navigational Coordinate Systems in Conjunction with Transports and/or Mobile Devices
US9064219B2 (en) 2010-05-05 2015-06-23 J. E. Dunn Construction Group, Inc. System and method for managing facility content and equipment information
US20160335731A1 (en) 2010-05-05 2016-11-17 Site 10.01, Inc. System and method for monitoring and managing information
US7994981B1 (en) 2010-09-07 2011-08-09 etherwhere Corporation System framework for mobile device location
US20130297555A1 (en) 2010-09-14 2013-11-07 Nest Labs, Inc. Occupancy pattern detection, estimation and prediction
US20120214507A1 (en) 2010-10-08 2012-08-23 HJ Laboratories, LLC Apparatus and method for providing indoor location, position, or tracking of a mobile computer using building information
US20120087212A1 (en) 2010-10-08 2012-04-12 Harry Vartanian Apparatus and method for providing indoor location or position determination of object devices using building information and/or powerlines
US9008979B2 (en) 2010-10-14 2015-04-14 Azbil Corporation Total energy quantity controlling device and method
US20120127306A1 (en) 2010-11-19 2012-05-24 Honeywell International Inc. Security video detection of personal distress and gesture commands
US20120127976A1 (en) 2010-11-23 2012-05-24 Symbol Technologies, Inc. Radio frequency identification system and related operating methods
US20120173456A1 (en) 2010-11-24 2012-07-05 Hirl Joseph P Decision support system for the management of energy use, contracting and capital investments for facilities
US20120204646A1 (en) 2011-02-10 2012-08-16 National Taiwan University Of Science And Technology Method for analyzing structure safety
US8280404B1 (en) * 2011-03-31 2012-10-02 Google Inc. Directional feedback
US20120296610A1 (en) 2011-05-17 2012-11-22 Ebenezer Hailemariam Occupant centric capture and visualization of building performance data
US8843350B2 (en) 2011-06-03 2014-09-23 Walter P. Moore and Associates, Inc. Facilities management system
US20130169681A1 (en) 2011-06-29 2013-07-04 Honeywell International Inc. Systems and methods for presenting building information
US9342928B2 (en) 2011-06-29 2016-05-17 Honeywell International Inc. Systems and methods for presenting building information
EP2726817B1 (en) 2011-06-30 2018-09-12 Qualcomm Incorporated Navigation in buildings with rectangular floor plan
US20130201311A1 (en) 2011-08-01 2013-08-08 Olympus Medical Systems Corp. Apparatus for estimating shape of insertion portion
US20160084936A1 (en) 2011-08-18 2016-03-24 Rivada Research, Llc. Method and System for Providing Enhanced Location Based Service (ELBS) Trilateration using a Single Device
US20130073059A1 (en) 2011-09-20 2013-03-21 Joseph Mark Brian User interface for determining resource consumption
US20130084886A1 (en) 2011-10-03 2013-04-04 Cellular Specialties, Inc. Pilot beacon system for indoor positioning
US20130120630A1 (en) 2011-11-10 2013-05-16 Samsung Electronics Co., Ltd. System and method for controlling temperature in mobile device
US20140368373A1 (en) 2011-12-20 2014-12-18 Sadar 3D, Inc. Scanners, targets, and methods for surveying
GB2498177A (en) 2011-12-21 2013-07-10 Max Christian Apparatus for determining a floor plan of a building
US20130169801A1 (en) 2011-12-28 2013-07-04 Pelco, Inc. Visual Command Processing
US20150168154A1 (en) 2012-01-03 2015-06-18 Google Inc. Systems, computer-implemented methods, and computer-readable media to provide building information to on-site mobile-devices
US20130197685A1 (en) 2012-01-27 2013-08-01 Fujitsu Limited Medium storing diagram generation program, diagram generation method, and diagram generation apparatus
US9952569B2 (en) 2012-04-19 2018-04-24 Schneider Electric Buildings Llc Association of a portable sensor device in a building management system
US8965741B2 (en) 2012-04-24 2015-02-24 Microsoft Corporation Context aware surface scanning and reconstruction
US20130288719A1 (en) 2012-04-27 2013-10-31 Oracle International Corporation Augmented reality for maintenance management, asset management, or real estate management
US20130345975A1 (en) 2012-06-05 2013-12-26 Apple Inc. Navigation application with adaptive display of graphical directional indicators
US20150142179A1 (en) 2012-06-15 2015-05-21 Mitsubishi Electric Corporation Air-conditioning management device, air-conditioning management method, and program
US10054914B2 (en) 2012-07-11 2018-08-21 Abb Research Ltd Presenting process data of a process control object on a mobile terminal
US20150177718A1 (en) 2012-07-11 2015-06-25 Abb Research Ltd Presenting process data of a process control object on a mobile terminal
US20160077532A1 (en) 2012-07-27 2016-03-17 Assa Abloy Ab Setback controls based on out-of-room presence information
US20150207316A1 (en) 2012-08-16 2015-07-23 Robert Bosch Gmbh Dc building system with energy storage and control system
US20140058572A1 (en) 2012-08-27 2014-02-27 Gridium, Inc. Systems and methods for energy consumption and energy demand management
US20160063671A1 (en) 2012-08-30 2016-03-03 Nokia Corporation A method and apparatus for updating a field of view in a user interface
US20150121222A1 (en) 2012-09-06 2015-04-30 Alberto Daniel Lacaze Method and System for Visualization Enhancement for Situational Awareness
US20150206348A1 (en) 2012-09-07 2015-07-23 Hitachi Maxell, Ltd. Reception device
US20140084909A1 (en) 2012-09-25 2014-03-27 Stmicroelectronics S.R.L. Block made of a building material
US9934335B2 (en) 2012-10-08 2018-04-03 Hexagon Technology Center Gmbh Method and system for virtual assembly of a structure
US20140107828A1 (en) 2012-10-15 2014-04-17 International Business Machines Corporation Method and System for Wafer Quality Predictive Modeling based on Multi-Source Information with Heterogeneous Relatedness
US20140129599A1 (en) 2012-11-07 2014-05-08 Intertrust Technologies Corporation Personalized data management systems and methods
US20160004805A1 (en) 2012-11-16 2016-01-07 Johnson Controls Technology Company Systems and methods for implementing energy conservation measures in a building
US20140146038A1 (en) 2012-11-28 2014-05-29 International Business Machines Corporation Augmented display of internal system components
US20140156455A1 (en) 2012-11-30 2014-06-05 Polar Technology, LLC Refrigerant Exchange, Recycling and Banking System and Computer Program Product
US20140188394A1 (en) 2012-12-27 2014-07-03 International Business Machines Corporation Determining seismic response characteristics of structures
US9529072B2 (en) 2013-01-04 2016-12-27 Ortovox Sportartikel Gmbh Search system and method for searching for a buried person
US20140244160A1 (en) 2013-01-04 2014-08-28 MoneyDesktop, Inc. a Delaware Corporation Augmented reality financial institution branch locator
US20140210856A1 (en) 2013-01-30 2014-07-31 F3 & Associates, Inc. Coordinate Geometry Augmented Reality Process for Internal Elements Concealed Behind an External Element
US20150356789A1 (en) 2013-02-21 2015-12-10 Fujitsu Limited Display device and display method
US20140278065A1 (en) 2013-03-14 2014-09-18 Robert Bosch Gmbh System and Method for Distortion Correction in Three-Dimensional Environment Visualization
US20160284075A1 (en) 2013-03-14 2016-09-29 Essess, Inc. Methods, apparatus, and systems for structural analysis using thermal imaging
US20140266755A1 (en) 2013-03-15 2014-09-18 Emerson Electric Co. Hvac system remote monitoring and diagnosis
US20140277594A1 (en) 2013-03-15 2014-09-18 Fisher-Rosemount Systems, Inc. Method and apparatus for seamless state transfer between user interface devices in a mobile control room
US20160132046A1 (en) 2013-03-15 2016-05-12 Fisher-Rosemount Systems, Inc. Method and apparatus for controlling a process plant with wearable mobile control devices
US20140267420A1 (en) 2013-03-15 2014-09-18 Magic Leap, Inc. Display system and method
US20140274151A1 (en) 2013-03-15 2014-09-18 Nextnav, Llc Systems and Methods for Using Three-Dimensional Location Information to improve Location Services
US9037124B1 (en) 2013-03-27 2015-05-19 Open Invention Network, Llc Wireless device application interaction via external control detection
WO2014204753A1 (en) 2013-06-21 2014-12-24 Qualcomm Incorporated Determination of positioning information of a mobile device using modulated light signals
US20150005903A1 (en) 2013-06-28 2015-01-01 Fisher-Rosemount Systems, Inc. Non-Intrusive Data Analytics in a Process Control System
KR101354688B1 (en) 2013-07-05 2014-01-27 한국건설기술연구원 System and method for supervision of construction site
US20150137967A1 (en) 2013-07-15 2015-05-21 Oneevent Technologies, Inc. Owner controlled evacuation system
US20160216879A1 (en) 2013-09-24 2016-07-28 Lg Electronics Inc. Mobile terminal and method for controlling same
US20150094081A1 (en) 2013-09-30 2015-04-02 Qualcomm Incorporated Determining coordinates of access points in an indoor position location system
US20150094865A1 (en) 2013-10-01 2015-04-02 Yonghyun Choi Heating, ventilation, and/or air conditioning controller
US20180035263A1 (en) 2013-10-08 2018-02-01 Gozio, Inc. Use of rf-based fingerprinting for indoor positioning by mobile technology platforms
US20150116132A1 (en) 2013-10-31 2015-04-30 Delta Systems, Inc. Smart monitoring system and method of operation
US20160202678A1 (en) 2013-11-11 2016-07-14 Osram Sylvania Inc. Human presence detection commissioning techniques
US20170006487A1 (en) 2013-11-27 2017-01-05 At&T Intellectual Property I, L.P. Direct interaction between a user and a communication network
US20150156423A1 (en) 2013-11-29 2015-06-04 Axis Ab System for following an object marked by a tag device with a camera
US20160048200A1 (en) 2013-12-06 2016-02-18 Lg Electronics Inc. Smart watch and control method thereof
US20150178633A1 (en) 2013-12-20 2015-06-25 Johnson Controls Technology Company Systems and methods for determining the uncertainty in parameters of an energy use model
US20160343093A1 (en) 2014-01-24 2016-11-24 Telvent Usa Llc Utility resource asset management system
US20150227123A1 (en) 2014-02-11 2015-08-13 Honeywell International Inc. Mobile extension for industrial operator consoles
US20160360429A1 (en) 2014-02-19 2016-12-08 Huawei Technologies Co., Ltd. Method and Device for Wirelessly Accessing Service
US20160026253A1 (en) 2014-03-11 2016-01-28 Magic Leap, Inc. Methods and systems for creating virtual and augmented reality
US20150262421A1 (en) 2014-03-14 2015-09-17 Matterport, Inc. Processing and/or transmitting 3d data
US20150287318A1 (en) 2014-04-04 2015-10-08 Rockwell Automation Technologies, Inc. Industrial-enabled mobile device
US20150294506A1 (en) 2014-04-15 2015-10-15 Huntington Ingalls, Inc. System and Method for Augmented Reality Display of Dynamic Environment Information
US10355351B2 (en) 2014-04-21 2019-07-16 Maxtena, Inc. Antenna array pointing direction estimation and control
US20150347854A1 (en) 2014-04-25 2015-12-03 Huntington Ingalls Incorporated System and Method for Using Augmented Reality Display in Surface Treatment Procedures
US20150327010A1 (en) 2014-05-07 2015-11-12 Johnson Controls Technology Company Systems and methods for detecting and using equipment location in a building management system
US10149141B1 (en) 2014-05-13 2018-12-04 Senseware, Inc. System, method and apparatus for building operations management
US20160026729A1 (en) 2014-05-30 2016-01-28 Reylabs Inc Systems and methods involving mobile indoor energy efficiency exploration, monitoring and/or display aspects
US20150356786A1 (en) 2014-06-09 2015-12-10 Huntington Ingalls, Inc. System and Method for Augmented Reality Display of Electrical System Information
US20170131426A1 (en) 2014-06-25 2017-05-11 Robert Bosch Gmbh Location Device and Method for Operating a Location Device
US9892559B2 (en) 2014-06-25 2018-02-13 Digital Electronics Corporation Portable terminal device, and portable control device
US20160019721A1 (en) 2014-07-16 2016-01-21 Huntington Ingalls Incorporated System and Method for Augmented Reality Display of Hoisting and Rigging Information
US20170234962A1 (en) 2014-08-12 2017-08-17 Philips Lighting Holding B.V. Method and apparatus for locating of a mobile device
US9703517B2 (en) 2014-09-02 2017-07-11 Adobe Systems Incorporated External device screen targeting
US20160066068A1 (en) 2014-09-03 2016-03-03 Oberon, Inc. Environmental Monitor Device with Database
US20160069675A1 (en) 2014-09-04 2016-03-10 Hitachi Construction Machinery Co., Ltd. Position calculating system and haulage vehicle
US20160071196A1 (en) 2014-09-08 2016-03-10 Leeo, Inc. Systems and methods for transferring data and revenue
US20170055126A1 (en) 2014-09-24 2017-02-23 James Thomas O'Keeffe System and method for user profile enabled smart building control
US20160091217A1 (en) 2014-09-29 2016-03-31 Koninklijke Philips N.V. Systems and methods for managing environmental conditions
US20160095188A1 (en) 2014-09-29 2016-03-31 Koninklijke Philips N.V. Systems and methods for managing environmental conditions
US20170235290A1 (en) 2014-10-14 2017-08-17 Yotta Green Ltd Smart home system and method
US20160135006A1 (en) * 2014-11-07 2016-05-12 Uber Technologies, Inc. Dynamically controlling the presentation of a graphic feature based on location information
US20160183057A1 (en) 2014-12-18 2016-06-23 Intel Corporation Method and system for hybrid location detection
US20160178383A1 (en) 2014-12-19 2016-06-23 Here Global B.V. User Interface for Displaying Navigation Information in a Small Display
WO2016130571A1 (en) 2015-02-09 2016-08-18 Carnegie Mellon University Indoor identification of individuals through footstep induced structural vibration
US9772396B2 (en) 2015-02-09 2017-09-26 Ricoh Company, Ltd. Relative orientation angle calculation method and device as well as relative positioning method
US20180031618A1 (en) 2015-02-11 2018-02-01 Yehudah FRIEDLANDER System for analyzing electricity consumption
US20160238692A1 (en) 2015-02-13 2016-08-18 Position Imaging, Inc. Accurate geographic tracking of mobile devices
US10042341B1 (en) 2015-02-19 2018-08-07 State Farm Mutual Automobile Insurance Company Systems and methods for monitoring building health
US20160258760A1 (en) 2015-03-04 2016-09-08 Lenovo (Singapore) Pte. Ltd. Providing directions to a location in a facility
US20180102858A1 (en) 2015-03-24 2018-04-12 Carrier Corporation System and method for determining rf sensor performance relative to a floor plan
US20180101803A1 (en) 2015-03-24 2018-04-12 Carrier Corporation Integrated system for sales, installation, and maintenance of building systems
US20160285416A1 (en) 2015-03-24 2016-09-29 Carrier Corporation System and method for energy harvesting system planning and performance
US20180075168A1 (en) 2015-03-24 2018-03-15 Carrier Corporation System and method for capturing and analyzing multidimensional building information
US20160284128A1 (en) 2015-03-27 2016-09-29 Rockwell Automation Technologies, Inc. Systems and methods for presenting an augmented reality
US20180084623A1 (en) 2015-04-30 2018-03-22 Zumtobel Lighting Gmbh Method and System for Improving Lighting Control and Method and System for Controlling a Lighting Device
US20160323149A1 (en) 2015-04-30 2016-11-03 International Business Machines Corporation Automatic Measurement Expression Update
US20170023659A1 (en) 2015-05-08 2017-01-26 5D Robotics, Inc. Adaptive positioning system
US20160343243A1 (en) 2015-05-21 2016-11-24 Google Inc. Method and system for consolidating events across sensors
US20160345137A1 (en) 2015-05-21 2016-11-24 Toshiba America Business Solutions, Inc. Indoor navigation systems and methods
US9668106B2 (en) 2015-05-28 2017-05-30 Hon Hai Precision Industry Co., Ltd. Indoor positioning system and method for indoor position
WO2016192916A1 (en) 2015-06-02 2016-12-08 Qualcomm Technologies International, Ltd. Intuitive way to point, access and control appliances & other objects in building interiors
US20160379083A1 (en) 2015-06-24 2016-12-29 Microsoft Technology Licensing, Llc Real-time, model-based object detection and pose estimation
US20180242907A1 (en) 2015-08-10 2018-08-30 Koninklijke Philips N.V. Determining metabolic parameters using wearables
US20170079001A1 (en) 2015-09-15 2017-03-16 Google Inc. Radio Beacon for Direction Detection
US20170115022A1 (en) 2015-10-21 2017-04-27 Setra Systems, Inc. Room condition monitoring system
US20190145648A1 (en) 2015-10-28 2019-05-16 Johnson Controls Technology Company Thermostat with halo light system and emergency directions
US20170169683A1 (en) 2015-12-09 2017-06-15 Fire & Risk Alliance, LLC System and methods for detecting, confirming, classifying, and monitoring a fire
US9792020B1 (en) 2015-12-30 2017-10-17 Palantir Technologies Inc. Systems for collecting, aggregating, and storing data, generating interactive user interfaces for analyzing data, and generating alerts based upon collected data
US20170200312A1 (en) 2016-01-11 2017-07-13 Jeff Smith Updating mixed reality thumbnails
US20170212668A1 (en) 2016-01-22 2017-07-27 Johnson Controls Technology Company Building energy management system with ad hoc dashboard
US20170237892A1 (en) 2016-02-16 2017-08-17 Fujitsu Limited Non-transitory computer-readable storage medium, control method, and computer
US10444324B2 (en) 2016-02-22 2019-10-15 Nida Tech Sweden Ab Single node location system and method
WO2017149526A2 (en) 2016-03-04 2017-09-08 May Patents Ltd. A method and apparatus for cooperative usage of multiple distance meters
US20170289344A1 (en) 2016-03-31 2017-10-05 International Business Machines Corporation Contextually Configuring Consistent Proximity Beacon Identifiers
US20170286568A1 (en) 2016-03-31 2017-10-05 M.C. Dean Inc. System and method for testing the validity of bim-designed electrical wiring pathways
US10222301B2 (en) 2016-05-04 2019-03-05 Embraer S.A. Structural health monitoring system with the identification of the damage through a device based in augmented reality technology
US20170363504A1 (en) 2016-06-21 2017-12-21 Thomas Arthur Winant System and method for determining the risk of failure of a structure
WO2017223008A1 (en) 2016-06-21 2017-12-28 Osram Sylvania Inc. Wireless beacon-enabled luminaire identification system and method for determining the position of a portable device
US20180018826A1 (en) 2016-07-15 2018-01-18 Beckhoff Automation Gmbh Method for controlling an object
US20180048693A1 (en) 2016-08-09 2018-02-15 The Joan and Irwin Jacobs Technion-Cornell Institute Techniques for secure data management
US20180159904A1 (en) 2016-10-06 2018-06-07 Abb Technology Oy Remote live support tool
US20180108079A1 (en) 2016-10-13 2018-04-19 River Ventures Spa Augmented Reality E-Commerce Platform
US20190025905A1 (en) 2016-10-18 2019-01-24 Raytheon Company Avionics maintenance training
US20200057825A1 (en) 2016-10-24 2020-02-20 Wemarathon System for improving the design, construction and operation of a structure
US20180113506A1 (en) 2016-10-25 2018-04-26 Oculus Vr, Llc Position tracking system that exploits arbitrary configurations to determine loop closure
US20180131907A1 (en) 2016-11-08 2018-05-10 Rockwell Automation Technologies, Inc. Virtual reality and augmented reality for industrial automation
US20180130260A1 (en) 2016-11-08 2018-05-10 Rockwell Automation Technologies, Inc. Virtual reality and augmented reality for industrial automation
US20180206096A1 (en) 2017-01-18 2018-07-19 Johnson Controls Technology Company Systems and methods for enhancing building management system interaction and visualization
US10278016B2 (en) 2017-02-03 2019-04-30 Qualcomm Incorporated Systems and methods for position estimation using proximity devices
US10872179B2 (en) 2017-02-22 2020-12-22 Middle Chart, LLC Method and apparatus for automated site augmentation
US20190057169A1 (en) 2017-02-22 2019-02-21 Middle Chart, LLC Building Vital Conditions Monitoring
US11106837B2 (en) * 2017-02-22 2021-08-31 Middle Chart, LLC Method and apparatus for enhanced position and orientation based information display
US20190392088A1 (en) 2017-02-22 2019-12-26 Middle Chart, LLC Smart construction with automated detection of adverse structure conditions and remediation
US20180285482A1 (en) 2017-03-28 2018-10-04 Faro Technologies, Inc. System and method of scanning an environment and generating two dimensional images of the environment
US20180295526A1 (en) 2017-04-07 2018-10-11 Wireless Applications Corp. Radio signal path design tool with graphical display to facilitate selection of alternative radio antenna sites
US20180328753A1 (en) 2017-05-09 2018-11-15 Raven Telemetry Inc. Local location mapping method and system
US20180357823A1 (en) 2017-06-08 2018-12-13 Honeywell International Inc. Apparatus and method for visual-assisted training, collaboration, and monitoring in augmented/virtual reality in industrial automation systems and other systems
US20180374269A1 (en) 2017-06-21 2018-12-27 INVRSE Reality Limited Augmented reality and virtual reality mobile device user interface automatic calibration
US20190096232A1 (en) 2017-06-27 2019-03-28 Oneevent Technologies, Inc. Augmented reality of a building
US10130285B1 (en) 2017-09-24 2018-11-20 Wisilica Inc Wearable apparatus for patient tracking
US20190156576A1 (en) 2017-11-20 2019-05-23 Bernard Ndolo Method and system for curating, accessing, and displaying a plurality of data records pertaining to premises, and a plurality of devices installed in the premises
US20190294834A1 (en) 2017-12-07 2019-09-26 Kacchip, LLC Virtual Mapping of an Indoor Space Using Indoor Position and Vector Tracking
US20190216333A1 (en) 2018-01-12 2019-07-18 Futurewei Technologies, Inc. Thermal face image use for health estimation
US20190228370A1 (en) 2018-01-24 2019-07-25 Andersen Corporation Project management system with client interaction
US20190268062A1 (en) 2018-02-23 2019-08-29 Precision Integrated Photonics, Inc. Disadvantaged node discovery
US20190281573A1 (en) 2018-03-06 2019-09-12 Motorola Mobility Llc Location Correction Apparatus and Method in a Real-Time Locating System
US20190355177A1 (en) 2018-05-15 2019-11-21 Honeywell International Inc. Building system maintenance using mixed reality
US20200072538A1 (en) 2018-05-30 2020-03-05 Lineage Logistics, LLC Thermal control system
US20190392640A1 (en) 2018-06-26 2019-12-26 Lenovo (Singapore) Pte. Ltd. Presentation of augmented reality images at display locations that do not obstruct user's view
US20200108926A1 (en) 2018-10-03 2020-04-09 Sarcos Corp. Aerial Vehicles Having Countermeasures Deployed From a Platform for Neutralizing Target Aerial Vehicles
US20200151450A1 (en) 2018-11-13 2020-05-14 Kabushiki Kaisha Toshiba Maintenance work support system and maintenance work support method
US10824774B2 (en) 2019-01-17 2020-11-03 Middle Chart, LLC Methods and apparatus for healthcare facility optimization
US20200286300A1 (en) 2019-03-06 2020-09-10 Quanta Computer Inc. Electronic device and method for adjusting size of three-dimensional object in augmented reality
US10825247B1 (en) 2019-11-12 2020-11-03 Zillow Group, Inc. Presenting integrated building information using three-dimensional building models

Non-Patent Citations (30)

* Cited by examiner, † Cited by third party
Title
"Wireless Indoor Localization Systems and Techniques: Survey and Comparative Study" Indonesian Journal of Electrical Engineering and Computer Science, vol. 3, No. 2, Aug. 2016, pp. 392-409 (Year: 2016).
Amekudzi, Adjo A., Rebecca Shelton, and Tim R. Bricker. "Infrastructure Rating Tool: Using Decision Support Tools to EnhanceASCE Infrastructure Report Card Process." Leadership and Management in Engineering 13.2 (2013): 76-82. (Year: 2013).
ASCE 2014 Report Card of Montana's Infrastructure, accessed at https://www.infrastructurereportcard.org/wp-contentluploads/2014/11/2014-Report-Card-for-Montanas-Infrastructure.pdf (Year: 2014).
Aukstakalnis, Steve. Practical augmented reality: A guide to the technologies, applications, and human factors for AR and VR.Addison-Wesley Professional, 2016. (Year: 2016).
Azeez et al., "Wireless Indoor Localization Systems and Techniques: Survey and Comparative Study" Indonesian Journal of Electrical Engineering and Computer Science, vol. 3, No. 2, Aug. 2016, pp. 392-409 (Year: 2016).
Brainergiser, "Large holographic projector—a real use case". Sep. 19, 2015, https://www.youtube.com/watch?v=JwnS-EKTW2A&feature=youtu.be.
Carmelo Scuro et al., IoT for Structural Health Monitoring, Dec. 2018, IEEE Instrumentation & Measurement Magazine, pp. 4-14.
Dong et ai, "Evaluation of the Reliability of RSSI for Indoor localization", 2012, pp. 6. downloaded from the internet https://ieeexplore.ieee.org/documentl6402492 (Year: 2012).
Edwards, Luke, "Holograms are finally here: Plasma lasers used to create images in mid-air." Pocket-Lint.com, Nov. 5, 2014, https://www.pocket-lint.com/gadgets/news/131622-holograms-are-finally-here-plasma-lasers-used-to-create-images-in-mid-air.
Gifford, Matthew, "Indoor Positioning with Ultrasonic/Ultrasound", Oct. 19, 2018, 7 pages, https://www.leverege.com/blogpost/ultrasonic-indoor-positioning.
Hexamite, "HX19V2 RFID Ultrasonic Positioning System", 1999, https://www.hexamite.com/hx19.htm.
Hu. Wi-Fi Based Indoor Positioning System Using Smartphones. Nov. 2013. [retrieved on Apr. 23, 2018]. Retrieved [from the Internet: (91 pages total).
International Search Report and Written Opinion dated Feb. 10, 2020 issued in connection with corresponding [International Application No. PCT/US2018/019154 (9 pages total).
International Search Report and Written Opinion dated Feb. 6, 2020 issued in connection with corresponding [International Application No. PCT/US2018/019185 (9 pages total).
International Search Report and Written Opinion dated May 14, 2018 issued in connection with corresponding International Application No. PCT/US2018/019154 (10 pages total).
International Search Report and Written Opinion dated May 7, 2018 issued in connection with corresponding [International Application No. PCT/US2018/019185 (7 pages total).
Khemapech et al., Bridge Structural Monitoring and Warning System Aplication in Thailand—Experiences Learned, 2017, TRON Forum ISBN 978-4-89362-330-0, pp. 1-8.
Kyle, "Property Management" Dearborn Real Estate, 2000—Business & Economics, pp. 40-41.
Liu et al.,"A Hybrid Smartphone Indoor Positioning Solution for Mobile LBS" Sensors 2012, issue 12, pp. 17208-17233 (Year: 2012).
Liu, "Survey of Wireless Based Indoor Localization Technologies" accessed at http://www.cse.wustl.edu/-jain/cse574-14/ftp/indoor/index.html, May 5, 2014,17 pg printout (Year: 2014).
Mordue, Stefan, Paul Swaddle, and David Philp. Building information modeling for dummies. John Wiley & Sons, 2015. (Year: 2015).
Qi, J.; Liu, G.-P. A Robust High-Accuracy Ultrasound Indoor Positioning System Based on a Wireless Sensor Network. Sensors 2017, 17, 2554.
Suermann, Patrick C. Evaluating the impact of building information modeling (BIM) on construction. Florida Univ GainesvilleGraduate School, 2009. (Year: 2009).
Thomson, C. P. H. From Point Cloud to Building Information Model: Capturing and Processing Survey Data Towards Automation forHigh Quality 3D Models to Aid a BIM Process. Diss. UCL (University College London), 2016. (Year: 2016).
Wang et al. Performance Evaluation of Automatically Generated BIM from Laser Scanner Data for Sustainability Analyses. 2015. [retrieved on Apr. 23, 2018]. Retrieved from the Internet: . (8 pages total).
Wang, Siqi, Jinsheng Du, and Jianyong Song. "A Framework of BIM-Based Bridge Health Monitoring System." 2016 InternationalConference on Civil, Transportation and Environment. Atlantis Press, 2016. (Year: 2016).
Wikipedia article "Building Information Modeling", archive data Jan. 15, 2016 on the Wayback machine (Year: 2016).
Yang et ai, "Wi Fi-Based Indoor Positioning", 2015, pp. 150-157 downloaded from the internet I EEE.com databases. (Year: 2015).
Zhihua Wang et al., A Review of Wearable Technologies for Elderly Care that Can Accurately Track Indoor Position, Recognize Physical Activities and Monitor Vital Signs in Real Time, Feb. 10, 2017, Sensors (Basel), 17(2), 341, p. 1-36; doi: 10.3390/s 17020341 (Year: 2017).
Zou et al., "Smartscanner: Know More in Walls with Your Smartphone!" IEEE Transactions on Mobile Computing, vol. 15, No. 11, Nov. 2016, pp. 2865-2877 (Year: 2016).

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11946761B2 (en) 2018-06-04 2024-04-02 The Research Foundation For The State University Of New York System and method associated with expedient determination of location of one or more object(s) within a bounded perimeter of 3D space based on mapping and navigation to a precise POI destination using a smart laser pointer device

Also Published As

Publication number Publication date
US20210374300A1 (en) 2021-12-02

Similar Documents

Publication Publication Date Title
US10872179B2 (en) Method and apparatus for automated site augmentation
US11194938B2 (en) Methods and apparatus for persistent location based digital content
US11893317B2 (en) Method and apparatus for associating digital content with wireless transmission nodes in a wireless communication area
US11106837B2 (en) Method and apparatus for enhanced position and orientation based information display
US7592909B2 (en) Location and tracking system using wireless technology
US10984146B2 (en) Tracking safety conditions of an area
KR20180063263A (en) Three-dimensional space detection system, positioning method and system
US12014450B2 (en) Methods and apparatus for secure persistent location based digital content associated with a two-dimensional reference
US11847741B2 (en) System and method of scanning an environment and generating two dimensional images of the environment
US20240152663A1 (en) Acoustic positioning transmitter and receiver system and method
WO2019153855A1 (en) Object information acquisition system capable of 360-degree panoramic orientation and position sensing, and application thereof
US20210271786A1 (en) Method and apparatus for construction and operation of connected infrastructure
US11475177B2 (en) Method and apparatus for improved position and orientation based information display
Singh et al. Ubiquitous hybrid tracking techniques for augmented reality applications
US20220164492A1 (en) Methods and apparatus for two dimensional location based digital content
Sonny et al. A Survey of Application of Machine Learning in Wireless Indoor Positioning Systems
CN114136306B (en) Expandable device and method based on relative positioning of UWB and camera
Kessel et al. Tracking and navigation for goods and people
Di Pietra Seamless Positioning and Navigation in Urban Environment
Wang Hybrid Indoor Positioning System Based on Wi-Fi Fingerprint and Ultrasound
Odengård Positioning systems for scale model cars-A survey of available systems and an application with Qualisys Track Manager
Waqar Motion assisted indoor smartphone positioning in sparse Wi-Fi environments

Legal Events

Date Code Title Description
FEPP Fee payment procedure

Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY

FEPP Fee payment procedure

Free format text: ENTITY STATUS SET TO SMALL (ORIGINAL EVENT CODE: SMAL); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY

STPP Information on status: patent application and granting procedure in general

Free format text: SPECIAL NEW

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

AS Assignment

Owner name: MIDDLE CHART, LLC, FLORIDA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:SANTARONE, MICHAEL S.;WODRICH, MICHAEL A.;DUFF, JASON E.;AND OTHERS;SIGNING DATES FROM 20210813 TO 20220623;REEL/FRAME:060294/0668

STPP Information on status: patent application and granting procedure in general

Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED

STCF Information on status: patent grant

Free format text: PATENTED CASE