WO2014070399A1 - Estimating the gravity vector in a world coordinate system using an accelerometer in a mobile device - Google Patents

Estimating the gravity vector in a world coordinate system using an accelerometer in a mobile device Download PDF

Info

Publication number
WO2014070399A1
WO2014070399A1 PCT/US2013/064172 US2013064172W WO2014070399A1 WO 2014070399 A1 WO2014070399 A1 WO 2014070399A1 US 2013064172 W US2013064172 W US 2013064172W WO 2014070399 A1 WO2014070399 A1 WO 2014070399A1
Authority
WO
WIPO (PCT)
Prior art keywords
mobile device
coordinate system
accelerometer
target plane
measurements
Prior art date
Application number
PCT/US2013/064172
Other languages
French (fr)
Inventor
Arvind RAMANANDAN
Mahesh Ramachandran
Christopher Brunner
Murali Ramaswamy Chari
Original Assignee
Qualcomm Incorporated
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Qualcomm Incorporated filed Critical Qualcomm Incorporated
Priority to KR1020157014145A priority Critical patent/KR20150082374A/en
Priority to CN201380057457.XA priority patent/CN104756039A/en
Priority to EP13780479.5A priority patent/EP2915015A1/en
Priority to JP2015540684A priority patent/JP2016503495A/en
Publication of WO2014070399A1 publication Critical patent/WO2014070399A1/en

Links

Classifications

    • GPHYSICS
    • G01MEASURING; TESTING
    • G01PMEASURING LINEAR OR ANGULAR SPEED, ACCELERATION, DECELERATION, OR SHOCK; INDICATING PRESENCE, ABSENCE, OR DIRECTION, OF MOVEMENT
    • G01P13/00Indicating or recording presence, absence, or direction, of movement
    • G01P13/02Indicating direction only, e.g. by weather vane
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/16Constructional details or arrangements
    • G06F1/1613Constructional details or arrangements for portable computers
    • G06F1/1633Constructional details or arrangements of portable computers not specific to the type of enclosures covered by groups G06F1/1615 - G06F1/1626
    • G06F1/1684Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675
    • G06F1/1694Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01CMEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/10Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01CMEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C25/00Manufacturing, calibrating, cleaning, or repairing instruments or devices referred to in the other groups of this subclass
    • G01C25/005Manufacturing, calibrating, cleaning, or repairing instruments or devices referred to in the other groups of this subclass initial alignment, calibration or starting-up of inertial devices
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0346Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors

Definitions

  • the subject matter disclosed herein relates generally to the use of an accelerometer for measuring gravity vectors.
  • An accelerometer also referred to as a motion sensor measures proper acceleration, which is the acceleration it experiences relative to a freefall (or the inertia). Proper acceleration is associated with the phenomenon of weight experienced by a proof mass that resides in a frame of reference of the accelerometer.
  • An accelerometer measures the weight per unit of the proof mass, a quantity also known as specific force, or g-foree.
  • an accelerometer behaves as a damped mass on a spring. When the accelerometer experiences acceleration, the position of the proof mass is displaced with respect to the frame. The displacement is measured to determine the acceleration.
  • a gyroscope (also referred to as a rotation sensor) measures the angular velocity of a system in the inertia! reference frame. By using the original orientation of the system in the inertia! reference frame as the initial condition and integrating the angular velocity, the system's current orientation can be known.
  • a gyroscope is a spinning rotor which maintains its orientation based on the principle of conservation of angular momentum. This phenomenon can be used for measuring and maintaining orientation in many applications, such as compasses and stabilizers in aircraft and spacecraft.
  • Accel erometers and gyroscopes have been incorporated into a variety of consumer electronics.
  • the integration of accelerometers and gyroscopes allows for more accurate and robust augmented reality (AR) applications, simultaneous localization and mapping (SLAM) applications, computer vision applications, navigation applications, stability control applications, and a wide range of other applications.
  • AR augmented reality
  • SLAM simultaneous localization and mapping
  • the gravity vector in the target coordinate system is known.
  • An example of a target coordinate system is the coordinate system in which an AR target is going to be displayed.
  • this gravity vector is generally not known in practice, and is generally measured in response to a request from an application such as an AR or SLAM application.
  • the accuracy requirement for the measurements is quite high (e.g., less than a degree); otherwise the measurements would be unusable. Higher accuracy would lead to a better performing application or more demanding use cases.
  • Existing accelerometer-assisted techniques for measuring the gravity vector in the target coordmate system typically have low accuracy due to error accumulations in multiple transformation steps where each step may introduce errors.
  • An accelerometer located within a mobile device is used to estimate a gravity vector on a target plane in a world coordinate system.
  • a processor of the mobile device receives a plurality of measurements from the accelerometer. Each of the measurements is taken when the mobile device is held stationary on the target plane and a surface of the mobile device faces and is in contact with a planar portion of the target plane.
  • the processor calculates an average of the measurements, and retrieves a rotational transformation between an accelerometer coordinate system and a device coordinate system from a memory in the mobile device, wherein the device coordmate system is aligned with the surface of the mobile device.
  • the rotational transformation is applied to the averaged measurements to obtain an estimated gravity vector in a world coordmate system defined by the target plane.
  • a mobile device comprises: an accelerometer, a memory to store a rotational transformation between a coordinate system of the accelerometer and a coordinate system of the mobile device, and a processor coupled to the memory and the accelerometer.
  • the processor is configured to: receive a plurality of measurements from the accelerometer, each of the measurements being taken when the mobile device is held stationary on a target plane and a surface of the mobile device faces and is in contact with a planar portion of the target plane; calculate an average of the measurements; retrieve the rotational transformation from the memor '-, wherein the coordinate system of the mobile device is aligned with the surface of the mobile de v ice: and apply the rotational transformation to the average to obtain an estimated gravity vector in a world coordinate system defined by the target plane.
  • a computer program product comprises a computer- readable medium, which comprises code for: receiving a plurality of measurements from an accelerometer, each of the measurements being taken when the mobile device is held stationary on a target plane and a surface of the mobile device faces and is in contact with a planar portion of the target plane; calculating an average of the measurements; retrieving a rotational transformation between a coordinate system of the accelerometer and a coordinate system of the mobile device from a memory in the mobile device, wherein the coordinate system of the mobile device is aligned with the surface of the mobile device; and applying the rotational transformation to the average to obtain an estimated gravity vector in a world coordinate system defined by the target plane,
  • a mobile device comprises: means for receiving a plurality of measurements from an accelerometer, each of the measurements being taken when the mobile device is held stationary on a target plane and a surface of the mobile device faces and is in contact with a planar portion of the target plane; means for calculating an average of the measurements; means for retrieving a rotational transformation between a coordinate system of the accelerometer and a coordinate system of the mobile device from a memory in the mobile device, wherein the coordinate system of the mobile device is aligned with the surface of the mobile device; and means for applying the rotational transformation to the average to obtain an estimated gravity vector in a world coordinate system defined by the target plane.
  • FIG. 1 is a block diagram of a mobile device in which embodiments of the invention may be practiced.
  • FIG. 2 A and FIG. 2B illustrate examples of a side profile of the mobile device of FIG. 1.
  • FIG. 3A, FIG. 3B, FIG. 3C and FIG. 3D illustrate examples of a target plane relative to which the gravity vector is to be estimated.
  • FIG. 4 is a flow diagram illustrating a method for estimating the gravity vector in a world coordinate system using an accelerometer within a mobile device according to one embodiment
  • Embodiments of the invention provide a method for estimating the gravity vector with respect to a target plane using an accelerometer in a mobile device, such as a mobile phone.
  • the target plane is a feature plane on which the AR target is going to be displayed.
  • the target plane can have any orientation; for example, the target plane can be aligned with the horizontal axis, aligned with the vertical axis, or tilted with respect to the horizontal or vertical axis.
  • This target plane defines a world coordinate system for AR applications.
  • the mobile device can serve as a convenient tool for estimating or measuring the gravity vector in the world coordinate system, it is appreciated that the estimation technique described herein is not limited to AR applications; it is applicable to a wide range of applications where the target plane may be a plane relative to which the gravity vector is unknown and needs to be measured.
  • the term "world coordinate system,” “tracking coordinate system” or “target coordinate system” refers to a coordinate system having a 2-D coordinate plane defined by the target plane. That is, the x-y (or x-z or y-z) coordinate plane of the world coordinate system is parallel to the target plane.
  • the term “accelerometer coordinate system” refers to the coordinate system of the accelerometer within the mobile device.
  • the term “device coordinate system” or “surface coordinate system” refers to a coordinate system having a 2-D coordinate plane defined by a surface of the mobile device. In one embodiment, a mobile device may have only one device coordinate system defined by one surface (either front or back surface) of the mobile device.
  • This surface is referred to as a "coordinate surface" of the mobile device.
  • both front and back surfaces of the mobile device can be the coordinate surface. If the front surface of a mobile device is not parallel to the back surface, only one surface (either front or back) of the mobile device that defines the device coordinate system is the coordinate surface.
  • a mobile device may have two device coordinate systems; one defined by the front surface and the other one defined by the back surface. The mobile device may select either the front surface or the back surface as the coordinate surface.
  • the mobile device can be configured to retrieve one of two rotational transformations from the memory for alignment with the front or back surface, where the two rotational transformations include a first transformation between the coordinate system of the accelerometer and the front surface of the mobile device, and a second transformation between the coordinate system of the accelerometer and the back surface of the mobile device.
  • a user places the coordinate surface of the mobile device firmly on the target plane such that the coordinate surface is parallel to the target plane. After the mobile device is firmly placed and motionless, the accelerometer within the mobile device makes one or more measurements.
  • Each measurement is a measured gravity vector in the accelerometer coordinate system
  • a conversion engine in ihe mobile device can convert the gravity measurements from the accelerometer coordinate system to the device coordinate system.
  • the gravity measurements in the device coordinate system are the same as those in the world coordinate system.
  • the gravity measurements can be averaged over a time window to obtain an accurate estimate of the gravity vector relative to the target plane in the world coordinate system.
  • the gravity vector measured in the world coordinate system can be used for AR, SLAM and various other applications.
  • the accelerometer is factory-calibrated such that the coordinate system of the accelerometer aligns with the surface of the mobile device.
  • Other calibration techniques such as user-performed calibration, can also be used.
  • the alignment result is a rotational transformation, which can be stored in the memory of the mobile device.
  • a mobile device can be aligned (calibrated) once and the alignment result can be used in the subsequent measurements.
  • FIG. 1 is a block diagram illustrating a system in which embodiments of the invention may be practiced.
  • the system may be a mobile device 100, which may include a processor 110, a memory 120, an interface 160 and one or more sensors such as an accelerometer 130 and a gyroscope 140.
  • the mobile device 100 may include both the accelerometer 130 and the gyroscope 140; in an alternative embodiment, the mobile device 100 may include only the accelerometer 130.
  • the mobile device 100 may also include a display device, a user interface (e.g., keyboard, touch-screen, etc.), a power device (e.g., a battery), as well as other components typically associated with a mobile communication device.
  • a user interface e.g., keyboard, touch-screen, etc.
  • a power device e.g., a battery
  • the interface 160 may be a wireless transceiver to transmit and receive wireless signals through a wireless link to/from a wireless network, or may be wired interface for direct connection to networks (e.g., the Internet).
  • the mobile device 100 may be a: mobile phone (e.g., cell phone, smart phone, etc.), personal digital assistant, mobile computer, tablet, personal computer, laptop computer, e-reader, or any type of mobile device that has motion sensing and/or rotation sensing capabilities.
  • the processor 1 10 may include a conversion engine 1 15, which may be implemented in hardware, firmware, software, or a combination of any of the above.
  • the processor 1 10 may be a general-purpose processor or a special-purpose processor configured to execute instructions for performing the operations of conversion engine 1 15 that retrieves a stored rotational transformation from the memory 120, where the rotational transformation transforms an acceleration measurement in the accelerometer coordinate system into a corresponding vector in the device coordinate system.
  • the conversion engine 1 15 may apply the rotational transformation to accelerometer measurements to compute the gravity vector in the world coordinate system in a process to be described below.
  • the memory 120 may be coupled to the processor 1 10 to store instructions for execution by the processor 1 10.
  • the memory 12.0 may store a device profile 121, which includes the rotational transformation between the accelerometer coordinate system and the device coordinate system.
  • the memory 120 may store more than one rotational transformation; e.g., one rotational transformation for the front surface and another rotational transformation for the back surface.
  • the device profile 121 of the mobile device may store the rotational transformation with other sensor calibration parameters such as the measuring scale, crosstalk of sensors and the alignment between the sensors and the camera, sensor biases, if any, on the mobile device.
  • circuitry of the mobile device 100 may operate under the control of a program, routine, or the execution of instructions to execute methods or processes in accordance with embodiments of the invention.
  • a program may be implemented in firmware or software (e.g. stored in the memory 120 and/or other locations) and may be implemented by processors, such as the processor 1 10, and/ or other circuitry of the mobile device 100.
  • processors such as the processor 1 10, and/ or other circuitry of the mobile device 100.
  • processor, microprocessor, circuitry, controller, etc. refer to any type of logic or circuitry capable of executing logic, commands, instructions, software, firmware, functionality and the like.
  • FIG, 2.A illustrates a side profile of the mobile device 100 according to one embodiment.
  • the mobile device 100 as viewed in FIG. 2A, has a top surface and a bottom surface, where the bottom surface may be either the front or back side of the mobile device 100.
  • FIG. 2A shows that a device coordinate system 182 having an x-y coordinate plane parallel to the bottom surface.
  • the bottom surface is a coordinate surface 170 for the mobile device 100, which is the surface to which the device coordinate system 182 aligns.
  • This coordinate surface 170 is to be placed on a target plane 180 that defines the world coordinate system.
  • FIG. 2B illustrates a side profile of the mobile device 100 according to another embodiment.
  • the mobile device 100 as viewed in FIG. 2B, has a concaved bottom surface, where the concaved bottom surface may be either the front or back side of the mobile device 100,
  • the convex hull of the concaved bottom surface defines a planar surface 175.
  • FIG. 2B shows that the x-y coordinate plane of the device coordinate system 182 is parallel to the planar surface 175,
  • the planar surface 175 is the coordinate surface for the mobile device 100, which is the surface to which the device coordinate system 182 aligns. This coordinate surface is to be placed on the target plane 180 that defines the world coordinate system,
  • the mobile device 100 can select either its front surface or the back surface as the coordinate surface, the selection is based on which surface of the mobile device 100 is placed and in contact with the target plane 180, The device coordinate system 182 is defined based on the selection. Once the selection is made, the mobile device 100 may retrieve the corresponding rotational transformation for the device coordinate system 182.
  • the acceierometer 130 measures the gravity vector when it is stationary.
  • the acceierometer coordinate system 183 is the coordinate system in which all of the acceierometer measurements lie.
  • the acceierometer coordinate system 183 is not necessarily aligned with the device coordinate system 182.
  • the acceierometer coordinate system 183 and the device coordinate system 182 are misaligned when the x- y plane of the surface coordinate system 182 is not aligned with the ax-ay plane of the acceierometer coordinate system 183,
  • the misalignment described herein is the rotational misalignment. This rotational misalignment can be calibrated by the factory manufacturing the mobile device 100, or by a user who performs a user calibration process.
  • the calibration result is the rotatio al transformation stored in the memory 120 of FIG. I .
  • the rotational transformation is in the form of a rotational matrix.
  • the rotational transformation transforms an accelerometer measurement from the accelerometer coordinate system 183 into the device coordinate system 182. If the accelerometer coordinate system 183 is aligned with the device coordinate system 182, the rotational matrix is an identify matrix and no transformation is necessary.
  • the gravity vector g which equals 9.81 m/ ' s 2 measured at sea level, points straight down to the earth center.
  • the gravity vector g may be represented by [0, 0, 9.81] in a coordinate system having a z axis pointing straight down to the earth center.
  • the gravity vector measured by the accelerometer 130 is g ', which has the same vector length (9.81 m/s z measured at sea level) as the gravity vector g, but may be a rotated version of g due to the orientation of the accelerometer 130 and calibration errors.
  • the measured gravit vector g ' may be [5, 2.69, 8] in the accelerometer coordinate system 183.
  • the gravity vector in the device coordinate system 182 will also be g
  • the gravity vector g" in the device coordinate system 182 is further rotated from g' by an angle in the 3-D Euclidian frame.
  • the rotational transformation stored in the mobile device 100 is a transformation from g ' to g ".
  • a bias exists in the gravity measurement if the length of the measured gravity vector is greater than 9.81.
  • Techniques for removing biases from gravity measurements include the extended Kalman filtering, which produces accurate results when the amount of bias is small (e.g., within +/- 3 degrees as sines and cosines can be approximated by straight lines in that region). Alternative filtering techniques may also be used.
  • FIG. 3A-3D illustrate examples of target planes 210-240 with different orientations, such as tilted with respect to the horizontal axis (FIG. 3A and FIG, 3B), vertical (FIG. 3C) and horizontal (FIG. 3D) orientations.
  • Each target plane 210-240 has a planar surface facing the mobile device 100, or at least a portion of the surface facing the mobile device 100 is planar.
  • a user may hold the mobile device 100 firmly on the planar surface of the target plane without motion.
  • the accelerometer 130 can then take measurements while the mobile device 100 is stationary and motionless.
  • the surface of the mobile device 100 facing the target plane 210-240 is its coordinate surface (e.g., the surface 170 or 175 of FIG. 2 A and FIG. 2B); that is, the mobile device's surface that is aligned with the device coordinate system 182.
  • This coordinate surface (170 or 175) is held parallel to the planar surface of the target plane 210-240 when the accelerometer 130 takes measurements.
  • FIG. 4 illustrates an embodiment of a method 400 for estimating the gravity vector.
  • the method 400 is performed by a mobile device, such as the processor 1 10 of FIG. 1A, using the measurements of the accelerometer 130.
  • the method 400 may be performed by hardware, software, firmware, or a combination of any of the above.
  • a processor of a mobile device receives a plurality of measurements from the accelerometer located within the mobile device (block 401 ). Each of the measurements is taken when the mobile device is held stationary on the target plane and a surface of the mobile device faces and is in contact with a planar portion of the target plane.
  • the processor calculates an average of the measurements (block 402), and retrieves a rotational transformation between an accelerometer coordinate system and a device coordinate system from a memory in the mobile device (block 403), wherein the device coordinate system is aligned with the surface of the mobile device.
  • Accelerometer bias if any, is removed from the average of the measurements.
  • the rotational transformation is applied to the averaged measurements (with the bias removed) to obtain an estimated gravity vector in a world coordinate system defined by the target plane (block 404).
  • the mobile device 100 may run an application that prompts the user to place the device on a target plane in order to start estimation of the gravity vector relative to the target plane.
  • the application may include instructions to direct the accelerometer 130 to take multiple measurements upon receiving a trigger from a user, and/or when the accelerometer 130 senses that the device is stationary.
  • the application may use the estimated gravity vector in the world coordinate system to compute additional parameters for AR, SLAM applications or other purposes, such as accelerator (and gyroscope) assisted AR, map building in SLAM, dynamic objects handling in SLAM, aligning SLAM maps to a horizontal coordinate system, and the like.
  • one additional parameter that the mobile device 100 may compute is the position vector of the mobile device 100 in the world coordinate system.
  • the position vector indicates how far the mobile device 100 is from the origin of the world coordinate system and which direction the mobile device 100 is moving.
  • the position vector can be computed using the accelerometer 130 while the mobile device 100 is moving.
  • the acceleration of the mobile device 100 in the world coordinate system can be extracted from the accelerometer measurements by subtracting the gravity vector relative to the target plane from the gravity vector measured by the accelerometer 130.
  • the velocity of the mobile device 100 can be obtained by integrating the acceleration over time, and the position vector of the mobile device 100 can be obtained by integrating the velocity over time.
  • the orientation of the mobile device 100 can be measured by the gyroscope 140 within the mobile device 100, which tracks angular rotation of the mobile device 100.
  • the mobile device 100 described above may communicate via one or more wireless communication links through a wireless network that are based on or otherwise support any suitable wireless communication technology.
  • computing device or server may associate with a network including a wireless network.
  • the network may comprise a body area network or a personal area network (e.g., an ultra- wideband network).
  • the network may comprise a local area network or a wide area network.
  • a wireless device may support or otherwise use one or more of a variety of wireless communication technologies, protocols, or standards such as, for example, CDMA, TDMA, OFDM, OFDMA, WiMAX, and Wi-Fi.
  • a wireless device may support or otherwise use one or more of a variety of corresponding modulation or multiplexing schemes.
  • A. wireless device may thus include appropriate components (e.g., air interfaces) to establish and communicate via one or more wireless communication links using the above or other wireless communication technologies.
  • a device may comprise a wireless transceiver with associated transmitter and receiver components (e.g., a transmitter and a receiver) that may include various components (e.g., signal generators and signal processors) that facilitate communication over a wireless medium.
  • a mobile wireless device may therefore wirelessiy communicate with other mobile devices, cell phones, other wired and wireless computers, Internet web-sites, etc.
  • CDMA Code Division Multiple Access
  • TDMA Time division multiple access
  • FDMA Frequency Division Multiple Access
  • OFDMA Orthogonal Frequency-Division Multiple Access
  • SC-FDMA Single Carrier FDMA
  • a CDMA system can implement a radio technology such as Universal Terresiriai Radio Access (UTRA), CDMA2G00, etc.
  • UTRA includes Wideband-CDMA (W-CDMA) and other variants of CDMA.
  • CDMA20G0 covers Interim Standard (1S)- 2000, IS-95 and 18-856 standards.
  • a TDMA system can implement a radio technology such as Global System for Mobile Communications (GSM).
  • GSM Global System for Mobile Communications
  • An OFDMA system can implement a radio technology such as Evolved Universal Terrestrial Radio Access; (Evolved UTRA or E-UTRA), Ultra Mobile Broadband (UMB), institute of Electrical and Electronics Engineers (IEEE) 802.1 1 (Wi-Fi), IEEE 802.16 (WiMAX), IEEE 802.20, Flash-OFDM.RTM., etc.
  • Universal Terrestrial Radio Access (UTRA) and E- UTRA are part of Universal Mobile Telecommunication System (UMTS).
  • 3GPP Long Term Evolution (LTE) is an upcoming release of UMTS that uses E-UTRA, which employs OFDMA on the downlink and SC-FDMA on the uplink.
  • UTRA, E-UTRA, UMTS, LTE and GSM are described in documents from an organization named "3rd Generation Partnership Project” (3 GPP).
  • CDMA2000 and UMB are described in documents from an organization named “3rd Generation Partnership Project 2" (3GPP2).
  • the techniques described herein may be incorporated into (e.g., implemented within or performed by) a variety of mobile apparatuses (e.g., devices).
  • a phone e.g., a cellular phone
  • PDA personal data assistant
  • a tablet e.g., a mobile computer
  • a laptop computer e.g., a tablet
  • an entertainment device e.g., a music or video device
  • a headset e.g., headphones, an earpiece, etc.
  • a medical device e.g., a biometric sensor, a heart rate monitor, a pedometer, an EKG device, etc.
  • a user I/O device e.g., a point-of-sale device, an entertainment device, or any other suitable device.
  • These devices may have different power and data requirements
  • DSP digital signal processor
  • ASIC application specific integrated circuit
  • FPGA field programmable gate array
  • a general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine.
  • a processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration,
  • a software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
  • An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium.
  • the storage medium may be integral to the processor.
  • the processor and the storage medium may reside in an ASIC.
  • the ASIC may reside in a user terminal. Tn the alternative, the processor and the storage medium may reside as discrete components in a user terminal.
  • the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software as a computer program product, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium.
  • Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another.
  • a storage media may be any available media that can be accessed by a computer.
  • Such computer- readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and thai can be accessed by a computer. Also, any connection is properly termed a computer-readable medium.
  • Disk and disc includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Hardware Design (AREA)
  • Human Computer Interaction (AREA)
  • Automation & Control Theory (AREA)
  • Manufacturing & Machinery (AREA)
  • Telephone Function (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

An accelerometer located within a mobile device is used to estimate a gravity vector on a target plane in a world coordinate system. The accelerometer makes multiple measurements, each measurement being taken when the mobile device is held stationary on the target plane and a surface of the mobile device faces and is in contact with a planar portion of the target plane. An average of the measurements is calculated. A rotational transformation between an accelerometer coordinate system and a mobile devices coordinate system is retrieved from a memory in the mobile device, where the mobile devices coordinate system is aligned with the surface of the mobile device. The rotational transformation is applied to the averaged measurements to obtain an estimated gravity vector in a world coordinate system defined by the target plane.

Description

ESTIMATING THE GRAVITY VECTOR IN A WORLD COORDINATE SYSTEM USING AN ACCELEROMETER IN A MOBILE DEVICE
BACKGROUND
[0001] The subject matter disclosed herein relates generally to the use of an accelerometer for measuring gravity vectors.
[8002] An accelerometer (also referred to as a motion sensor) measures proper acceleration, which is the acceleration it experiences relative to a freefall (or the inertia). Proper acceleration is associated with the phenomenon of weight experienced by a proof mass that resides in a frame of reference of the accelerometer. An accelerometer measures the weight per unit of the proof mass, a quantity also known as specific force, or g-foree. Conceptually, an accelerometer behaves as a damped mass on a spring. When the accelerometer experiences acceleration, the position of the proof mass is displaced with respect to the frame. The displacement is measured to determine the acceleration.
10003] A gyroscope (also referred to as a rotation sensor) measures the angular velocity of a system in the inertia! reference frame. By using the original orientation of the system in the inertia! reference frame as the initial condition and integrating the angular velocity, the system's current orientation can be known. Conceptually, a gyroscope is a spinning rotor which maintains its orientation based on the principle of conservation of angular momentum. This phenomenon can be used for measuring and maintaining orientation in many applications, such as compasses and stabilizers in aircraft and spacecraft.
[0004] Accel erometers and gyroscopes have been incorporated into a variety of consumer electronics. The integration of accelerometers and gyroscopes allows for more accurate and robust augmented reality (AR) applications, simultaneous localization and mapping (SLAM) applications, computer vision applications, navigation applications, stability control applications, and a wide range of other applications.
10005] For many of the above-mentioned applications, e.g., for AR and computer vision applications, one of the assumptions is that the gravity vector in the target coordinate system is known. An example of a target coordinate system is the coordinate system in which an AR target is going to be displayed. However, this gravity vector is generally not known in practice, and is generally measured in response to a request from an application such as an AR or SLAM application. The accuracy requirement for the measurements is quite high (e.g., less than a degree); otherwise the measurements would be unusable. Higher accuracy would lead to a better performing application or more demanding use cases. Existing accelerometer-assisted techniques for measuring the gravity vector in the target coordmate system typically have low accuracy due to error accumulations in multiple transformation steps where each step may introduce errors.
SUMMARY
[0ΘΘ6] An accelerometer located within a mobile device is used to estimate a gravity vector on a target plane in a world coordinate system. In one embodiment, a processor of the mobile device receives a plurality of measurements from the accelerometer. Each of the measurements is taken when the mobile device is held stationary on the target plane and a surface of the mobile device faces and is in contact with a planar portion of the target plane. The processor calculates an average of the measurements, and retrieves a rotational transformation between an accelerometer coordinate system and a device coordinate system from a memory in the mobile device, wherein the device coordmate system is aligned with the surface of the mobile device. The rotational transformation is applied to the averaged measurements to obtain an estimated gravity vector in a world coordmate system defined by the target plane.
[80Θ7] In another embodiment, a mobile device comprises: an accelerometer, a memory to store a rotational transformation between a coordinate system of the accelerometer and a coordinate system of the mobile device, and a processor coupled to the memory and the accelerometer. The processor is configured to: receive a plurality of measurements from the accelerometer, each of the measurements being taken when the mobile device is held stationary on a target plane and a surface of the mobile device faces and is in contact with a planar portion of the target plane; calculate an average of the measurements; retrieve the rotational transformation from the memor '-, wherein the coordinate system of the mobile device is aligned with the surface of the mobile de v ice: and apply the rotational transformation to the average to obtain an estimated gravity vector in a world coordinate system defined by the target plane.
[8ΘΘ8] In another embodiment, a computer program product comprises a computer- readable medium, which comprises code for: receiving a plurality of measurements from an accelerometer, each of the measurements being taken when the mobile device is held stationary on a target plane and a surface of the mobile device faces and is in contact with a planar portion of the target plane; calculating an average of the measurements; retrieving a rotational transformation between a coordinate system of the accelerometer and a coordinate system of the mobile device from a memory in the mobile device, wherein the coordinate system of the mobile device is aligned with the surface of the mobile device; and applying the rotational transformation to the average to obtain an estimated gravity vector in a world coordinate system defined by the target plane,
[0009] In yet another embodiment, a mobile device comprises: means for receiving a plurality of measurements from an accelerometer, each of the measurements being taken when the mobile device is held stationary on a target plane and a surface of the mobile device faces and is in contact with a planar portion of the target plane; means for calculating an average of the measurements; means for retrieving a rotational transformation between a coordinate system of the accelerometer and a coordinate system of the mobile device from a memory in the mobile device, wherein the coordinate system of the mobile device is aligned with the surface of the mobile device; and means for applying the rotational transformation to the average to obtain an estimated gravity vector in a world coordinate system defined by the target plane.
BRIEF DESCRIPTION OF THE DRAWINGS
[8018] FIG. 1 is a block diagram of a mobile device in which embodiments of the invention may be practiced.
[0011] FIG. 2 A and FIG. 2B illustrate examples of a side profile of the mobile device of FIG. 1.
[8012] FIG. 3A, FIG. 3B, FIG. 3C and FIG. 3D illustrate examples of a target plane relative to which the gravity vector is to be estimated.
[0013] FIG. 4 is a flow diagram illustrating a method for estimating the gravity vector in a world coordinate system using an accelerometer within a mobile device according to one embodiment
DETAILED DESCRIPTION
[8014] The word "exemplary" or "example" is used herein to mean "serving as an example, instance, or illustration." Any aspect or embodiment described herein as "exemplary" or as an "example" in not necessarily to be construed as preferred or advantageous over other aspects or embodiments. [8015] Embodiments of the invention provide a method for estimating the gravity vector with respect to a target plane using an accelerometer in a mobile device, such as a mobile phone. For AR applications, the target plane is a feature plane on which the AR target is going to be displayed. The target plane can have any orientation; for example, the target plane can be aligned with the horizontal axis, aligned with the vertical axis, or tilted with respect to the horizontal or vertical axis. This target plane defines a world coordinate system for AR applications. The mobile device can serve as a convenient tool for estimating or measuring the gravity vector in the world coordinate system, it is appreciated that the estimation technique described herein is not limited to AR applications; it is applicable to a wide range of applications where the target plane may be a plane relative to which the gravity vector is unknown and needs to be measured.
[8016] As used herein, the term "world coordinate system," "tracking coordinate system" or "target coordinate system" refers to a coordinate system having a 2-D coordinate plane defined by the target plane. That is, the x-y (or x-z or y-z) coordinate plane of the world coordinate system is parallel to the target plane. The term "accelerometer coordinate system" refers to the coordinate system of the accelerometer within the mobile device. The term "device coordinate system" or "surface coordinate system" refers to a coordinate system having a 2-D coordinate plane defined by a surface of the mobile device. In one embodiment, a mobile device may have only one device coordinate system defined by one surface (either front or back surface) of the mobile device. This surface is referred to as a "coordinate surface" of the mobile device. For a mobile device having its front surface parallel to its back surface, both front and back surfaces of the mobile device can be the coordinate surface. If the front surface of a mobile device is not parallel to the back surface, only one surface (either front or back) of the mobile device that defines the device coordinate system is the coordinate surface. In another embodiment, a mobile device may have two device coordinate systems; one defined by the front surface and the other one defined by the back surface. The mobile device may select either the front surface or the back surface as the coordinate surface. In one embodiment, the mobile device can be configured to retrieve one of two rotational transformations from the memory for alignment with the front or back surface, where the two rotational transformations include a first transformation between the coordinate system of the accelerometer and the front surface of the mobile device, and a second transformation between the coordinate system of the accelerometer and the back surface of the mobile device. [8017] In one embodiment, a user places the coordinate surface of the mobile device firmly on the target plane such that the coordinate surface is parallel to the target plane. After the mobile device is firmly placed and motionless, the accelerometer within the mobile device makes one or more measurements. Each measurement is a measured gravity vector in the accelerometer coordinate system, if the rotational transformation between the device coordinate system and the accelerometer coordinate system is known, a conversion engine in ihe mobile device can convert the gravity measurements from the accelerometer coordinate system to the device coordinate system. As the coordinate surface of the mobile device is parallel to the target plane, the gravity measurements in the device coordinate system are the same as those in the world coordinate system. The gravity measurements can be averaged over a time window to obtain an accurate estimate of the gravity vector relative to the target plane in the world coordinate system.
[8018] The gravity vector measured in the world coordinate system can be used for AR, SLAM and various other applications. In one embodiment, the accelerometer is factory-calibrated such that the coordinate system of the accelerometer aligns with the surface of the mobile device. Other calibration techniques, such as user-performed calibration, can also be used. The alignment result is a rotational transformation, which can be stored in the memory of the mobile device. Thus, a mobile device can be aligned (calibrated) once and the alignment result can be used in the subsequent measurements.
[8019] FIG. 1 is a block diagram illustrating a system in which embodiments of the invention may be practiced. The system may be a mobile device 100, which may include a processor 110, a memory 120, an interface 160 and one or more sensors such as an accelerometer 130 and a gyroscope 140. In one embodiment, the mobile device 100 may include both the accelerometer 130 and the gyroscope 140; in an alternative embodiment, the mobile device 100 may include only the accelerometer 130. It should be appreciated that the mobile device 100 may also include a display device, a user interface (e.g., keyboard, touch-screen, etc.), a power device (e.g., a battery), as well as other components typically associated with a mobile communication device. For example, the interface 160 may be a wireless transceiver to transmit and receive wireless signals through a wireless link to/from a wireless network, or may be wired interface for direct connection to networks (e.g., the Internet). Thus, the mobile device 100 may be a: mobile phone (e.g., cell phone, smart phone, etc.), personal digital assistant, mobile computer, tablet, personal computer, laptop computer, e-reader, or any type of mobile device that has motion sensing and/or rotation sensing capabilities.
[8(528] In one embodiment, the processor 1 10 may include a conversion engine 1 15, which may be implemented in hardware, firmware, software, or a combination of any of the above. In one embodiment, the processor 1 10 may be a general-purpose processor or a special-purpose processor configured to execute instructions for performing the operations of conversion engine 1 15 that retrieves a stored rotational transformation from the memory 120, where the rotational transformation transforms an acceleration measurement in the accelerometer coordinate system into a corresponding vector in the device coordinate system. The conversion engine 1 15 may apply the rotational transformation to accelerometer measurements to compute the gravity vector in the world coordinate system in a process to be described below.
[8021] The memory 120 may be coupled to the processor 1 10 to store instructions for execution by the processor 1 10. The memory 12.0 may store a device profile 121, which includes the rotational transformation between the accelerometer coordinate system and the device coordinate system. In an embodiment where the mobile device 100 may choose one of its surfaces as the coordinate surface, the memory 120 may store more than one rotational transformation; e.g., one rotational transformation for the front surface and another rotational transformation for the back surface. According to embodiments of the invention, the device profile 121 of the mobile device may store the rotational transformation with other sensor calibration parameters such as the measuring scale, crosstalk of sensors and the alignment between the sensors and the camera, sensor biases, if any, on the mobile device.
[8(522] It should be appreciated that embodiments of the invention as will be hereinafter described may be implemented in conjunction with the execution of instructions by the processor 1 10 of the mobile device 100 and/or other circuitry of the mobile device 100 and/or other devices. Particularly, circuitry of the mobile device 100, including but not limited to the processor 1 10, may operate under the control of a program, routine, or the execution of instructions to execute methods or processes in accordance with embodiments of the invention. For example, such a program may be implemented in firmware or software (e.g. stored in the memory 120 and/or other locations) and may be implemented by processors, such as the processor 1 10, and/ or other circuitry of the mobile device 100. Further, it should be appreciated that the terms processor, microprocessor, circuitry, controller, etc., refer to any type of logic or circuitry capable of executing logic, commands, instructions, software, firmware, functionality and the like.
[0023] FIG, 2.A illustrates a side profile of the mobile device 100 according to one embodiment. The mobile device 100, as viewed in FIG. 2A, has a top surface and a bottom surface, where the bottom surface may be either the front or back side of the mobile device 100. FIG. 2A shows that a device coordinate system 182 having an x-y coordinate plane parallel to the bottom surface. As such, the bottom surface is a coordinate surface 170 for the mobile device 100, which is the surface to which the device coordinate system 182 aligns. This coordinate surface 170 is to be placed on a target plane 180 that defines the world coordinate system.
[0024] FIG. 2B illustrates a side profile of the mobile device 100 according to another embodiment. The mobile device 100, as viewed in FIG. 2B, has a concaved bottom surface, where the concaved bottom surface may be either the front or back side of the mobile device 100, The convex hull of the concaved bottom surface defines a planar surface 175. FIG. 2B shows that the x-y coordinate plane of the device coordinate system 182 is parallel to the planar surface 175, As such, the planar surface 175 is the coordinate surface for the mobile device 100, which is the surface to which the device coordinate system 182 aligns. This coordinate surface is to be placed on the target plane 180 that defines the world coordinate system,
[8025] In an embodiment where the mobile device 100 can select either its front surface or the back surface as the coordinate surface, the selection is based on which surface of the mobile device 100 is placed and in contact with the target plane 180, The device coordinate system 182 is defined based on the selection. Once the selection is made, the mobile device 100 may retrieve the corresponding rotational transformation for the device coordinate system 182.
[0026] The acceierometer 130 measures the gravity vector when it is stationary. The acceierometer coordinate system 183 is the coordinate system in which all of the acceierometer measurements lie. The acceierometer coordinate system 183 is not necessarily aligned with the device coordinate system 182. The acceierometer coordinate system 183 and the device coordinate system 182 are misaligned when the x- y plane of the surface coordinate system 182 is not aligned with the ax-ay plane of the acceierometer coordinate system 183, The misalignment described herein is the rotational misalignment. This rotational misalignment can be calibrated by the factory manufacturing the mobile device 100, or by a user who performs a user calibration process. The calibration result is the rotatio al transformation stored in the memory 120 of FIG. I . In one embodiment, the rotational transformation is in the form of a rotational matrix. The rotational transformation transforms an accelerometer measurement from the accelerometer coordinate system 183 into the device coordinate system 182. If the accelerometer coordinate system 183 is aligned with the device coordinate system 182, the rotational matrix is an identify matrix and no transformation is necessary.
[0027] The gravity vector g, which equals 9.81 m/'s2 measured at sea level, points straight down to the earth center. The gravity vector g may be represented by [0, 0, 9.81] in a coordinate system having a z axis pointing straight down to the earth center. The gravity vector measured by the accelerometer 130 is g ', which has the same vector length (9.81 m/sz measured at sea level) as the gravity vector g, but may be a rotated version of g due to the orientation of the accelerometer 130 and calibration errors. For example, the measured gravit vector g ' may be [5, 2.69, 8] in the accelerometer coordinate system 183. If the accelerometer coordinate system 183 and the device coordinate system 182 are aligned, the gravity vector in the device coordinate system 182 will also be g However, when the accelerometer coordinate system 183 and the device coordinate system 182 are misaligned, the gravity vector g" in the device coordinate system 182 is further rotated from g' by an angle in the 3-D Euclidian frame. The rotational transformation stored in the mobile device 100 is a transformation from g ' to g ". A bias exists in the gravity measurement if the length of the measured gravity vector is greater than 9.81. Techniques for removing biases from gravity measurements include the extended Kalman filtering, which produces accurate results when the amount of bias is small (e.g., within +/- 3 degrees as sines and cosines can be approximated by straight lines in that region). Alternative filtering techniques may also be used.
[8028] FIG. 3A-3D illustrate examples of target planes 210-240 with different orientations, such as tilted with respect to the horizontal axis (FIG. 3A and FIG, 3B), vertical (FIG. 3C) and horizontal (FIG. 3D) orientations. Each target plane 210-240 has a planar surface facing the mobile device 100, or at least a portion of the surface facing the mobile device 100 is planar. To estimate the gravity vector relative to the target plane 210-240, a user may hold the mobile device 100 firmly on the planar surface of the target plane without motion. The accelerometer 130 can then take measurements while the mobile device 100 is stationary and motionless. The surface of the mobile device 100 facing the target plane 210-240 is its coordinate surface (e.g., the surface 170 or 175 of FIG. 2 A and FIG. 2B); that is, the mobile device's surface that is aligned with the device coordinate system 182. This coordinate surface (170 or 175) is held parallel to the planar surface of the target plane 210-240 when the accelerometer 130 takes measurements.
[8029] FIG. 4 illustrates an embodiment of a method 400 for estimating the gravity vector. In one embodiment, the method 400 is performed by a mobile device, such as the processor 1 10 of FIG. 1A, using the measurements of the accelerometer 130. In one embodiment, the method 400 may be performed by hardware, software, firmware, or a combination of any of the above.
[0036] In one embodiment, a processor of a mobile device receives a plurality of measurements from the accelerometer located within the mobile device (block 401 ). Each of the measurements is taken when the mobile device is held stationary on the target plane and a surface of the mobile device faces and is in contact with a planar portion of the target plane. The processor calculates an average of the measurements (block 402), and retrieves a rotational transformation between an accelerometer coordinate system and a device coordinate system from a memory in the mobile device (block 403), wherein the device coordinate system is aligned with the surface of the mobile device. Accelerometer bias, if any, is removed from the average of the measurements. The rotational transformation is applied to the averaged measurements (with the bias removed) to obtain an estimated gravity vector in a world coordinate system defined by the target plane (block 404).
[Θ031 In one embodiment, the mobile device 100 may run an application that prompts the user to place the device on a target plane in order to start estimation of the gravity vector relative to the target plane. The application may include instructions to direct the accelerometer 130 to take multiple measurements upon receiving a trigger from a user, and/or when the accelerometer 130 senses that the device is stationary. After applying the rotational transformation to the measurements according to the method 400, the application may use the estimated gravity vector in the world coordinate system to compute additional parameters for AR, SLAM applications or other purposes, such as accelerator (and gyroscope) assisted AR, map building in SLAM, dynamic objects handling in SLAM, aligning SLAM maps to a horizontal coordinate system, and the like. [8032] For example, when a user wants to localize the mobile device 100 relative to the world (i.e., a target plane), one additional parameter that the mobile device 100 may compute is the position vector of the mobile device 100 in the world coordinate system. The position vector indicates how far the mobile device 100 is from the origin of the world coordinate system and which direction the mobile device 100 is moving. The position vector can be computed using the accelerometer 130 while the mobile device 100 is moving. The acceleration of the mobile device 100 in the world coordinate system can be extracted from the accelerometer measurements by subtracting the gravity vector relative to the target plane from the gravity vector measured by the accelerometer 130. The velocity of the mobile device 100 can be obtained by integrating the acceleration over time, and the position vector of the mobile device 100 can be obtained by integrating the velocity over time. The orientation of the mobile device 100 can be measured by the gyroscope 140 within the mobile device 100, which tracks angular rotation of the mobile device 100.
[0033] It should be appreciated that when the mobile device 100 described above is a wireless mobile device, it may communicate via one or more wireless communication links through a wireless network that are based on or otherwise support any suitable wireless communication technology. For example, in some aspects computing device or server may associate with a network including a wireless network. In some aspects the network may comprise a body area network or a personal area network (e.g., an ultra- wideband network). In some aspects the network may comprise a local area network or a wide area network. A wireless device may support or otherwise use one or more of a variety of wireless communication technologies, protocols, or standards such as, for example, CDMA, TDMA, OFDM, OFDMA, WiMAX, and Wi-Fi. Similarly, a wireless device may support or otherwise use one or more of a variety of corresponding modulation or multiplexing schemes. A. wireless device may thus include appropriate components (e.g., air interfaces) to establish and communicate via one or more wireless communication links using the above or other wireless communication technologies. For example, a device may comprise a wireless transceiver with associated transmitter and receiver components (e.g., a transmitter and a receiver) that may include various components (e.g., signal generators and signal processors) that facilitate communication over a wireless medium. As is well known, a mobile wireless device may therefore wirelessiy communicate with other mobile devices, cell phones, other wired and wireless computers, Internet web-sites, etc. [8034] The techniques described herein can be used for various wireless communication systems such as Code Division Multiple Access (CDMA), Time division multiple access (TDMA), Frequency Division Multiple Access (FDMA), Orthogonal Frequency-Division Multiple Access (OFDMA), Single Carrier FDMA (SC-FDMA) and other systems. The terms "system" and "network" are often used interchangeably. A CDMA system can implement a radio technology such as Universal Terresiriai Radio Access (UTRA), CDMA2G00, etc. UTRA includes Wideband-CDMA (W-CDMA) and other variants of CDMA. CDMA20G0 covers Interim Standard (1S)- 2000, IS-95 and 18-856 standards. A TDMA system can implement a radio technology such as Global System for Mobile Communications (GSM). An OFDMA system can implement a radio technology such as Evolved Universal Terrestrial Radio Access; (Evolved UTRA or E-UTRA), Ultra Mobile Broadband (UMB), institute of Electrical and Electronics Engineers (IEEE) 802.1 1 (Wi-Fi), IEEE 802.16 (WiMAX), IEEE 802.20, Flash-OFDM.RTM., etc. Universal Terrestrial Radio Access (UTRA) and E- UTRA are part of Universal Mobile Telecommunication System (UMTS). 3GPP Long Term Evolution (LTE) is an upcoming release of UMTS that uses E-UTRA, which employs OFDMA on the downlink and SC-FDMA on the uplink. UTRA, E-UTRA, UMTS, LTE and GSM are described in documents from an organization named "3rd Generation Partnership Project" (3 GPP). CDMA2000 and UMB are described in documents from an organization named "3rd Generation Partnership Project 2" (3GPP2).
[0035] The techniques described herein may be incorporated into (e.g., implemented within or performed by) a variety of mobile apparatuses (e.g., devices). For example, one or more aspects taught herein may be incorporated into a phone (e.g., a cellular phone), a personal data assistant ("PDA"), a tablet, a mobile computer, a laptop computer, a tablet, an entertainment device (e.g., a music or video device), a headset (e.g., headphones, an earpiece, etc.), a medical device (e.g., a biometric sensor, a heart rate monitor, a pedometer, an EKG device, etc.), a user I/O device, a point-of-sale device, an entertainment device, or any other suitable device. These devices may have different power and data requirements
[8036] Those of skill in the art would understand that information and signals may be represented using any of a variety of different technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and chips that may be referenced throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.
[8(537] Those of skill would further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this mterchangeahility of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
[8038] The various illustrative logical blocks, modules, and circuits described in connection with the embodiments disclosed herein may be implemented or performed with a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration,
[0039] The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal. Tn the alternative, the processor and the storage medium may reside as discrete components in a user terminal.
[8(548] In one or more exemplary embodiments, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software as a computer program product, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a computer. By way of example, and not limitation, such computer- readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and thai can be accessed by a computer. Also, any connection is properly termed a computer-readable medium. For example, if the software is transmitted from a web site, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. Disk and disc, as used herein, includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.
[8(541] The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

What is claimed is:
1 , A method for estimating a gravity vector on a target plane using an accelerometer located within a mobile device, comprising:
receiving a plurality of measurements by a processor of the mobile device from the accelerometer, each of the measurements being iaken when the mobile de vice is held stationary on the target plane and a surface of the mobile device faces and is in contact with a planar portion of the target plane;
calculating an average of the measurements;
retrieving a rotational transformation between a coordinate system of the accelerometer and a coordinate system of the mobile device from a memory in the mobile device, wherein the coordinate system of the mobile device is aligned with the surface of the mobile device; and
applying the rotational transformation to the average to obtain an estimated gravity vector in a world coordinate system defined by the target plane.
2, The method of claim 1, wherein the rotational transformation is calibrated by a factory manufacturing the mobile device or by a user of the mobile device.
3. The method of claim 1, further comprising: removing a bias of the accelerometer from the average of the measurements.
4. The method of claim 1 , wherein the rotational transformation is a rotation matrix between accelerometer axes and the surface of ihe mobile device.
5, The method of claim 1, wherein the target plane is tilted with respect to a horizontal axis.
6. The method of claim 1 , %rherein the target plane is aligned to a horizontal axis or a vertical axis.
7 The method of claim 1 , further comprising: prompting a user of the mobile device to start estimation of the gravity vector relative to the target plane; and
starting the measurements by ihe accelerometer in response to a irigger from the user and upon detecting the mobile device is motionless.
8. The method of claim 1, further comprising:
selecting a front surface or a back surface of the mobile de vice as the surface that defines the coordinate system of the mobile device based on which one of the front surface and the back surface is placed and in contact with the target plane.
9. The method of claim 8, wherein selecting further comprises:
retrieving one of rotational transformations from the memory of the mobile device, wherein the rotational transformations in the memory include a first transformation between ihe coordinate system of the accelerometer and the front surface of the mobile device, and a second transformation between the coordinate system of the accelerometer and the back surface of the mobile device.
10. A mobile device comprising:
an accelerometer;
a memory to store a rotational transformation between a coordinate system of the accelerometer and a coordinate system of the mobile device; and
a processor coupled to the memory and the accelerometer, the processor configured to:
receive a plurality of measurements from the accelerometer, each of the measurements being taken when the mobile device is held stationary on a target plane and a surface of the mobile device faces and is in contact with a planar portion of the target plane;
calculate an a v erage of the measurements:
retrieve the rotational transformation from the memory, wherein the coordinate system of the mobile device is aligned with the surface of the mobile device; and
apply the rotational transformation to the average to obtain an estimated gravity vector in a world coordinate system defined by the target plane.
1 1. The mobile device of claim 10, wherein the rotational transformation is calibrated by a factory manufacturing the mobile device or a user of the mobile device.
12. The mobile device of claim 10, wherein the processor is further configured to: remove a bias of the accelerometer from the average of the measurements.
13. The mobile device of claim 10, wherein the rotational transformation is a rotation matrix between accelerometer axes and the surface of the mobile device.
14. The mobile device of claim 10, wherein the mobile device is a mobile phone.
15. The mobile device of claim 10, wherein the surface of the mobile device is planar or concave.
16. The mobile device of claim 10, wherein the processor is further configured to:
prompt a user of the mobile device to start estimation of the gravity vector relative to the target plane; and
start the measurements of the accelerometer in response to a trigger from the user and upon detecting the mobile device is motionless.
17. The mobile device of claim 10, wherein the processor is further configured to select a front surface or a back surface of the mobile device as the surface that defines the coordinate system of the mobile device based on which one of the front surface and the back surface is placed and in contact with the target plane.
18. The mobile device of claim 17, wherein the memory stores rotational transformations that include a first transformation between the coordinate system of the accelerometer and the front surface of the mobile device, and a second transformation between the coordinate system of the accelerometer and the back surface of the mobile device, and wherein the processor is configured to retrieve one of the rotational transformations from the memory of the mobile de vice.
19. A computer program product comprising: a computer-readable medium comprising code for:
receiving a plurality of measurements from an accelerometer in a mobile device, each of the measurements being taken when the mobile device is held stationary on a target plane and a surface of the mobile device faces and is in contac t with a planar portion of the target plane;
calculating an average of the measurements;
retrieving a rotational transformation between a coordinate system of the accelerometer and a c oordinate system of the mobile device fro a memory in the mobile device, wherein the coordinate system of the mobile device is aligned with the surface of the mobile device; and
applying the rotational transformation to the average to obtain an estimated gravity vector in a %rorld coordinate system defined by the target plane,
20. The computer program product of claim 19, wherein the rotational transformation is calibrated by a factory manufacturing the mobile device or a user of the mobile device.
21. The computer program product of claim 19, further comprising code for removing a bias of the accelerometer from the average of the measurements.
22. The computer program product of claim 19, wherein the rotational transformation is a rotation matrix between accelerometer axes and the surface of the mobile device.
23. The computer program product of claim 19, wherein the target plane is tilted with respect to a horizontal axis.
24. The computer program product of claim 19, wherein the target plane is aligned to a horizontal axis or a vertical axis.
25. The computer program product of claim 19, further comprising code for:
prompting a user of the mobile device to start estimation of the gravity vector relative to the target plane; and starting the measurements by the accelerometer in response to a trigger from the user and upon detecting the mobile device is motionless.
26. The computer program product of claim 9, further comprising code for:
selecting a front surface or a back surface of the mobile device as the surface that defines the coordinate system of the mobile device based on which one of the front surface and the back surface is placed and in eoniact with the target plane.
27. The computer program product of claim 26, further comprising code for:
retrieving one of rotational transformations from the memory of the mobile device, wherein the rotational transformations in the memory include a first transformation between the coordinate system of the accelerometer and the front surface of the mobile device, and a second transformation between the coordinate system of the accelerometer and the back surface of the mobile device.
28. A mobile device comprising:
means for receiving a plurality of measurements from an accelerometer, each of the measurements being taken when the mobile device is held stationary on a target plane and a surface of the mobile device faces and is in contact with a planar portion of the target plane;
means for calculating an average of ihe measurements;
means for retrieving a rotational transformation between a coordinate system of the accelerometer and a coordinate system of the mobile device from a memory in the mobile device, wherein the coordinate system of the mobile device is aligned with the surface of ihe mobile device; and
means for applying the rotational transformation to the average to obtain an estimated gravity vector in a world coordinate system defined by the target plane.
29. The mobile device of claim 28, wherein the rotational transformation is calibrated by a factory manufacturing the mobile device or a user of the mobile device.
30. The mobile device of claim 28, further comprising: means for removing a bias of the accelerometer from the average of the measurements.
31. The mobile device of claim 28, wherein the rotational transformation is a rotation matrix between accelerometer axes and the surface of the mobile device.
32. The mobile device of claim 28, wherein the mobile device is a mobile phone.
33. The mobile device of claim 28, wherein the target plane is tilted with respect to a horizontal axis.
34. The mobile device of claim 28, wherein the target plane is aligned to a horizontal axis or a vertical axis.
35. The mobile device of claim 28, wherein the surface of the mobile device is planar or concave.
36. The mobile device of claim 28, further comprising:
means for prompting a user of the mobile device to start estimation of the gravity vector relative to the target plane; and
means for starting the measurements by the accelerometer in response to a trigger from the user and upon detecting the mobile device is motionless.
37. The mobile device of claim 28, further comprising:
means for selecting a front surface or a back surface of the mobile device as the surface that defines the coordinate system of the mobile device based on which one of the front surface and the back surface is placed and in contact with the target plane.
38. The mobile device of claim 37, further comprising:
means for retrieving one of rotational transformations from the memory of the mobile device, wherein the rotaiional transformations in the memory include a first transformation between the coordinate system of the accelero meter and the front surface of the mobile device, and a second transformation between the coordinate system of the accelerometer and the back surface of the mobile device.
PCT/US2013/064172 2012-11-02 2013-10-09 Estimating the gravity vector in a world coordinate system using an accelerometer in a mobile device WO2014070399A1 (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
KR1020157014145A KR20150082374A (en) 2012-11-02 2013-10-09 Estimating the gravity vector in a world coordinate system using an accelerometer in a mobile device
CN201380057457.XA CN104756039A (en) 2012-11-02 2013-10-09 Estimating the gravity vector in a world coordinate system using an accelerometer in a mobile device
EP13780479.5A EP2915015A1 (en) 2012-11-02 2013-10-09 Estimating the gravity vector in a world coordinate system using an accelerometer in a mobile device
JP2015540684A JP2016503495A (en) 2012-11-02 2013-10-09 Estimation of gravity vector in world coordinate system using accelerometer in mobile device

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US201261722087P 2012-11-02 2012-11-02
US61/722,087 2012-11-02
US13/767,784 US20140129176A1 (en) 2012-11-02 2013-02-14 Estimating the gravity vector in a world coordinate system using an accelerometer in a mobile device
US13/767,784 2013-02-14

Publications (1)

Publication Number Publication Date
WO2014070399A1 true WO2014070399A1 (en) 2014-05-08

Family

ID=50623152

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2013/064172 WO2014070399A1 (en) 2012-11-02 2013-10-09 Estimating the gravity vector in a world coordinate system using an accelerometer in a mobile device

Country Status (7)

Country Link
US (1) US20140129176A1 (en)
EP (1) EP2915015A1 (en)
JP (1) JP2016503495A (en)
KR (1) KR20150082374A (en)
CN (1) CN104756039A (en)
TW (1) TWI546540B (en)
WO (1) WO2014070399A1 (en)

Families Citing this family (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9444279B1 (en) * 2013-05-21 2016-09-13 Google Inc. Wireless charging identification using sensors
KR102158843B1 (en) 2013-08-05 2020-10-23 삼성전자주식회사 Method for user input by using mobile device and mobile device
CN105510632B (en) * 2015-11-24 2018-12-28 上海汽车集团股份有限公司 Obtain the method and apparatus of pickup data
US10564177B2 (en) * 2016-08-01 2020-02-18 Samsung Electronics Co., Ltd. System and method for user activity recognition using accelerometer
US10210662B2 (en) 2016-12-09 2019-02-19 Fyusion, Inc. Live augmented reality using tracking
US10665024B2 (en) 2016-12-12 2020-05-26 Fyusion, Inc. Providing recording guidance in generating a multi-view interactive digital media representation
US9996945B1 (en) 2016-12-12 2018-06-12 Fyusion, Inc. Live augmented reality guides
CN111108413A (en) * 2017-08-25 2020-05-05 尹迪泰特有限责任公司 Mobile device with sensor
KR102192141B1 (en) * 2018-10-15 2020-12-16 주식회사 삼양감속기 Apparatus for orientation mapping of multi-axis vibration sensor measurment and Driving mathod thereof
CN111383282B (en) * 2018-12-29 2023-12-01 杭州海康威视数字技术股份有限公司 Pose information determining method and device
CN110308304B (en) * 2019-06-27 2021-03-23 湘电风能有限公司 Wind generating set wind vane zero calibration method
CN111238527B (en) * 2020-01-15 2023-09-29 桂林市优创电子科技有限公司 Step counting method based on triaxial acceleration sensor
US11941231B2 (en) 2021-08-29 2024-03-26 Snap Inc. Camera interfaces to interact with IoT devices
US20230063194A1 (en) * 2021-08-29 2023-03-02 Yu Jiang Tham Controlling iot devices through ar object interaction
US11954774B2 (en) 2021-08-29 2024-04-09 Snap Inc. Building augmented reality experiences with IoT devices
CN114674405B (en) * 2022-03-21 2024-03-01 广州极飞科技股份有限公司 Gravity measurement method, gravity measurement device, computer equipment and computer readable storage medium

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090013755A1 (en) * 2007-07-10 2009-01-15 Polstar Technologies Inc. Calibration jig and algorithms for accelerometer
US20090303204A1 (en) * 2007-01-05 2009-12-10 Invensense Inc. Controlling and accessing content using motion processing on mobile devices
US20100121599A1 (en) * 2007-04-04 2010-05-13 Nxp, B.V. Auto-calibration of orientation sensing systems
US20120203486A1 (en) * 2011-02-08 2012-08-09 Research In Motion Limited System and Method for Calibrating an Accelerometer
US20120221290A1 (en) * 2011-02-28 2012-08-30 Anand Ravindra Oka Portable electronic device adapted to provide an improved attitude matrix

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4908767A (en) * 1987-01-16 1990-03-13 Valentine Research, Inc. Acceleration measuring system
JP2004093552A (en) * 2002-07-10 2004-03-25 Hitachi Metals Ltd Acceleration detection device
WO2006016671A1 (en) * 2004-08-12 2006-02-16 Asahi Kasei Emd Corporation Acceleration measuring device
CN101067628B (en) * 2007-06-02 2013-08-07 中北大学 Vector correcting method for non-gyro accelerometer array mounting error
US8645093B2 (en) * 2009-11-04 2014-02-04 Qualcomm Incorporated Calibrating multi-dimensional sensor for offset, sensitivity, and non-orthogonality
US8626465B2 (en) * 2010-03-30 2014-01-07 Apple Inc. Calibrating sensor measurements on mobile devices

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090303204A1 (en) * 2007-01-05 2009-12-10 Invensense Inc. Controlling and accessing content using motion processing on mobile devices
US20100121599A1 (en) * 2007-04-04 2010-05-13 Nxp, B.V. Auto-calibration of orientation sensing systems
US20090013755A1 (en) * 2007-07-10 2009-01-15 Polstar Technologies Inc. Calibration jig and algorithms for accelerometer
US20120203486A1 (en) * 2011-02-08 2012-08-09 Research In Motion Limited System and Method for Calibrating an Accelerometer
US20120221290A1 (en) * 2011-02-28 2012-08-30 Anand Ravindra Oka Portable electronic device adapted to provide an improved attitude matrix

Also Published As

Publication number Publication date
TW201432265A (en) 2014-08-16
KR20150082374A (en) 2015-07-15
US20140129176A1 (en) 2014-05-08
EP2915015A1 (en) 2015-09-09
TWI546540B (en) 2016-08-21
JP2016503495A (en) 2016-02-04
CN104756039A (en) 2015-07-01

Similar Documents

Publication Publication Date Title
US20140129176A1 (en) Estimating the gravity vector in a world coordinate system using an accelerometer in a mobile device
US9714955B2 (en) Method for aligning a mobile device surface with the coordinate system of a sensor
US20150285835A1 (en) Systems and methods for sensor calibration
Zhou et al. Use it free: Instantly knowing your phone attitude
EP2745076B1 (en) Reducing power consumption or error of a digital compass integrated in a mobile device
US9229084B2 (en) Magnetometer calibration
US9915550B2 (en) Method and apparatus for data fusion of a three-axis magnetometer and three axis accelerometer
KR101608878B1 (en) Rest detection using accelerometer
US8676528B2 (en) System and method for calibrating an accelerometer
US9797727B2 (en) Method and apparatus for determination of misalignment between device and vessel using acceleration/deceleration
KR102029823B1 (en) Geomagnetic sensor calibration apparatus and method thereof
US9752879B2 (en) System and method for estimating heading misalignment
CN107636420B (en) Techniques for pedestrian dead reckoning
US9151622B2 (en) Method and apparatus for estimating moving direction of user and computer-readable storage medium having recorded thereon the method
KR20160028379A (en) Method and apparatus for using multiple filters for enhanced portable navigation
TW201428297A (en) Angular velocity estimation using a magnetometer and accelerometer
CN109764870B (en) Carrier initial course estimation method based on transformation estimation modeling scheme
WO2018067549A1 (en) Method and apparatus for orientation of inertial measuring unit
CN114964214B (en) Extended Kalman filtering attitude calculation method of attitude heading reference system
KR101352245B1 (en) Apparatus and method for calibrating azimuth mems magnetic sensor
TW201139996A (en) Method of automatic orientation detection
WO2015035497A1 (en) Method and apparatus for determination of misalignment between device and vessel using radius of rotation

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13780479

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2015540684

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 2013780479

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 20157014145

Country of ref document: KR

Kind code of ref document: A