US11796304B2 - System and method for tracking a shape - Google Patents
System and method for tracking a shape Download PDFInfo
- Publication number
- US11796304B2 US11796304B2 US17/050,010 US201917050010A US11796304B2 US 11796304 B2 US11796304 B2 US 11796304B2 US 201917050010 A US201917050010 A US 201917050010A US 11796304 B2 US11796304 B2 US 11796304B2
- Authority
- US
- United States
- Prior art keywords
- transponders
- model
- models
- parameter values
- surface model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active, expires
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B7/00—Measuring arrangements characterised by the use of electric or magnetic techniques
- G01B7/28—Measuring arrangements characterised by the use of electric or magnetic techniques for measuring contours or curvatures
- G01B7/287—Measuring arrangements characterised by the use of electric or magnetic techniques for measuring contours or curvatures using a plurality of fixed, simultaneously operating transducers
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B7/00—Measuring arrangements characterised by the use of electric or magnetic techniques
- G01B7/16—Measuring arrangements characterised by the use of electric or magnetic techniques for measuring the deformation in a solid, e.g. by resistance strain gauge
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B7/00—Measuring arrangements characterised by the use of electric or magnetic techniques
- G01B7/28—Measuring arrangements characterised by the use of electric or magnetic techniques for measuring contours or curvatures
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/74—Systems using reradiation of radio waves, e.g. secondary radar systems; Analogous systems
- G01S13/75—Systems using reradiation of radio waves, e.g. secondary radar systems; Analogous systems using transponders powered from received waves, e.g. using passive transponders, or using passive reflectors
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/86—Combinations of radar systems with non-radar systems, e.g. sonar, direction finder
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/87—Combinations of radar systems, e.g. primary radar and secondary radar
- G01S13/878—Combination of several spaced transmitters or receivers of known location for determining the position of a transponder or a reflector
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/88—Radar or analogous systems specially adapted for specific applications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/88—Radar or analogous systems specially adapted for specific applications
- G01S13/89—Radar or analogous systems specially adapted for specific applications for mapping or imaging
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
- G16H50/50—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for simulation or modelling of medical disorders
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H20/00—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
- G16H20/30—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to physical therapies or activities, e.g. physiotherapy, acupressure or exercising
Definitions
- This disclosure relates generally to shape tracking and, in non-limiting embodiments, systems, methods, and computer program products for tracking one or more shapes of a surface.
- Detecting shapes of surfaces poses several technical challenges.
- Infrastructure-based solutions such as depth cameras, function only in environments in which they are deployed and must be arranged within the line-of-sight to the surface being tracked.
- Structure-from-motion techniques in computer vision and RF imaging reconstruct the three-dimensional (3-D) surface of objects using images captured from multiple vantage points.
- Some vision-based systems use commodity cameras coupled with objects labeled with 3-D bar codes to accurately learn the shape of an object from a single image.
- 3-D imaging solutions using ultra-wide band RADAR that require instrumenting the environment with multiple large antennas. Monitoring the structural integrity of large structural objects, such as bridges, requires specialized instruments such as inclinometers and highly-sensitive motion sensors.
- Utilizing RF transponders to track objects requires the use of multiple reader antennas to perform triangulation.
- Using a single antenna reports only a single phase measurement per transponder, which may be insufficient to infer the coordinates for the transponder in 3-D space.
- the use of a single antenna has also been explored for shape determinations, but such systems have limitations and require fixed, static arrangements. There are no existing solutions for tracking the shape of an object in a portable manner that accounts for the multipath of signals.
- a method for tracking a shape of a surface comprising a plurality of radio frequency (RF) transponders arranged thereon, the method comprising: generating, with at least one processor, a first plurality of surface models based on the surface, each surface model comprising surface parameter values and representing a different shape; determining, with at least one processor, an estimated position of the plurality of RF transponders on each surface model of the first plurality of surface models based on the surface model and a predetermined spatial arrangement of the plurality of RF transponders; communicating, with an antenna of a reader device, at least one activation signal to the plurality of RF transponders arranged on the surface, the surface comprising an unknown shape; receiving, with the antenna, a plurality of response signals from at least a subset of the plurality of RF transponders; determining, with at least one processor, radio environment parameters for each surface model of the first plurality of surface models based on the surface parameter values, the estimated position
- RF radio frequency
- the method further comprises: generating a second plurality of surface models based on the at least one selected surface model, each surface model of the second plurality of surface models comprising surface parameter values and representing a different shape; determining, with at least one processor, an estimated position of the plurality of RF transponders on each surface model of the second plurality of surface models based on the surface model and the predetermined spatial arrangement of the plurality of RF transponders; determining, with at least one processor, radio environment parameters for each surface model of the second plurality of surface models based on the surface parameter values, the estimated position of the plurality of RF transponders, and the plurality of response signals; and determining, with at least one processor, at least one second selected surface model of the second plurality of surface models based on the plurality of response signals and the radio environment parameters for the plurality of surface models.
- the surface parameter values for the second plurality of surface models are generated by mutating and/or crosslinking the at least one selected surface model.
- the method further comprises predicting a surface geometry of the unknown shape based on the at least one selected surface model or a subsequent selected surface model from a subsequent plurality of surface models.
- the method further comprises randomly generating the surface parameter values for the first plurality of surface models.
- the surface parameter values for the first plurality of surface models are based on material properties of the surface.
- each surface model is represented as a function of the surface parameter values and defines at least one curve.
- the radio environment parameters comprise phase shift.
- the method further comprises arranging the plurality of RF transponders on the surface as a one-dimensional or two-dimensional array.
- a system for tracking a shape comprising: a plurality of radio frequency (RF) transponders arranged on a surface adapted to be formed into a plurality of shapes; a reader device in communication with an antenna; and at least one processor in communication with the reader device, the at least one processor programmed and/or configured to: generate a first plurality of surface models based on the surface, each surface model comprising surface parameter values and representing a different shape; determine an estimated position of the plurality of RF transponders on each surface model of the first plurality of surface models based on the surface model and a predetermined spatial arrangement of the plurality of RF transponders; communicate, with the antenna, at least one activation signal to each RF transponder of the plurality of transponders arranged on the surface, the surface comprising an unknown shape; receive, with the antenna, a plurality of response signals from at least a subset of the plurality of RF transponders; determine radio environment parameters for each surface model of the first pluralit
- the at least one processor is further programmed or configured to: generate a second plurality of surface models based on the at least one selected surface model, each surface model of the second plurality of surface models comprising surface parameter values and representing a different shape; determine an estimated position of the plurality of RF transponders on each surface model of the second plurality of surface models based on the surface model and the predetermined spatial arrangement of the plurality of RF transponders; determine radio environment parameters for each surface model of the second plurality of surface models based on the surface parameter values, the estimated position of the plurality of RF transponders, and the plurality of response signals; and determine at least one second selected surface model of the second plurality of surface models based on the plurality of response signals and the radio environment parameters for the plurality of surface models.
- the surface parameter values for the second plurality of surface models are generated by mutating and/or crosslinking the at least one selected surface model.
- the at least one processor is further programmed or configured to predict a surface geometry of the unknown shape based on the at least one selected surface model or a subsequent selected surface model from a subsequent plurality of surface models.
- the at least one processor is further programmed or configured to randomly generate the surface parameter values for the first plurality of surface models.
- the surface parameter values for the first plurality of surface models are based on material properties of the surface.
- each surface model is represented as a function of the surface parameter values and defines at least one curve.
- the radio environment parameters comprise phase shift.
- the plurality of RF transponders are arranged on the surface as a one-dimensional or two-dimensional array.
- a computer program product for tracking a shape, wherein a plurality of radio frequency (RF) transponders are arranged on a surface adapted to be formed into a plurality of shapes, comprising at least one non-transitory computer-readable medium including program instructions that, when executed by at least one processor, cause the at least one processor to: generate a first plurality of surface models based on the surface, each surface model comprising surface parameter values and representing a different shape; determine an estimated position of the plurality of RF transponders on each surface model of the first plurality of surface models based on the surface model and a predetermined spatial arrangement of the plurality of RF transponders; receive a plurality of signals from at least a subset of the plurality of RF transponders; determine radio environment parameters for each surface model of the first plurality of surface models based on the surface parameter values, the estimated position of the plurality of RF transponders, and the plurality of signals; and determine at least one selected surface model of the first pluralit
- the program instructions further cause the at least one processor to: generate a second plurality of surface models based on the at least one selected surface model, each surface model of the second plurality of surface models comprising surface parameter values and representing a different shape; determine an estimated position of the plurality of RF transponders on each surface model of the second plurality of surface models based on the surface model and the predetermined spatial arrangement of the plurality of RF transponders; determine radio environment parameters for each surface model of the second plurality of surface models based on the surface parameter values, the estimated position of the plurality of RF transponders, and the plurality of response signals; and determine at least one second selected surface model of the second plurality of surface models based on the plurality of response signals and the radio environment parameters for the plurality of surface models.
- a method for tracking a shape of a surface comprising a plurality of radio frequency (RF) transponders arranged thereon, the method comprising: generating, with at least one processor, a first plurality of surface models based on the surface, each surface model comprising surface parameter values and representing a different shape; determining, with at least one processor, an estimated position of the plurality of RF transponders on each surface model of the first plurality of surface models based on the surface model and a predetermined spatial arrangement of the plurality of RF transponders; communicating, with an antenna of a reader device, at least one activation signal to the plurality of RF transponders arranged on the surface, the surface comprising an unknown shape; receiving, with the antenna, a plurality of response signals from at least a subset of the plurality of RF transponders; determining, with at least one processor, radio environment parameters for each surface model of the first plurality of surface models based on the surface parameter values, the estimated position of the plurality of RF transpond
- Clause 2 The method of clause 1, further comprising: generating a second plurality of surface models based on the at least one selected surface model, each surface model of the second plurality of surface models comprising surface parameter values and representing a different shape; determining, with at least one processor, an estimated position of the plurality of RF transponders on each surface model of the second plurality of surface models based on the surface model and the predetermined spatial arrangement of the plurality of RF transponders; determining, with at least one processor, radio environment parameters for each surface model of the second plurality of surface models based on the surface parameter values, the estimated position of the plurality of RF transponders, and the plurality of response signals; and determining, with at least one processor, at least one second selected surface model of the second plurality of surface models based on the plurality of response signals and the radio environment parameters for the plurality of surface models.
- Clause 3 The method of clauses 1 or 2, wherein the surface parameter values for the second plurality of surface models are generated by mutating and/or crosslinking the at least one selected surface model.
- Clause 4 The method of any of clauses 1-3, further comprising predicting a surface geometry of the unknown shape based on the at least one selected surface model or a subsequent selected surface model from a subsequent plurality of surface models.
- Clause 5 The method of any of clauses 1-4, further comprising randomly generating the surface parameter values for the first plurality of surface models.
- Clause 6 The method of any of clauses 1-5, wherein the surface parameter values for the first plurality of surface models are based on material properties of the surface.
- Clause 7 The method of any of clauses 1-6, wherein each surface model is represented as a function of the surface parameter values and defines at least one curve.
- Clause 8 The method of any of clauses 1-7, wherein the radio environment parameters comprise phase shift.
- Clause 9 The method of any of clauses 1-8, further comprising arranging the plurality of RF transponders on the surface as a one-dimensional or two-dimensional array.
- a system for tracking a shape comprising: a plurality of radio frequency (RF) transponders arranged on a surface adapted to be formed into a plurality of shapes; a reader device in communication with an antenna; and at least one processor in communication with the reader device, the at least one processor programmed and/or configured to: generate a first plurality of surface models based on the surface, each surface model comprising surface parameter values and representing a different shape; determine an estimated position of the plurality of RF transponders on each surface model of the first plurality of surface models based on the surface model and a predetermined spatial arrangement of the plurality of RF transponders; communicate, with the antenna, at least one activation signal to each RF transponder of the plurality of transponders arranged on the surface, the surface comprising an unknown shape; receive, with the antenna, a plurality of response signals from at least a subset of the plurality of RF transponders; determine radio environment parameters for each surface model of the first plurality of surface models based on the surface
- Clause 11 The system of clause 10, wherein the at least one processor is further programmed or configured to: generate a second plurality of surface models based on the at least one selected surface model, each surface model of the second plurality of surface models comprising surface parameter values and representing a different shape; determine an estimated position of the plurality of RF transponders on each surface model of the second plurality of surface models based on the surface model and the predetermined spatial arrangement of the plurality of RF transponders; determine radio environment parameters for each surface model of the second plurality of surface models based on the surface parameter values, the estimated position of the plurality of RF transponders, and the plurality of response signals; and determine at least one second selected surface model of the second plurality of surface models based on the plurality of response signals and the radio environment parameters for the plurality of surface models.
- Clause 12 The system of clauses 10 or 11, wherein the surface parameter values for the second plurality of surface models are generated by mutating and/or crosslinking the at least one selected surface model.
- Clause 13 The system of any of clauses 10-12, wherein the at least one processor is further programmed or configured to predict a surface geometry of the unknown shape based on the at least one selected surface model or a subsequent selected surface model from a subsequent plurality of surface models.
- Clause 14 The system of any of clauses 10-13, wherein the at least one processor is further programmed or configured to randomly generate the surface parameter values for the first plurality of surface models.
- Clause 15 The system of any of clauses 10-14, wherein the surface parameter values for the first plurality of surface models are based on material properties of the surface.
- Clause 16 The system of any of clauses 10-15, wherein each surface model is represented as a function of the surface parameter values and defines at least one curve.
- Clause 17 The system of any of clauses 10-16, wherein the radio environment parameters comprise phase shift.
- Clause 18 The system of any of clauses 10-17, wherein the plurality of RF transponders are arranged on the surface as a one-dimensional or two-dimensional array.
- a computer program product for tracking a shape wherein a plurality of radio frequency (RF) transponders are arranged on a surface adapted to be formed into a plurality of shapes, comprising at least one non-transitory computer-readable medium including program instructions that, when executed by at least one processor, cause the at least one processor to: generate a first plurality of surface models based on the surface, each surface model comprising surface parameter values and representing a different shape; determine an estimated position of the plurality of RF transponders on each surface model of the first plurality of surface models based on the surface model and a predetermined spatial arrangement of the plurality of RF transponders; receive a plurality of signals from at least a subset of the plurality of RF transponders; determine radio environment parameters for each surface model of the first plurality of surface models based on the surface parameter values, the estimated position of the plurality of RF transponders, and the plurality of signals; and determine at least one selected surface model of the first plurality of surface models based on the pluralit
- Clause 20 The computer program product of clause 19, wherein the program instructions further cause the at least one processor to: generate a second plurality of surface models based on the at least one selected surface model, each surface model of the second plurality of surface models comprising surface parameter values and representing a different shape; determine an estimated position of the plurality of RF transponders on each surface model of the second plurality of surface models based on the surface model and the predetermined spatial arrangement of the plurality of RF transponders; determine radio environment parameters for each surface model of the second plurality of surface models based on the surface parameter values, the estimated position of the plurality of RF transponders, and the plurality of response signals; and determine at least one second selected surface model of the second plurality of surface models based on the plurality of response signals and the radio environment parameters for the plurality of surface models.
- FIG. 1 is a schematic diagram of a system for tracking a shape according to a non-limiting embodiment
- FIGS. 2 A- 2 D illustrate different arrangements of RF transponders according to non-limiting embodiments
- FIG. 3 is a flow diagram for a method of tracking a shape according to a non-limiting embodiment
- FIGS. 4 A and 4 B show Bézier curves and surfaces usable with non-limiting embodiments of a system and method for tracking shapes
- FIG. 5 shows test data for observed phase measurements according to a non-limiting embodiment of a system and method for tracking shapes
- FIG. 6 shows signals traversing along multiple paths from RF transponders to a reader device according to non-limiting embodiments of a system and method for tracking shapes
- FIG. 7 shows an implementation of the MUSIC algorithm according to non-limiting embodiments of a system and method for tracking shapes
- FIG. 8 shows test data for various surface materials using a non-limiting embodiment of a system for tracking shapes
- FIG. 9 shows test data for material flexibility using a non-limiting embodiment of a system for tracking shapes
- FIGS. 10 A and 10 B show a surface for tracking user interactions according to a non-limiting embodiment
- FIG. 11 shows test data for the non-limiting embodiment shown in FIGS. 10 A and 10 B ;
- FIG. 12 shows test data for measuring a user's breathing according to a non-limiting embodiment.
- the terms “communication” and “communicate” may refer to the reception, receipt, transmission, transfer, provision, and/or the like, of information (e.g., data, signals, messages, instructions, commands, and/or the like).
- one unit e.g., a device, a system, a component of a device or system, combinations thereof, and/or the like
- to be in communication with another unit means that the one unit is able to directly or indirectly receive information from and/or transmit information to the other unit.
- This may refer to a direct or indirect connection (e.g., a direct communication connection, an indirect communication connection, and/or the like) that is wired and/or wireless in nature.
- two units may be in communication with each other even though the information transmitted may be modified, processed, relayed, and/or routed between the first and second unit.
- a first unit may be in communication with a second unit even though the first unit passively receives information and does not actively transmit information to the second unit.
- a first unit may be in communication with a second unit if at least one intermediary unit processes information received from the first unit and communicates the processed information to the second unit.
- computing device may refer to one or more electronic devices configured to process data.
- a computing device may, in some examples, include the necessary components to receive, process, and output data, such as a display, a processor, a memory, an input device, and a network interface.
- a computing device may be a mobile device.
- a mobile device may include a cellular phone (e.g., a smartphone or standard cellular phone), a portable computer, a wearable device (e.g., watches, glasses, lenses, clothing, and/or the like), a personal digital assistant (PDA), and/or other like devices.
- the computing device may also be a desktop computer or other form of non-mobile computer.
- FIG. 1 depicts a system 1000 for tracking a shape of a surface according to a non-limiting embodiment.
- the system 1000 includes a reader device 100 and a plurality of radio frequency (RF) transponders 102 a - h arranged on a surface 101 .
- the surface 101 may be comprised of any material or combination of materials.
- the surface 101 may be shapeable in that the surface 101 may be adapted (e.g., arranged, manipulated, deformed, depressed, bent, stretched, stressed, compressed, reconfigured, etc.) into a plurality of different shapes.
- a surface may be substantially one-dimensional (1-D), such as a string or wire, two-dimensional (2-D), such as a garment or carpet, or three-dimensional (3-D), such as a curved object.
- the RF transponders 102 a - h are passive RF transponders that are arranged on one or more surfaces (e.g., affixed to the surface, integrated with the surface, and/or the like).
- the surface may form any type of object or portion of object capable of forming different shapes, such as an article of clothing, a carpet, a toy, a tool, a building, a bridge, and/or the like.
- the RF transponders 102 a - h are pictured in FIG.
- the plurality of RF transponders 102 a - h may be arranged in a 1-D array or in any other manner. Any number of RF transponders 102 a - h may be used.
- the arrangement of RF transponders 102 a - h is predetermined and known for the surface 101 when the surface 101 is in a predetermined shape (e.g., flat, straight, planar, a default configuration, etc.).
- the RF transponders 102 a - h may be lightweight, machine-washable, battery-free, and ultra-high frequency (UHF) such that they can be incorporated into a usable material without restricting use of the material.
- each RF transponder 102 a - h may include a limited amount of memory and therefore may contain a 96-bit or 128-bit serial number, rather than the 2 kilobytes of data that is sometimes included with RF transponders having Electronic Product Code (EPC) memory.
- EPC Electronic Product Code
- the RF transponders may be constructed with conductive yarns woven directly into the surface 101 . It will be appreciated by those skilled in the art that any RF transponders may be used, with or without EPC memory, of different shapes, sizes, and manufacture.
- the reader device 100 may include any device having an antenna 103 (or in communication with an antenna 103 ) and a processor and/or circuitry for activating and communicating with the plurality of passive RF transponders 102 a - h .
- the reader device 100 is a single-antenna handheld RFID-reader.
- the reader device 100 may also include a battery that can be periodically recharged.
- the reader device 100 may be a multi-purpose computing device, such as a mobile device (e.g., a smartphone or portable computing device), configured with software and/or circuitry to transmit one or more activation signals, receive a plurality of response signals, and process the received signals. It will be appreciated that the reader device 100 may include any device capable of communicating with RF transponders 102 a - h . Although the reader device 100 may include a plurality of antennas in some non-limiting embodiments, it will be appreciated that in such embodiments only one antenna needs to be used by the reader device 100 to communicate with the RF transponders 102 a - h . The antenna 101 may be internal or external to the reader device 100 .
- the reader device 100 is in communication with a computing device 104 .
- the reader device 100 may be part of or integrated with the computing device 104 , such as a mobile device that includes hardware and/or software for interrogating RF transponders using an antenna.
- the computing device 104 may be independent from the reader device 100 and may communicate with the reader device 100 via a wired or wireless connection. Various other arrangements are possible.
- the reader device 100 may be arranged on the surface 101 , on an associated object, or may be located anywhere else within range of the RF transponders 102 a - h .
- the reader device 100 may be fixed or may move relative to the surface 101 .
- the reader device 100 may be located in a user's pocket, attached to an article of clothing, placed on a table or floor, and/or the like.
- the reader device 100 and RF transponders 102 a - h may move relative to each other at unknown trajectories.
- the reader device 100 using the antenna 103 , communicates one or more activation signals configured to activate the passive RF transponders 102 a - h .
- the activation signal(s) energize each of the passive RF transponders 102 a - h which, in response to being energized, emit one or more response signals.
- the one or more response signals are received by the antenna 103 of the reader device 100 and communicated to the computing device 104 .
- Each of the RF transponders 102 a - h are located at separate distances from the reader device 100 .
- Each distance e.g., an actual distance or a relative distance
- the computing device 104 may determine a distance of each RF transponder 102 a - h based on a phase of each of the response signals.
- the reader device 100 and/or computing device 104 may support detecting the phase of the response signals.
- Various other arrangements are possible.
- the reader device 100 may distinguish between response signals and therefore between RF transponders 102 a - h based on a modulation of each of the response signals.
- each RF transponder 102 a - h may be configured to modulate the response signals (e.g., backscatter signals) using ON-OFF keying by changing an impedance of a respective antenna of the RF transponder.
- the reader device 100 in response to receiving the response signals, may then demodulate each response signal to determine which response signal came from which RF transponder 102 a - h based on a predefined modulation schema.
- a unique identifier corresponding to each RF transponder may be packaged in a respective response signal as a header or otherwise as part of the communicated data.
- the parameters defining the arrangement of RF transponders 102 a - h may be predefined and stored in memory that is accessible to the reader device 100 or the computing device 104 that processes the signals.
- the transponder layout information may be stored in a network-accessible database such that the layout can be identified based on unique identifiers or modulations associated with the RF transponders.
- the transponder layout information may also be stored in EPC memory in one or each of the RF transponders. It will be appreciated that various other arrangements for storing transponder layout information may be used.
- the computing device 104 tracks the shape of the surface 101 without localizing individual RF transponders 102 a - h by implementing mathematical models (e.g., higher-order Bézier curves) of surfaces that represent the surfaces with formulas having a limited number of unknown coefficients.
- the computing device 104 optimizes the unknown coefficients that best fit the measurements obtained from response signals received from the RF transponders 102 a - h .
- the computing device 104 By directly optimizing for surface coefficients, rather than attempting to locate the location of each RF transponder 102 a - h , the computing device 104 reduces the number of variables necessary for determining a shape of the surface 101 and, as a result, reduces the amount of computational resources needed to determine the shape.
- FIGS. 2 A- 2 D various arrangements of RF transponders are shown on different surfaces according to non-limiting embodiments.
- an array of RF transponders 202 is arranged on an article of clothing or directly on a person's body to measure and track the curvature of a person's body (e.g., spine).
- Such an arrangement allows for the system to track a posture of a user in real-time or near-real-time during day-to-day activities or specific activities.
- the RF transponders may be integrated into an article of clothing and/or may be arranged otherwise on the body.
- FIG. 2 A shows an arrangement on a spine of a person
- one or more body parts may be tracked with the system 1000 including, but not limited to, the chest (e.g., to measure breathing), arms and legs (e.g., to measure muscle usage), and/or the like.
- a reader device (not shown in FIG. 2 A ) may be arranged in the user's pocket, on a table or other surface, or elsewhere within range of the transponders 202 arranged on the person's body or clothing.
- FIG. 2 B shows an array of RF transponders 204 on a carpet or other shapeable floor surface, such as a mat, floor board, and/or the like.
- the system is able to track the presence, location, and/or weight of people and/or objects on the floor surface by tracking deformations in the floor surface.
- a reader device (not shown in FIG. 2 B ) may be in a person's pocket or otherwise arranged on a body, on a floor or other surface, or elsewhere within range of the RF transponders 204 arranged on the floor surface.
- FIG. 2 C shows an array of RF transponders 206 on a toy that is being actively deformed from a first shape to a second shape.
- different responses e.g., lights, sounds, games, etc.
- an array of RF transponders 206 on or integrated with a plush toy may allow for different outputs depending on whether it is hugged, dropped, squeezed, and/or the like.
- interactive toys may be provided with RF transponders to generate visual renderings on a computing device based on the shape of an object.
- FIG. 2 D shows an array of RF transponders 208 on a bridge.
- RF transponders may be arranged in a line (e.g., a 1-D array) on a bridge to obtain a 2-D curve or, in other embodiments, may be arranged in a 2-D array to obtain additional shape information.
- RF transponders may be arranged vertically along the sides of a building, across floors and/or ceilings, and/or the like. It will be appreciated that the shape of any portion of a structure may be tracked.
- the method may begin at step 300 in which an activation signal is transmitted from an antenna of a reader device to a plurality of RF transponders arranged on a surface.
- the activation signal may be communicated in a manner to energize a plurality of RF transponders within range of the antenna such that the transponders communicate a response signal in response to being energized.
- the antenna receives at least one response signal from each of a plurality of transponders.
- several signals are received from each transponder via multiple paths.
- the plurality of transponders that respond may be less than the total number of RF transponders due to signal occlusion or other errors.
- the shape of the surface when the transponders communicate the response signals (e.g., a current shape) is the shape that the method determines and outputs at step 314 .
- steps 300 and 302 may be performed subsequent to step 304 .
- a plurality of surface models is generated for the surface being tracked.
- An initial performance of step 304 may result in a first generation (e.g., a first plurality) of surface models for use in a genetic natural selection algorithm.
- the surface parameters that define a surface geometry of each surface model may be randomly generated within constraining parameters based on material properties of the surface, such as stretch, bend, elasticity, and/or the like.
- S(p) may be developed to mathematically represent the geometry of the surface in 2-D or 3-D space.
- the surface parameters may be constrained to allow for surfaces that stretch or curve to the extent that the material(s) of which the surface is comprised can adapt.
- the size of the value p may be significantly smaller than the number of RF transponders on the surface.
- the algorithm optimizes for p to fit the observed signals, even if there is only one channel measurement per RF transponder.
- a shape representation may capture various material properties, e.g., elongation, stiffness, and/or the like, and is intuitive for real world applications, such as touch detection, sag of bridges, spine curvature, and/or the like. In non-limiting embodiments, these considerations may be realized by using higher-order Bézier curves. Such curves may be used to represent the shape of a 1-D surface (such as a string of RF transponders) or a 2-D surface.
- an nth-order Bézier curve in 2-D space for a 1-D array of RF transponders that can deform to various shapes in 2-D space is represented as the following formula:
- FIGS. 4 A and 4 B show Bézier curves and surfaces with control points represented by triangles that may be used in non-limiting embodiments to represent the shape of a surface.
- FIG. 4 A shows a cubic (3rd order) Bézier curve that may be used to characterize many curves on the human body (e.g. spine, stomach, etc.) as well as many everyday surfaces (e.g. carpets, bridges, buildings, etc.) while minimizing over-fitting.
- the first control point may be set to the origin and the final control point may be constrained to lie on the x-axis because only the shape of the surface (e.g., relative coordinates of the control points) is being determined and not the location of the surface in 3-D space.
- a 2-D surface may be modeled using Bézier surfaces based on a Bézier curve in 3-D space.
- FIG. 4 B shows a representation of Bézier surfaces according to non-limiting embodiments.
- a general cubic Bézier surface is constructed as the tensor product of two Bézier curves as defined by the following formula:
- larger shapes formed by structures such as buildings or bridges may not be fully modeled using a cubic Bézier surface.
- a large number of control points may be introduced in specific parts of the surface.
- the surface may be partitioned into smaller segments with overlaps, each of which may be modeled as a separate cubic Bézier surface.
- the extremities of these segments may be stitched together to recover the shape of the entire surface using the following process that assumes continuity and differentiability of the surface it models: (1) intersecting any two surfaces at the RF transponder(s) common to them (e.g., at their extremities); and (2) rotating the second surface so that the normal vectors to the tangent of the surfaces at the intersecting RF transponder(s) align.
- constraining parameters may include maximum curvature and stretch.
- surface models for fabrics may be used to measure the amount of energy required to deform the shape into any curve, depending on constraints of stiffness and elasticity that the material is associated with. Any surface model candidates in a particular generation having a required energy that exceeds a threshold may be eliminated from each generation of Bézier surfaces in the genetic natural selection algorithm. In this manner, outliers may be removed whose shapes are unlikely to form in the real-world due to material constraints.
- Non-limiting embodiments may constrain the surface parameters by stretch by measuring the energy expended to deform the surface into a specific length.
- the net displacement of each RF transponder from each other, relative to their prior configuration at the time the RF transponders were attached to the surface, is represented in the following formula:
- D the prior known spacing between transponders
- d i the surface distance between tag i and i+1.
- the energy required to bend the surface into a specific curvature, starting from an initially planar surface may be determined by the computing device. Such a determination may be based on a fast energy-based surface wrinkle model to calculate the surface smoothness.
- the curvature of the Bézier surface is determined by computing the angular rotation between control points. Specifically, ⁇ j is the angle formed by the segments connecting control points C j ⁇ 1, C j , and C j +1, such that the computing device may then determine the result of the following formulas:
- E stiff ⁇ i n - 1 ⁇ E ⁇ ( C i )
- ⁇ ⁇ E ⁇ ( C i ) ⁇ i - 1 i + 1 ⁇ ( e - ⁇ j - e - ⁇ )
- k e , k s , k g represent elasticity, bending, and density constraining parameters, respectively.
- E total the above energy
- an estimated position of where each RF transponder would be located on each surface model is determined.
- the estimated position may be determined based on the surface model and a predetermined spatial arrangement of the plurality of RF transponders on a surface having a known shape (e.g., planar, flat, straight, default, etc.). For example, given a Bézier surface with constraints on curvature and elasticity, the coordinates of the RF transponders on the surface may be determined based on a projection.
- the RF transponders are arranged in a rectangular array at a predetermined distance D from each other when the surface is in a predetermined position.
- a geometric projection of the positions of the RF transponders is projected from its initial configuration (e.g., a planar arrangement) to that of the new shape. This accounts for both the curvature of the Bézier surface as well as the stretch that the surface experiences uniformly.
- radio environment parameters are determined for each surface model generated at step 304 .
- the signals received by the reader device from the RF transponders may be affected by radio environment parameters.
- radio environment parameters are determined for each shape of the generation. Such a determination may be based on the surface model and the locations of RF transponders on them.
- the radio environment parameters that best-fit the observed signals are determined for each surface model using the following formula:
- E * arg ⁇ ⁇ min E ⁇ ⁇ ⁇ h est ⁇ ( S ⁇ ( p ) , E ) - h obs ⁇ 2
- h est is the estimated radio environment parameters for the surface model S(p) and h obs is the observed radio environment parameters determined from signals received from the RF transponders at step 302 .
- the radio environment parameters may include the following components that influence the wireless channels as the response signals traverse from the RF transponders to the reader device: (1) phase shifts at the RF transponder that may be produced, for example, by change in orientation of the RF transponder that causes shifts in the phase of signals observed from it; (2) signal multipath as a result of the multiple paths a wireless signal from the RF transponder traverses as it reflects off walls, furniture, objects, people, and/or the like; and (3) phase shifts at the reader device that may be arbitrarily introduced by the radio frontend of the reader device to signals received across different RF transponders.
- phase offsets due to the orientation of each RF transponder is an additional variable that can be inferred as part of the radio environment parameters.
- the placement of the RF transponders is predetermined. For example, with planar RF transponders, the orientation of each transponder may be orthogonal to the tangential plane to the surface at the location of the transponder.
- An ambiguity of whether each transponder is facing up or down can be eliminated during the time the RF transponders are arranged on the surface by ensuring that all transponders are oriented along the same direction initially (e.g., all face-up or face-down) to avoid measurement errors. Further, RF transponders of the same make and manufacture may be used as such a set of transponders are likely to have predictable phase shifts based on changes in orientation.
- phase offsets from the orientation of RF transponders can be corrected for without introducing additional variables to estimate in its optimization.
- multiple signal paths are shown between RF transponders on a surface 101 and an antenna 103 of a reader device according to non-limiting embodiments of a system and method for tracking shapes. As shown, some paths involve reflecting off of surfaces 602 , 604 .
- the multiple paths represented are: ( ⁇ 1 , ⁇ 1 ), ( ⁇ 2 , ⁇ 2 ), ( ⁇ 3 , ⁇ 3 ).
- the attenuations and phase shifts experienced by a signal along different paths the signal traverses, e.g., as a signal reflects off various objects 602 , 604 in the environment between each transponder and antenna, may be estimated.
- the algorithm accounts for the fact that adjacent RF transponders will experience similar multipath characteristics due to their proximity.
- the RF transponders on the surface are treated as an array of antennas.
- the computing device may then apply antenna array algorithms to process the known wireless channels at the array of transponders as well as their known geometric configurations, as previously determined, to separate the various signal paths.
- Non-limiting embodiments may utilize the Multiple Signal Classification (MUSIC) algorithm to perform antenna array processing.
- MUSIC Multiple Signal Classification
- the RF transponders may be distorted into arbitrary shapes whose candidate set of surface models have already been determined.
- the MUSIC algorithm may be extended to operate under arbitrary array configurations to estimate the angle-of-arrival ⁇ 1 from a single-antenna reader device to the transponders.
- the transponders are at a known but arbitrary geometry with polar coordinates (r i , ⁇ i ) as shown in FIG. 7 .
- the MUSIC algorithm plots the power of the signal P( ⁇ ) received along each direction ⁇ with a peak(s) at the true direction(s)-of-arrival (e.g., ⁇ 1 ), also as depicted in FIG. 7 .
- the power of the received response signal from any spatial azimuthal angle ⁇ and polar angle ⁇ is represented by the following formula:
- ( ⁇ ) ⁇ is the conjugate-transpose operator.
- the computing device can obtain the magnitude and phase of the signals corresponding to each path by solving the following optimization provided by the following formula:
- the above optimization formula is a least-squares optimization and can be solved in polynomial-time.
- the computing device applies the least-squares method to obtain, for each signal path k, the attenuation a k and phase shift ⁇ k experienced by the signal.
- the computing device accounts for phase shifts introduced by the RF chains (e.g., an arrangement of transmitters, receivers, cables, amplifiers, instruments, and/or other like components) of the reader device.
- the RF chains e.g., an arrangement of transmitters, receivers, cables, amplifiers, instruments, and/or other like components
- the computing device accounts for phase shifts introduced by the RF chains (e.g., an arrangement of transmitters, receivers, cables, amplifiers, instruments, and/or other like components) of the reader device.
- one or more surface models are selected from the first plurality of surface models as a best-fit for the current unknown shape of the surface being tracked based on the observed signals from the RF transponders.
- the selection may be the first selection of a plurality of iterative selections in a genetic natural selection algorithm to optimize for the best candidate surface model.
- the first iteration of surfaces may not be a sufficient best-fit and, as a result, the method may proceed to step 304 to generate a next generation of surface models.
- the following formula may be used to find the surface parameters p that best fit:
- a goodness-of-fit metric may be generated in non-limiting embodiments based on the observed channels, the surface parameters of each surface model, and the radio environment parameters for each surface model determined at step 308 .
- a surface model S having RF transponders positioned at polar coordinates may be represented as n dimensional vectors: (r, ⁇ , ⁇ ).
- h obs denotes the vector of n wireless channels obtained from these transponders.
- the goodness-of-fit of the candidate shape S may be represented as follows:
- N is the number of initial surface models in the first generation (e.g., the first plurality of surface models) generated during the first iteration of step 304 .
- the algorithm converges in approximately 20 iterations from a randomly chosen seed. It will be appreciated that numerous variations are possible depending on the availability of computing resources, the size and complexity of the surface models, the number of RF transponders, and/or other like considerations, and that any number of iterations may be implemented.
- the natural selection algorithm favors individual surface models that fit the observed channels, thereby resulting in progressive improvements in the goodness-of-fit metric with each iteration. In testing of non-limiting embodiments, results achieved sub-centimeter errors in displacement of surfaces measuring 40 cm per-side in 80% of tests. The test data for such non-limiting embodiments is shown in FIG. 8 D .
- the goodness-of-fit metric of one or more surface models in the current generation (e.g., iteration) is sufficient.
- the goodness-of-fit metric may be determined to be a best-fit at step 312 if it satisfies a threshold distance.
- the method may then proceed to step 314 and the shape of the surface at the time the response signals were received from the RF transponders may be determined and output based on the one or more best-fitting surface models.
- the first generation of surface models may not be a sufficient best-fit and, as a result, the method may proceed to step 304 to generate a next generation of surface models as part of the genetic natural selection algorithm.
- only one iteration may be used.
- a goodness-of-fit metric is determined to identify which shapes (surface models) to pass on to the next generation.
- the one or more surface models identified at step 310 may be passed to step 316 after step 312 .
- the one or more selected surface models are processed according to cross-over and/or mutation operations to influence the next generation of surface models generated at step 304 .
- a cross-over operation combines two or more surface models to result in a hybrid surface model (e.g., hybrid shape).
- a cross-over operation may amplify positive characteristics and focus on fine-grained solutions to the goodness-of-fit metric.
- the computing device may perform a cross-over operation of two surface models by averaging the corresponding control point coordinates for the surface models.
- the resulting surface model is a hybrid shape that inherits features from both of the surface models.
- a mutation operation randomly alters the surface parameters of the surface models, within constraints based on the previously selected surface models.
- a mutation operation facilitates the computing system to avoid local optima.
- the computing device may introduce a random off-set to one of the control points in one or more selected surface models.
- the off-set may be from a Gaussian distribution with a standard deviation of D, which is the mean displacement between the RF transponders.
- non-limiting embodiments may use other approaches, rather than a generic natural selection algorithm, to reconstruct a shape from the radio signals received from the RF transponders (e.g., phase and signal strength readings).
- a Monte Carlo search method deep learning (e.g., deep q-learning or other techniques), and/or other techniques may be utilized to determine different RF transponder arrangements to generate a specific pattern of backscatter signals.
- RF transponders may be chosen or designed based on how well the RF transponders interact with the human body and/or how minimally the phase of signals outputted by the RF transponders changes across different orientations. For example, RF transponders that perform consistently while both attached to a body and detached from a body may help reduce the amount of attenuation caused by the human body. Further, RF transponders may be chosen or designed based on radio sensitivity and/or directivity. RF transponders having a relatively weaker directivity may be desirable because a strong directivity may require the RF transponders to directly face the reader device.
- RF transponders may also be chosen or designed to be as small, thin, and flexible as possible, allowing for the RF transponders to be integrated into materials, such as garments, in a non-intrusive and non-restrictive manner.
- the RF transponders may be Omni-ID IQ 150 RF transponders manufactured and sold by Omni-ID, Inc. These example RF transponders are 1.2 ⁇ 5.2 cm, light-weight, thin, and have consistent performance on and off a body. It will be appreciated that various types of RF transponders may be used.
- the reader device 100 may be any suitable device having an antenna or in communication with an antenna and configured to communicate one or more activation signals and receive a plurality of response signals from the RF transponders 102 a - h .
- the reader device may be an Impinj Speedway RFID reader equipped with a single Ettus VERT900 antenna, which provides a software interface for wireless channels.
- the lmpinj Speedway RFID reader and other reader devices support multiple antennas, implementations of the system may involve disabling and/or not using all but one antenna. It will be appreciated that various types and implementations of reader devices may be used.
- a single antenna may be used to reduce the cost of hardware and the amount of effort required to calibrate the antenna and reader device.
- reader devices may be configured to “frequency hop” across 50 channels from 902 MHz to 928 MHz at an interval of approximately 0.2 seconds.
- the reader device 100 may generate an observation data stream that contains all the low-level wireless channel data, including a Universally Unique Identifier (UUID), phase, signal strength, frequency, and/or the like.
- UUID Universally Unique Identifier
- the reader device or a separate computing device may track a shape in real-time or near-real-time.
- the reader device or a computing device may use a sliding window technique to extract quasi-simultaneous readings for different RF transponders from a data stream of received response signals.
- the reader device extracts the response signals of the same frequency from the data stream.
- the readings of each RF transponder occurs in a short time period (e.g., 0.1 seconds) such that, when the time period expires, the data is processed.
- Some of the RF transponders may not communicate back to the reader device due to potential body or object occlusion.
- measurements may be inputted to a process or function that generates an estimate of a value based on a series of sequential measurements such as, but not limited to, a Kalman filter.
- a sliding window process may generate numerous independent array readings (e.g., 30 ) per second.
- Non-limiting implementations of a system for tracking a shape were tested with various surface materials.
- RF transponders were attached to a 7 cm ⁇ 40 cm paper frame to evaluate performance in various multipath-rich settings.
- RF transponders were attached to 0.5 mm thick latex sheets (one 3 cm ⁇ 15 cm and the other 30 cm ⁇ 40 cm) to evaluate system performance under stretch and curvature.
- a 3 mm thick cotton fabric was used for another test to evaluate performance with day-to-day clothing. Further tests were performed on woven conductive fabrics to verify if the system operates on conductive soft surfaces that may be generally used to shield external electromagnetic signals.
- a non-limiting implementation of the system was evaluated in a variety of multipath rich settings in an 8.5 m ⁇ 2 m indoor space with the RF transponder(s) and reader device separated by up to 3 m.
- the absolute error in the position of RF transponders measured in mm on the surface of the material, was used as a performance metric.
- a camera-based fiducial tracking system was implemented using ARToolKit and fiducial markers were affixed to the surfaces.
- the ARToolkit calibration program indicates a sub-millimeter baseline accuracy. Based on the fiducial marker tracking results, a surface interpolation was performed to reconstruct the surface's shape and identify ground truth positions for the RF transponders.
- a number of RF transponders is chosen for a surface based on various considerations including, for example, accuracy, inter-transponder signal coupling, processing requirements, type of material, size of surface, and/or the like. Although an increased number of transponders provides more measurements to use in the calculations, closely proximate transponders may cause inter-transponder signal coupling.
- FIGS. 8 A-D testing data is shown for the above-described tests. As shown in FIG. 8 A , all three configurations resulted in an average error between 1.3 and 1.9 cm. Among the various configurations, the spacing of 3 cm resulted in a slightly higher accuracy. However, the remaining configurations only experienced a minor reduction in accuracy relative to this configuration. Based on the tested transponder positions, a spacing of 4 cm with 10 transponders on the string was found to be sufficient to compute the unknown variables that characterize the shape of each surface.
- the multiple paths of the response signals from the RF transponders to the reader device may be disambiguated.
- tests were performed on non-limiting embodiments of a system for tracking shapes.
- the environments in which the tests were conducted are categorized as three groups: direct line-of-sight (e.g., with minimal obstructions in the environment), multipath-rich settings (e.g., with presence of large metallic and other reflectors in the environment), and non-line-of-sight (e.g., RF signals traverse only through reflected paths to the tags).
- a 1-D string of RF transponders with a spacing of 3 cm was used for the test.
- FIGS. 8 C and 8 D depict the error in transponder position across the different multipath environments.
- a mean error of 1.13 cm, 1.77 cm, and 2.30 cm was observed across the three classes of environments: line-of-sight, multipath, and non-line-of-sight, respectively. While the accuracy diminished by a measurable amount in the absence of direct signal paths, the tested non-limiting implementation remained largely robust to the multipath environments due to the algorithms described herein.
- Tests were also performed on the effect of material flexibility.
- non-limiting implementations of a system for tracking a shape were tested under two different types of stress related to flexibility: stretch and bend.
- Four objects were created to accommodate stretch and bend using different types of materials: a latex rubber string (30 cm), a latex rubber surface (20 cm ⁇ 20 cm), a cotton string (30 cm), and a cotton surface (30 cm ⁇ 40 cm).
- a latex rubber string (30 cm)
- a latex rubber surface (20 cm ⁇ 20 cm) a cotton string (30 cm)
- a cotton surface (30 cm ⁇ 40 cm.
- the objects were stretched and placed horizontally on a frame and weights were attached. Convex, concave, and wave shapes were tested in each configuration.
- the shape deformation ground truth was captured by the fiducial tracking system in real-time to capture changes in shape
- FIG. 9 shows results for different types of material according to the above-described tests.
- An error between 0.5 cm and 2.6 cm is observed across different materials and different deformations, indicating that the system is operable across various types of materials despite the presence or absence of stretch.
- the accuracy under both stretch and bend is less than the accuracy under only stretch, primarily due to the more complex shapes that result from both deformations. More iterations may be necessary to converge such complex shapes in the natural selection algorithm described herein to optimize the goodness-of-fit metric.
- Table 1 shows an overview of the absolute mean errors in different configurations:
- the non-limiting implementations tested are able to determine the transponder positions at a sub-centimeter accuracy and determine a fine-grained shape at 1 cm for a 0.4 m string and 2 cm for a 0.3 ⁇ 0.4 m surface.
- the surface may be a 3-D depressible (e.g., soft) touch screen or interactive surface.
- an interactive surface 1001 is shown having a front side 1003 and a back side 1004 .
- the front side 1003 shows the surface 1001 divided into ten regions 1002 a - j for differentiating interactions.
- the back side 1004 shows 35 RF transponders 1005 arranged over the surface.
- a user may place his or her hand on the front side 1003 of the surface 1001 , depressing a region of the surface overlapping region 1002 g .
- a single detected depression e.g., change in shape
- FIG. 11 a confusion matrix is shown from touch-based experiments performed on the front side 1003 of the surface 1001 shown in FIG. 10 A .
- the confusion matrix shows that 500 touch-point samples were measured with a 87.3% mean accuracy.
- the system for tracking a shape may be used to track the curvature of a spine (e.g., a posture).
- a user may wear clothing with RF transponders aligned on the user's back.
- a reader device may be mobile and carried with the user.
- a 1-D array of RF transponders having 4 cm spacing between each transponder may be positioned along a person's spine to measure the person's posture.
- an ARToolkit camera-based system was used to develop a baseline. Tests showed a mean error of 17 mm in the positions of the RF transponders.
- the system for tracking a shape may be used to track a user's breathing.
- a 1-D array of RF transponders may be arranged along a user's chest.
- a user may then self-report a breathing rate for a predetermined time period (e.g., 30 seconds) for a predetermined number of times (e.g., 12) after different activities of different intensities.
- FIG. 12 shows test results from tests performed with a non-limiting implementation of tracking a user's breathing in which a 95% accuracy was observed.
- the system for tracking a shape may be used to track a shape of a floor surface (e.g., wood, carpet, mat) and/or furniture (e.g., a chair, couch, mattress, etc.). Such applications may be used to determine the presence, location, and impact (e.g., weight, speed, etc.) of one or more objects or persons.
- the system for tracking a shape may be used to track a shape of architectural structures, such as bridges, roads, buildings, and/or the like. Such applications may be used to determine the sag, vibration, stress, load, and/or other like features of a structure. It will be further appreciated that non-limiting embodiments of a system for tracking shapes may be used in various other contexts and applications where it is desirable to track the shape of any object or person over any period of time.
- the system for tracking a shape may be combined with a system for tracking body movement.
- the non-limiting embodiments described herein may be used in combination with the systems and methods described in International Patent Application No. PCT/US18/64470, titled System and Method for Tracking a Body and filed on Dec. 7, 2018, the entirety of which is hereby incorporated by reference.
Abstract
Description
where 0≤t≤1 and Ci are the n+1 control points which define the set of parameters p that define the curve. A Bézier curve will always start with the first control point and end with the last control point, while the remaining control points define the tangents of the curve.
where each underlying curve is given by the following formula:
S(p={C i ,C j },u,v)=B i(u)B j(v)C i ,C j
where D is the prior known spacing between transponders and di is the surface distance between tag i and i+1.
E total =k e E elastic +k s E stiff +k g E gravity
where (n, αi, βi) are the polar coordinates of the RF transponders, λ is the signal wavelength, j is the square root of −1, En are the noise eigenvectors of hobs h† obs, hobs represents the vector of observed wireless channels across the RF transponders (e.g., at
steps), where N is the number of initial surface models in the first generation (e.g., the first plurality of surface models) generated during the first iteration of
TABLE 1 | ||||
Bend | Stretch | Both | ||
String | 13 mm | 5.3 mm | 17 mm | ||
Surface | 19 mm | 8.7 |
24 mm | ||
Claims (20)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US17/050,010 US11796304B2 (en) | 2018-04-24 | 2019-04-24 | System and method for tracking a shape |
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201862762210P | 2018-04-24 | 2018-04-24 | |
US17/050,010 US11796304B2 (en) | 2018-04-24 | 2019-04-24 | System and method for tracking a shape |
PCT/US2019/028873 WO2019209920A1 (en) | 2018-04-24 | 2019-04-24 | System and method for tracking a shape |
Publications (2)
Publication Number | Publication Date |
---|---|
US20210180937A1 US20210180937A1 (en) | 2021-06-17 |
US11796304B2 true US11796304B2 (en) | 2023-10-24 |
Family
ID=68295745
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/050,010 Active 2040-12-11 US11796304B2 (en) | 2018-04-24 | 2019-04-24 | System and method for tracking a shape |
Country Status (2)
Country | Link |
---|---|
US (1) | US11796304B2 (en) |
WO (1) | WO2019209920A1 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20220205853A1 (en) * | 2020-12-24 | 2022-06-30 | Yokogawa Electric Corporation | Force detector and force detection system |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11500086B2 (en) * | 2020-09-28 | 2022-11-15 | Mitsubishi Electric Research Laboratories, Inc. | System and method for tracking a deformation |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20140327676A1 (en) | 2008-07-23 | 2014-11-06 | Disney Enterprises, Inc. | View Point Representation for 3-D Scenes |
US20150025788A1 (en) | 2011-12-20 | 2015-01-22 | Sadar 3D, Inc. | Systems, apparatus, and methods for acquisition and use of image data |
US20150371104A1 (en) | 2013-02-08 | 2015-12-24 | Chuck Fung | Method, system and processor for instantly recognizing and positioning an object |
US20170188980A1 (en) * | 2016-01-06 | 2017-07-06 | Empire Technology Development Llc | Wearable sensor based body modeling |
US20180192919A1 (en) * | 2017-01-06 | 2018-07-12 | Panasonic Intellectual Property Management Co., Ltd. | Sensor and method |
US20190079176A1 (en) * | 2017-09-14 | 2019-03-14 | Qualcomm Incorporated | Extended localization range and asset tracking |
WO2019113441A1 (en) | 2017-12-08 | 2019-06-13 | Carnegie Mellon University | System and method for tracking a body |
-
2019
- 2019-04-24 WO PCT/US2019/028873 patent/WO2019209920A1/en active Application Filing
- 2019-04-24 US US17/050,010 patent/US11796304B2/en active Active
Patent Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20140327676A1 (en) | 2008-07-23 | 2014-11-06 | Disney Enterprises, Inc. | View Point Representation for 3-D Scenes |
US20150025788A1 (en) | 2011-12-20 | 2015-01-22 | Sadar 3D, Inc. | Systems, apparatus, and methods for acquisition and use of image data |
US20150371104A1 (en) | 2013-02-08 | 2015-12-24 | Chuck Fung | Method, system and processor for instantly recognizing and positioning an object |
US20170188980A1 (en) * | 2016-01-06 | 2017-07-06 | Empire Technology Development Llc | Wearable sensor based body modeling |
US20180192919A1 (en) * | 2017-01-06 | 2018-07-12 | Panasonic Intellectual Property Management Co., Ltd. | Sensor and method |
US20190079176A1 (en) * | 2017-09-14 | 2019-03-14 | Qualcomm Incorporated | Extended localization range and asset tracking |
WO2019113441A1 (en) | 2017-12-08 | 2019-06-13 | Carnegie Mellon University | System and method for tracking a body |
US20200375497A1 (en) * | 2017-12-08 | 2020-12-03 | Carnegie Mellon University | System and Method for Tracking a Body |
Non-Patent Citations (41)
Title |
---|
"Bezier Curve", Wikipedia, pp. 1-9, http://web.archive.org/web/20170313055951/https://en.wikipedia.org/wiki/Bezier_curve, Mar. 13, 2017. |
"Futures of the Scientific Imagination", National Science Foundation, 2017, pp. 1-6, https://www.nsf.gov/news/special_reports/futures/. |
"Genetic algorithm", Wikipedia, pp. 1-10, https://web.archive.org/web/20171211190316/https://en.wikipedia.org/wiki/Genetic_algorithm, Dec. 11, 2017. |
"Motion Capture Systems by OptiTrack", Natural Point, 2011, pp. 1-2, https://web.archive.org/web/20140208010400/http://www.naturalpoint.com/optitrack/. |
"SensorMania", Mediamatic, pp. 1-5, https://www.mediamatic.net/en/page/27491/sensormania, Jul. 7, 2011. |
"The world's most widely used tracking library for augmented reality.", Artoolkit, pp. 1-4, https://web.archive.org/web/20171219083739/https://archive.artoolkit.org/, Dec. 19, 2017. |
Dementyev et al., "SensorTape: Modular and Programmable 3D-Aware Dense Sensor Network on a Tape", In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology, Nov. 2015, pp. 1-11, https://www.researchgate.net/publication/301462154_SensorTape_Modular_and_Programmable_3D-Aware_Dense_Sensor_Network_on_a_Tape. |
Ding et al., "FEMO: A Platform for Free-weight Exercise Monitoring with RFIDs", In Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems, Nov. 1-4, 2015, pp. 141-154. |
Hing N. Ng et al., "Computer Graphics Techniques for Modeling Cloth", Computer Graphics and Applications, 1996, pp. 28-41, vol. 16:5, IEEE. |
Hou et al., "TagBreathe: Monitor Breathing with Commodity RFID Systems", In Distributed Computing Systems (ICDCS), 2017, pp. 1-10, IEEE. |
Izadi et al., "KinectFusion: Real-time 3D Reconstruction and Interaction Using a Moving Depth Camera*", In proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, Oct. 16-19, 2011, pp. 1-10. |
Jin et al., "Towards Wearable Everyday Body-Frame Tracking using Passive RFIDs", In proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Dec. 2017, pp. 1-23, vol. 1: 4, Article 145. |
Karrer et al., "Pinstripe: Eyes-free Continuous Input on Interactive Clothing", In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, May 7-12, 2011, pp. 1313-1322. |
Lelthinger et al., "Jamming User Interfaces: Programmable particle stiffness and sensing for malleable and shape-changing devices", In Proceedings of the 25th annual ACM symposium on User interface software and technology, Oct. 2012, pp. 1-11, https://www.researchgate.net/publication/262399679_Jamming_user_interfaces_Programmable_particle_stiffness_and_sensing_for_malleable_and_shape-changing_devices. |
Li et al., "ID-Match: A Hybrid Computer Vision and RFID System for Recognizing Individuals in Groups", 2016 CHI Conference, May 7-12, 2016, pp. 4933-4944. |
Li et al., "IDSense: A Human Object Interaction Detection System Based on Passive UHF RFID", In proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, Apr. 18-23, 2015, pp. 1-10. |
Ma et al., "Minding the Billions: Ultra-wideband Localization for Deployed RFID Tags", MobiCom '17, Oct. 16-20, 2017, pp. 1-13. |
Millot et al., "An UWB Through-The-Wall Radar with 3D Imaging, Detection and Tracking Capabilities", European Radar Conference (EuRAD), Sep. 2015, pp. 1-4. |
Moeslund et al., "A Survey of Computer Vision-Based Human Motion Capture", Computer Vision and Image Understanding, 2001, pp. 231-268, vol. 18. |
Neumann et al., "Lidar Imaging of Topography with Millimeter Ranging Precision for Proximity Science and Operations from Rovers or Spacecraft", pp. 1-2, 2017. |
Ni et al., "LANDMARC: Indoor Location Sensing Using Active RFID*", pp. 1-10, 2004. |
Occhiuzzi et al., "Modeling, Design and Experimentation of Wearable RFID Sensor Tag", IEEE Transactions on Antennas and Propagation, Aug. 2010, pp. 2490-2498, vol. 58:8. |
Parzer et al., "SmartSleeve: Real-time Sensing of Surface and Deformation Gestures on Flexible, Interactive Textiles, using a Hybrid Gesture Detection Pipeline", In proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology, Oct. 22-25, 2017, pp. 1-13. |
Poupyrev et al., "Project Jacquard: Interactive Digital Textiles at Scale", In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, May 7-12, 2016, pp. 4216-4227. |
Rendl et al., "FlexSense: A Transparent Self-Sensing Deformable Surface", In Proceedings of the 27th annual ACM symposium on User interface software and technology, Oct. 2014, pp. 1-11, https://www.researchgate.net/publication/266477992_FlexSense_A_Transparent_Self-Sensing_Deformable_Surface. |
Schneegass et al., "GestureSleeve: Using Touch Sensitive Fabrics for Gestural Input on the Forearm for Controlling Smartwatches", In proceedings of the ACM International Symposium on Wearable Computers, Sep. 12-16, 2016, pp. 108-115. |
Shangguan et al., "Enabling Gesture-based Interaction with Objects", In proceedings of the 15th Annual International Conference on Mobile Systems, Applications and Services, Jun. 19-23, 2017, pp. 239-251. |
Smiley, "RFID Readers For Mobile Phones From TSL", RFID Insider, Aug. 15, 2014, pp. 1-3, https://web.archive.org/web/20141107203441/http://blog.atlasrfidstore.com:80/rfi d-readers-mobile-phones-tsl. |
Terzopoulos et al., "Elastically Deformable Models", ACM Computer Graphics, Jul. 1987, pp. 205-214, vol. 21:4. |
Walter et al., "Approximate Arc Length Parameterization", In proceedings of the 9th Brazilian Symposium on Computer Graphics and Image Processing, 1996, pp. 143-150. |
Wang et al., "Arc-length Parameterized Spline curves for real-time simulation", In proceedings of the 5th International Conference on Curves and Surfaces, 2002, pp. 387-396. |
Wang et al., "Fast Energy-Based Surface Wrinkle Modeling", Computers and Graphics pp. 1-24, 2006. |
Wang et al., "RF-IDraw: Virtual Touch Screen in the Air Using RF Signals", In proceedings of the ACM SIGCOMM Computer Communication Review, Aug. 17-22, 2014, pp. 1-12. |
Wei et al., "Gyro in the Air: Tracking 3D Orientation of Batteryless Internet-of-Things", In proceedings of the 22nd Annual International Conference on Mobile Computing and Networking, Oct. 3-7, 2016, pp. 1-14. |
Xiong et al., "ArrayTrack: A Fine-Grained Indoor Location System", In proceedings of the 10th USENIX Symposium on Networked Systems Design and Implementation, 2013, pp. 71-84. |
Yang et al., "Tagoram: Real-Time Tracking of Mobile RFID Tags to High Precision Using COTS Devices", In proceedings of the 20th Annual International Conference on Mobile Computing and Networking, Sep. 7-11, 2014, pp. 1-12. |
Yao et al., "biologic: Natto Cells as Nanoactuators for Shape Changing Interfaces", In proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, Apr. 18-23, 2015, pp. 1-10. |
Yao et al., "PneUI: Pneumatically Actuated Soft Composite Materials for Shape Changing Interfaces", In proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology, Oct. 8-11, 2013, pp. 1-10. |
Zhang et al., "Bridge-Deflection Estimation through Inclinometer Data Considering Structural Damages", Journal of Bridge Engineering, Oct. 2016, pp. 1-11. |
Zhang et al., "Electrick: Low-Cost Touch Sensing Using Electric Field Tomography", In proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, May 6-11, 2017, pp. 1-14. |
Zhu et al., "Object Recognition and Navigation using a Single Networking Device", Jun. 19-23, 2017, pp. 1-13. |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20220205853A1 (en) * | 2020-12-24 | 2022-06-30 | Yokogawa Electric Corporation | Force detector and force detection system |
Also Published As
Publication number | Publication date |
---|---|
US20210180937A1 (en) | 2021-06-17 |
WO2019209920A9 (en) | 2020-01-30 |
WO2019209920A1 (en) | 2019-10-31 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Jin et al. | Wish: Towards a wireless shape-aware world using passive rfids | |
CN108279413B (en) | Sensor and biological action estimation method | |
Yang et al. | RFID-pose: Vision-aided three-dimensional human pose estimation with radio-frequency identification | |
Shangguan et al. | Enabling gesture-based interactions with objects | |
Ding et al. | Trio: Utilizing tag interference for refined localization of passive RFID | |
CN107923960B (en) | System and method for locating tags in space | |
US11826139B2 (en) | System and method for tracking a body | |
CN107576957B (en) | Sensor and estimation method | |
US11796304B2 (en) | System and method for tracking a shape | |
JP6868846B2 (en) | Sensors and methods | |
CN105992514B (en) | Method for locating an animal | |
Wagner et al. | Passive RFID tomographic imaging for device-free user localization | |
WO2014203041A1 (en) | System and method for rfid indoor localization | |
Faulkner et al. | CapLoc: Capacitive sensing floor for device-free localization and fall detection | |
Shi et al. | mPose: Environment-and subject-agnostic 3D skeleton posture reconstruction leveraging a single mmWave device | |
Acharya et al. | Ensemble learning approach via kalman filtering for a passive wearable respiratory monitor | |
CN106774901A (en) | Remote PC body-sensings input method based on localization by ultrasonic | |
D’Avella et al. | RFID Gazebo-based simulator with RSSI and phase signals for UHF tags localization and tracking | |
Wang et al. | HMO: Ordering RFID tags with static devices in mobile environments | |
Wang et al. | Active RFID based indoor localization | |
Wagner et al. | Approaches for device-free multi-user localization with passive RFID | |
Tajin et al. | UHF RFID channel emulation testbed for wireless IoT systems | |
Sun | Rfitness: Enabling smart yoga mat for fitness posture detection with commodity passive rfids | |
Chawla et al. | Efficient RFID-based mobile object localization | |
JP2020524261A (en) | 6 DOF tracking of objects using sensors |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: CARNEGIE MELLON UNIVERSITY, PENNSYLVANIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:JIN, HAOJIAN;WANG, JINGXIAN;KUMAR, SWARUN KUMAR SURESH;AND OTHERS;SIGNING DATES FROM 20190426 TO 20190430;REEL/FRAME:054146/0762 |
|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY |
|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO SMALL (ORIGINAL EVENT CODE: SMAL); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: APPLICATION DISPATCHED FROM PREEXAM, NOT YET DOCKETED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
AS | Assignment |
Owner name: NATIONAL SCIENCE FOUNDATION, VIRGINIA Free format text: CONFIRMATORY LICENSE;ASSIGNOR:CARNEGIE-MELLON UNIVERSITY;REEL/FRAME:059267/0555 Effective date: 20210518 |
|
AS | Assignment |
Owner name: NATIONAL SCIENCE FOUNDATION, VIRGINIA Free format text: CONFIRMATORY LICENSE;ASSIGNOR:CARNEGIE-MELLON UNIVERSITY;REEL/FRAME:059728/0139 Effective date: 20210518 Owner name: NATIONAL SCIENCE FOUNDATION, VIRGINIA Free format text: CONFIRMATORY LICENSE;ASSIGNOR:CARNEGIE-MELLON UNIVERSITY;REEL/FRAME:059728/0134 Effective date: 20210518 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |