US11341867B2 - Movement tracking and simulation device and method - Google Patents

Movement tracking and simulation device and method Download PDF

Info

Publication number
US11341867B2
US11341867B2 US16/083,628 US201716083628A US11341867B2 US 11341867 B2 US11341867 B2 US 11341867B2 US 201716083628 A US201716083628 A US 201716083628A US 11341867 B2 US11341867 B2 US 11341867B2
Authority
US
United States
Prior art keywords
imu
data
processors
images
hand piece
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US16/083,628
Other languages
English (en)
Other versions
US20190073924A1 (en
Inventor
Wouter BIEMANS
Dyon BODE
Luuk STEITNER
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nissin Dental Products Inc
Original Assignee
Nissin Dental Products Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nissin Dental Products Inc filed Critical Nissin Dental Products Inc
Publication of US20190073924A1 publication Critical patent/US20190073924A1/en
Assigned to MOOG BV reassignment MOOG BV ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: STEITNER, Luuk, BODE, Dyon, BIEMANS, Wouter
Assigned to NISSIN DENTAL PRODUCTS INC. reassignment NISSIN DENTAL PRODUCTS INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: MOOG B.V.
Application granted granted Critical
Publication of US11341867B2 publication Critical patent/US11341867B2/en
Active legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/006—Mixed reality
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B23/00—Models for scientific, medical, or mathematical purposes, e.g. full-sized devices for demonstration purposes
    • G09B23/28—Models for scientific, medical, or mathematical purposes, e.g. full-sized devices for demonstration purposes for medicine
    • G09B23/283—Models for scientific, medical, or mathematical purposes, e.g. full-sized devices for demonstration purposes for medicine for dentistry or oral hygiene
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/014—Hand-worn input/output arrangements, e.g. data gloves
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0346—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/292—Multi-camera tracking
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/002—Specific input/output arrangements not covered by G06F3/01 - G06F3/16
    • G06F3/005—Input arrangements through a video camera
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30036—Dental; Teeth
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
    • H04N23/681—Motion detection
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/14—Picture signal circuitry for video frequency region
    • H04N5/144—Movement detection

Definitions

  • the present invention relates to tracking the position and orientation of a moving hand piece and outputting, to a display, an image which moves in accordance with the tracked position of the hand piece. More specifically, the invention relates to tracking the movement of a hand piece in a dental simulation environment to simulate use of a dental tool.
  • Machines to simulate dentistry techniques for training purposes are known and include, for example, the Simodont machine manufactured by the applicant. These machines implement virtual reality technology to allow a student to practice various dentistry procedures.
  • the simulation machines generally comprise a display screen which outputs 3D images for viewing by the user wearing 3D glasses.
  • Positioned below the display screen is at least one hand piece which is fixed to the machine by a mechanism comprising a series of linkages and electric motors which measure the relative position of the hand piece (as it is moved by the user) and the force applied by a user as the user conducts a dentistry operation (for example, drilling into a tooth) on a virtual 3D image of a tooth, set of teeth or jaw.
  • the hand piece simulates a dentist's drill and provide haptic feedback to the student as the student performs their drilling.
  • the purpose of the machine is to provide a simulation environment for trainee dentists to practice with before they are required to perform the same techniques on a real, rather than a virtual, patient, the more realistic the simulation processes can be, the more immersed the student is in their training and the better the outcome, for both student and future patient.
  • a common tool used in dentistry is a small mirror on the end of a handle which is used by a dentist to view teeth and gums and other areas of the mouth which are not clearly visible from the dentist's angle of view.
  • a dentist's mouth mirror is simulated by a hand piece (which, unlike the hand piece to simulate a dentist's drill, does not provide haptic feedback) which is connected to the machine by a series of linkages.
  • the mirror hand piece is moved, the extent and angle of movement of the each linkages is measured and the relative position of the mirror hand piece is calculated.
  • the virtual image of the dentist's mouth mirror on the display is moved accordingly.
  • the linkages limit the range of motion.
  • the number of linkages required to provide adequate range of motion to the student compromises position accuracy.
  • the weight of the linkages can be felt by a user (thereby lessening the ‘realness’ of the simulation) and the linkages are subject to wear and breakage through over-use or misuse.
  • an apparatus for dental simulation according to appended claim 1 .
  • the apparatus allows for accurate movement tracking of a hand held object and to accurately translate the tracked movement into animation of a dentistry tool in a dental simulation environment.
  • a hand piece according to appended claim 15 . Movement of the hand piece can be tracked by a medical simulation machine by virtue of a light source and means to measure orientation of the hand piece.
  • a computer implemented method according to appended claim 25 .
  • the method enables the accurate and efficient generation of instructions to replicate movement of a physical object to movement of a virtual object displayed on a viewing screen.
  • FIG. 1 is a perspective view of a dental simulation machine
  • FIG. 2 is a cut-away perspective view of a dental simulation machine
  • FIG. 3 is a schematic diagram of a mirror tracking device according to an embodiment of the present invention.
  • FIG. 4 a is a perspective view of a portion of a dental simulation machine according to an embodiment of the invention.
  • FIG. 4 b is a profile view of the portion of a dental simulation machine shown in FIG. 4 a;
  • FIG. 5 is a schematic diagram of the main hardware components of a tracking system in accordance with an embodiment of the present invention.
  • a dental simulation machine is shown generally at FIG. 1 .
  • the machine 100 is used by students of dentistry to practice dentistry techniques using augmented reality techniques.
  • a student sits on a chair (not shown) facing viewing screen 101 .
  • Hand rest 102 defines, generally, an area in which the student simulates dentistry techniques.
  • Power button 105 provides machine on/off functionality and height adjustment switch 106 allows the user to adjust the height of a position of the machine 100 , including hand rest 102 and viewing screen 101 .
  • Viewing screen 101 displays virtual 3D moving images whose movements correspond with movement of a hand piece (which generally resides within area of hand rest 101 ) by a student.
  • the student views the images on viewing screen 101 wearing passive 3D glasses.
  • Mouse 104 allows the user of the machine to adjust the relative position and orientation of the images on viewing screen 101 in a virtual three dimensional space
  • Foot pedal 107 facilitates control of operation of a simulated drill (or other powered dentistry tool).
  • the machine 100 also comprises training screen 103 (which may be a touch sensitive screen) which is used by the student to access information relevant to their training, such as training programs, individual lessons, scoring and marking data, mentor comments, and to review previous training material. While a simulation process is being performed, the images displayed on viewing screen 101 are also output to training screen 103 to allow an onlooker to view the student's use of the machine.
  • training screen 103 which may be a touch sensitive screen
  • FIG. 2 shows some internal components of a dental simulation machine where reference signs of like components in FIG. 1 are increased by 100.
  • the main processing capability of machine 200 is provided by central computer 213 .
  • Projectors 211 and optical box 210 provide the means to output 3D images onto display screen 201 .
  • Hand rest 102 of FIG. 1 is also shown in FIG. 2 .
  • Hand piece 208 (‘hand piece’ is generally used to mean any device or implement which can be held by hand) is used to simulate a dentist's drill.
  • Hand piece 208 comprises a handle which generally looks and feels like the handle of a dentist's drill. However, the distal end of hand piece 208 (i.e. where a burr would be located) is connected to haptic display 209 by a series of linkages.
  • the linkages and haptic display together comprise electric motors, a force sensor, an angular position sensor, and Inertial Measurement Unit (IMU) and potentiometers.
  • Operation of a dentist's drill is simulated by a continuous feedback mechanism, controlled by haptic control unit 212 , whereby the force applied and movement to the hand piece 208 by the user is fed to haptic control unit 212 which instructs appropriate operation of haptic display 209 to provide the required counter-force (to simulate interaction with a solid object), vibration (to simulate operation of a powered drilling tool) and restriction of movement in accordance with a set software routine (which may, for example, relate to a particular training session involving a particular dentistry operation).
  • a set software routine which may, for example, relate to a particular training session involving a particular dentistry operation.
  • FIG. 3 shows a mirror hand piece 320 which is used, alone or in conjunction with hand piece 208 , to simulate the use of a dentist's mouth mirror.
  • Mirror hand piece 320 comprises a housing 321 , the dimensions of which are chosen to generally match the dimensions of the handle of a standard dentist's mouth mirror. Accordingly, housing 321 is preferably less than 12 mm in diameter, and is preferably less than or equal to 100 mm in length. The material from which housing 321 is made is chosen to allow external detection of light emitted by a light source/emitter 322 (where ‘light’ refers to any electromagnetic radiation). Preferably, the light emitter 322 is an infrared light emitting diode (LED). Light emitter 322 acts as an active infrared marker, as will be described further below. Housing 321 may be any suitable material such as metal, but is preferably a roughened plastic, which is also suitably ergonomic. The weight of mirror hand piece 320 is less than 30 grams and is intended to be gripped by a user's thumb and forefinger at channel 325 .
  • LED infrared light emitting diode
  • Proximate end 323 of mirror hand piece 320 is connected to a camera PCB via a cable (not shown in FIG. 3 ).
  • hand piece 320 comprises a battery and transmitter and communicates with camera PCB 435 wirelessly.
  • Distal end 322 of housing 321 contains the infrared LED. The position of the LED is optically tracked (described further below) and is mapped to a virtual image of a mirror portion of a dentist's mouth mirror which is displayed, during a simulation exercise, on viewing screen 101 .
  • Housing 321 of mirror hand piece 320 further comprises a small PCB which includes an Inertial Measurement Unit (IMU) 324 .
  • IMU Inertial Measurement Unit
  • An IMU is a component which is widely used is various devices (including gaming consoles and mobile phones) and comprises gyroscopes and accelerometers to measure position and orientation in three dimensions. For each translational degree of freedom, standard IMUs integrate acceleration data to determine position values. For each rotational degree of freedom, angular velocity data is integrated to provide angular position data. IMUs may also include magnetometers to provide orientation data relative to the Earth's magnetic field. The magnetometer orientation data is used to re-calibrate the angular position data provided by the gyroscopes to compensate for inaccuracy caused by the accumulation of small integration errors. Detail regarding the operational principles of IMUs will be accessible to those skilled in the art.
  • FIGS. 4 a and 4 b The general space within which movement of the mirror hand piece 320 is tracked is shown in FIGS. 4 a and 4 b as area 430 .
  • Area 430 is located above hand rest 435 and generally defines the area in which a student operates one or more mirror hand piece 320 and/or hand piece 208 .
  • Two upper cameras 431 and 432 are fixed to optical box 210 . Their field of view is directed downwards to the centre 436 of area 430 .
  • Two lower cameras 433 and 434 are located below hand rest 435 , as can be seen from FIG. 4 b , and are fixed to a structure which extends laterally from the main housing of machine 100 on which the hand rest 435 is located. In an alternative embodiment, all cameras are fixed to a structure that can be secured to hand rest 435 , such that the structure can be attached and detached from the dental simulation machine 100 .
  • Lower cameras 433 and 434 are directed upwards to centre 436 of tracking area 430 .
  • Tracking area 430 is approximately 250 mm ⁇ 200 mm ⁇ 152 mm, as generally defined by the combined fields of view of cameras 431 , 432 , 433 and 434 .
  • Optical tracking avoids interference with operation of electric motors of the haptic display 209 .
  • an LED is integrated into the hand rest 435 to allow the user to view the position of the mirror hand piece 324 relative to hand rest 435 on viewing screen 201 .
  • Cameras 431 , 432 , 433 and 434 are infrared cameras which detect the infrared emitted by LED 322 and capture images at 25 frames per second.
  • the image feed comprising a series of images captured by each camera while operational, is transmitted to a processor on camera PCB 437 .
  • a processor on PCB 437 For each image, a processor on PCB 437 performs a line scan of each line of pixels of a captured image to identify pixels whose grayscale brightness/intensity values is above a predefined threshold value.
  • the predefined threshold value defines a range of intensity values (between a minimum intensity value of the threshold value and an unspecified maximum value) which are deemed to contain relevant information. Pixel data relating to pixels having intensity values below the threshold are disregarded.
  • pixel data relating to pixels having intensity values above the threshold is analysed by a processor of PCB 437 to identify clusters of pixels.
  • the largest cluster of pixels is identified and it is the centre of the largest cluster which identifies the position of LED 322 within the image.
  • the processor of PCB 437 adopts a ‘blob tracking’ technique and does not further analyse pixels after determining that their brightness value is below the threshold value, only a portion of the entire image is processed further. As such, the processing resource required to identify the LED in 2D is less than if every pixel of the image required analysis.
  • a processor of PCB 437 determines, for each image from each camera, the x and y coordinates of the centre of the largest cluster of pixels and so generates, in real-time, a series of 2D position coordinates for each camera feed. Four series of 2D coordinates are sent to main computer 213 .
  • camera PCB 437 In addition to receiving images from each camera, camera PCB 437 also receives data from IMU 324 in mirror hand piece 320 via a serial protocol. Along with the series of x, y coordinates for each camera feed, camera PCB passes position and orientation data from IMU 324 to main computer 213 of machine 100 .
  • Main computer 213 processes data received from camera PCB 437 using a dynamic loading mechanism, the principles of which will be accessible to a person skilled in the art.
  • computer 213 uses a set of four x, y coordinate pairs (each pair corresponding to LED coordinates from respective cameras), computer 213 performs, in real time and within a dynamic loading library, a triangulation operation to generate x, y, z coordinates which specify the position of LED 322 within area 430 .
  • the rate of generation of x, y, z coordinates is the same as the camera frame rate.
  • the accuracy of the x, y, z data is improved, and the processing resource to perform the triangulation increased, with increasing numbers of cameras. It will be understood that a minimum of two cameras is required to provide sufficient data to perform triangulation and generate 3 dimensional position data.
  • the IMU 234 generates position and orientation data 100 times per second and so the main computer 213 receives update position and orientation data at the rate. It is the position data from the IMU (providing position data more frequently than the camera PCB) which is the primary source of position data for animating a virtual image of a dentist's mouth mirror on viewing screen 201 .
  • the IMU position data received by the main computer 213 may be inaccurate.
  • the x, y, z position data using the x, y coordinates from the camera data is used to correct and re-calibrate the IMU position data.
  • Data relating to the orientation and rotation of mirror hand piece 320 is also provided by the IMU, and is also received by the main computer 213 with the IMU position data.
  • the orientation and rotational data provide a complete description, in real time, of the movement of the hand piece 320 in three dimensions.
  • the absolute position accuracy of the position data of the mirror hand piece 320 is less than or equal to 2 mm and the absolute orientation accuracy of the orientation data of the mirror hand piece 320 is less than or equal to 1 degree.
  • the relative position resolution is less than 0.15 mm and the relative orientation resolution is less than 0.1 degree.
  • the total system latency is less than 30 ms.
  • Main computer 213 generates instructions based on the position and orientation data of the tracked mirror hand piece and instructs a simulation program to adjust the relative position and orientation of a virtual image of a mouth mirror on viewing screen 201 in accordance with the determined position and orientation data of the mirror hand piece.
  • the virtual image of a dentist's mouth mirror is animated in 3 dimensions accordingly, and the student is able to view, using the mirror, a virtual three dimensional image of one or more teeth from different angles.
  • the image represented on the mirror face of the virtual mirror must be rendered four times, rather than two (for standard stereoscopic viewing).

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Health & Medical Sciences (AREA)
  • Algebra (AREA)
  • Mathematical Physics (AREA)
  • Public Health (AREA)
  • Chemical & Material Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Medicinal Chemistry (AREA)
  • Epidemiology (AREA)
  • Computational Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Pure & Applied Mathematics (AREA)
  • Business, Economics & Management (AREA)
  • Educational Administration (AREA)
  • Educational Technology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • Software Systems (AREA)
  • Multimedia (AREA)
  • Dental Tools And Instruments Or Auxiliary Dental Instruments (AREA)
US16/083,628 2016-03-10 2017-03-07 Movement tracking and simulation device and method Active 2039-02-16 US11341867B2 (en)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
GB1604115 2016-03-10
GB1604115.4 2016-03-10
GB1604115.4A GB2548341A (en) 2016-03-10 2016-03-10 Movement tracking and simulation device and method
PCT/EP2017/055339 WO2017153416A1 (en) 2016-03-10 2017-03-07 Movement tracking and simulation device and method

Publications (2)

Publication Number Publication Date
US20190073924A1 US20190073924A1 (en) 2019-03-07
US11341867B2 true US11341867B2 (en) 2022-05-24

Family

ID=55952135

Family Applications (1)

Application Number Title Priority Date Filing Date
US16/083,628 Active 2039-02-16 US11341867B2 (en) 2016-03-10 2017-03-07 Movement tracking and simulation device and method

Country Status (9)

Country Link
US (1) US11341867B2 (ko)
EP (1) EP3427245A1 (ko)
JP (1) JP6789310B2 (ko)
KR (1) KR102418773B1 (ko)
CN (1) CN108780614B (ko)
AU (1) AU2017230317B2 (ko)
CA (1) CA3016853A1 (ko)
GB (1) GB2548341A (ko)
WO (1) WO2017153416A1 (ko)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10839547B2 (en) * 2017-09-28 2020-11-17 Samsung Electronics Co., Ltd. Camera pose determination and tracking
US11367192B2 (en) 2019-03-08 2022-06-21 Align Technology, Inc. Foreign object filtering for intraoral scanning
JP6936826B2 (ja) * 2019-03-18 2021-09-22 株式会社モリタ製作所 画像処理装置、表示システム、画像処理方法、および画像処理プログラム
CN114206252A (zh) * 2019-08-09 2022-03-18 斯密托利弗私人有限公司 用于模拟牙科手术的设备和方法
CN110827975B (zh) * 2019-11-11 2022-10-14 延边修恩生物科技有限公司 一种口腔扫描装置及其使用方法
CN112349167A (zh) * 2020-11-05 2021-02-09 北京众绘虚拟现实技术研究院有限公司 一种双手力反馈的口腔实践技能考试设备

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020089544A1 (en) * 2000-08-22 2002-07-11 Dirk Jahn System and method for combined use of different display/appliance types with system-controlled, context-dependent information display
JP2002296019A (ja) 2001-03-30 2002-10-09 Iwate Prefecture ３次元形状計測システム
US20060076917A1 (en) 2004-10-12 2006-04-13 Samsung Gwangju Electronics Co., Ltd. Method compensating gyro sensor for robot cleaner
JP2009192098A (ja) 2008-02-12 2009-08-27 Nippon Hoso Kyokai <Nhk> 物体軌道演算装置及びそのプログラム
JP2009294326A (ja) 2008-06-03 2009-12-17 Eped Inc 歯科臨床と教育訓練シミュレーション追跡システム及びその評価方法
US20100167248A1 (en) * 2008-12-31 2010-07-01 Haptica Ltd. Tracking and training system for medical procedures
WO2012088285A2 (en) 2010-12-22 2012-06-28 Infinite Z, Inc. Three-dimensional tracking of a user control device in a volume
US8716973B1 (en) * 2011-02-28 2014-05-06 Moog Inc. Haptic user interface
US20140306888A1 (en) 2013-04-12 2014-10-16 Nintendo Co., Ltd. Storage medium having stored thereon information processing program, information processing system, information processing apparatus, and information processing execution method
US20140320667A1 (en) * 2010-08-31 2014-10-30 Cast Group Of Companies Inc. System and Method for Tracking
US20170071713A1 (en) 2014-05-28 2017-03-16 J. Morita Mfg. Corp. Root canal treating device

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060084050A1 (en) * 2004-01-12 2006-04-20 Haluck Randy S Portable virtual reality medical demonstration and training apparatus
US7780089B2 (en) * 2005-06-03 2010-08-24 Hand Held Products, Inc. Digital picture taking optical reader having hybrid monochrome and color image sensor array
GB2451461A (en) * 2007-07-28 2009-02-04 Naveen Chawla Camera based 3D user and wand tracking human-computer interaction system
US9119655B2 (en) * 2012-08-03 2015-09-01 Stryker Corporation Surgical manipulator capable of controlling a surgical instrument in multiple modes
CA2849212C (en) * 2011-04-28 2015-10-13 Technologies Holdings Corp. Vision system for robotic attacher
BR112015001369A2 (pt) * 2012-07-26 2017-07-04 Olive Medical Corp sistema de câmera com sensor de imagem cmos monolítico de área mínima
US9563266B2 (en) * 2012-09-27 2017-02-07 Immersivetouch, Inc. Haptic augmented and virtual reality system for simulation of surgical procedures
US9605950B2 (en) * 2013-05-22 2017-03-28 Cognex Corporation System and method for efficient surface measurement using a laser displacement sensor
ES2763912T3 (es) * 2013-08-20 2020-06-01 Curefab Tech Gmbh Seguimiento óptico
US9723300B2 (en) * 2014-03-17 2017-08-01 Spatial Intelligence Llc Stereoscopic display
EP2988288A1 (en) * 2014-08-22 2016-02-24 Moog B.V. Medical simulator handpiece
CN105069795B (zh) * 2015-08-12 2017-12-22 深圳锐取信息技术股份有限公司 运动对象跟踪方法及装置

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020089544A1 (en) * 2000-08-22 2002-07-11 Dirk Jahn System and method for combined use of different display/appliance types with system-controlled, context-dependent information display
JP2002296019A (ja) 2001-03-30 2002-10-09 Iwate Prefecture ３次元形状計測システム
US20060076917A1 (en) 2004-10-12 2006-04-13 Samsung Gwangju Electronics Co., Ltd. Method compensating gyro sensor for robot cleaner
JP2009192098A (ja) 2008-02-12 2009-08-27 Nippon Hoso Kyokai <Nhk> 物体軌道演算装置及びそのプログラム
JP2009294326A (ja) 2008-06-03 2009-12-17 Eped Inc 歯科臨床と教育訓練シミュレーション追跡システム及びその評価方法
US20100167248A1 (en) * 2008-12-31 2010-07-01 Haptica Ltd. Tracking and training system for medical procedures
US20140320667A1 (en) * 2010-08-31 2014-10-30 Cast Group Of Companies Inc. System and Method for Tracking
WO2012088285A2 (en) 2010-12-22 2012-06-28 Infinite Z, Inc. Three-dimensional tracking of a user control device in a volume
US8716973B1 (en) * 2011-02-28 2014-05-06 Moog Inc. Haptic user interface
US20140306888A1 (en) 2013-04-12 2014-10-16 Nintendo Co., Ltd. Storage medium having stored thereon information processing program, information processing system, information processing apparatus, and information processing execution method
US20170071713A1 (en) 2014-05-28 2017-03-16 J. Morita Mfg. Corp. Root canal treating device

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
Dec. 3, 2021, Office Action issued by the China National Intellectual Property Administration in the corresponding Chinese Patent Application No. 201780016289.8.
He Yu et al., Study on Imaging Tracking Simulation of the CCD, System Simulation Technology & Application, 2008, pp. 569-571, vol. 10.
May 7, 2021, Office Action issued by the China National Intellectual Property Administration in the corresponding Chinese Patent Application No. 201780016289.8 with English language concise statement of relevance.

Also Published As

Publication number Publication date
AU2017230317B2 (en) 2022-10-20
GB2548341A (en) 2017-09-20
US20190073924A1 (en) 2019-03-07
CN108780614A (zh) 2018-11-09
CA3016853A1 (en) 2017-09-14
CN108780614B (zh) 2022-05-31
GB201604115D0 (en) 2016-04-27
WO2017153416A1 (en) 2017-09-14
KR102418773B1 (ko) 2022-07-07
JP2019512299A (ja) 2019-05-16
EP3427245A1 (en) 2019-01-16
JP6789310B2 (ja) 2020-11-25
AU2017230317A1 (en) 2018-09-27
KR20180121605A (ko) 2018-11-07

Similar Documents

Publication Publication Date Title
US11341867B2 (en) Movement tracking and simulation device and method
US20210174706A1 (en) Injection site training system
KR20210064129A (ko) 가상현실 기반 방사선학 실습 장치 및 방법
US20200363867A1 (en) Blink-based calibration of an optical see-through head-mounted display
US20150004581A1 (en) Interactive physical therapy
CN105898241A (zh) 一种运动视觉呈现系统及方法
CN106535812A (zh) 带有基于定量三维成像的触觉反馈的手术系统
US11567564B2 (en) Interactive exercise and training system and method
TW200900123A (en) Self-contained inertial navigation system for interactive control using movable controllers
CN108572728A (zh) 信息处理设备、信息处理方法和程序
KR20160090042A (ko) 3d hmd를 활용한 아케이드 게임시스템
KR102062129B1 (ko) 치과 발치 훈련 시스템
RU2670649C1 (ru) Способ изготовления перчатки виртуальной реальности (варианты)
JP4760344B2 (ja) 仮想空間体験システム
US12011647B2 (en) Surface interactions in a virtual reality (VR) environment
RU2673406C1 (ru) Способ изготовления перчатки виртуальной реальности
US10657726B1 (en) Mixed reality system and method for determining spatial coordinates of dental instruments
Yang et al. Perceptual issues of a passive haptics feedback based MR system
CN110657796B (zh) 一种虚拟现实辅助定位装置和方法
Ekram Measurement of control parameters for omnidirectional treadmills using RGBD camera
WO2024171171A1 (en) Surgery simulation system and method
CN116869514A (zh) 步态测试的引导方法、装置、电子设备和可读存储介质

Legal Events

Date Code Title Description
FEPP Fee payment procedure

Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

AS Assignment

Owner name: MOOG BV, NETHERLANDS

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:BIEMANS, WOUTER;BODE, DYON;STEITNER, LUUK;SIGNING DATES FROM 20181001 TO 20181218;REEL/FRAME:048573/0527

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

AS Assignment

Owner name: NISSIN DENTAL PRODUCTS INC., JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MOOG B.V.;REEL/FRAME:053234/0061

Effective date: 20200604

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

STPP Information on status: patent application and granting procedure in general

Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED

STCF Information on status: patent grant

Free format text: PATENTED CASE