CN110609388A - Augmented reality processing method and system for ultrasonic image display - Google Patents
Augmented reality processing method and system for ultrasonic image display Download PDFInfo
- Publication number
- CN110609388A CN110609388A CN201910902733.7A CN201910902733A CN110609388A CN 110609388 A CN110609388 A CN 110609388A CN 201910902733 A CN201910902733 A CN 201910902733A CN 110609388 A CN110609388 A CN 110609388A
- Authority
- CN
- China
- Prior art keywords
- elastic plate
- augmented reality
- limiting
- fixed
- ultrasonic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
- G02B27/0176—Head mounted characterised by mechanical features
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
- A61B8/46—Ultrasonic, sonic or infrasonic diagnostic devices with special arrangements for interfacing with the operator or the patient
- A61B8/461—Displaying means of special interest
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/0093—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
Landscapes
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Physics & Mathematics (AREA)
- Optics & Photonics (AREA)
- Biomedical Technology (AREA)
- Medical Informatics (AREA)
- Pathology (AREA)
- Radiology & Medical Imaging (AREA)
- Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Heart & Thoracic Surgery (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Molecular Biology (AREA)
- Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Ultra Sonic Daignosis Equipment (AREA)
Abstract
The invention discloses an augmented reality processing method and system for ultrasonic image display, which comprises augmented reality glasses and an ultrasonic probe with a visual mark, wherein the augmented reality glasses comprise a head-mounted display module and a host, more than one camera is mounted on the augmented reality glasses, and the ultrasonic probe with the visual mark is distributed with a circle of visual mark at the position of a probe body and can reflect/emit visible light or infrared light.
Description
Technical Field
The invention relates to an augmented reality technology and an ultrasonic technology, in particular to an augmented reality processing system and method for ultrasonic image display.
Background
In the ultrasound system used in the medical market at present, the medical display screen is adopted for displaying images and is generally placed above the ultrasound trolley. The patient lies in one side of the doctor, the doctor can not see the patient when observing the screen due to the literacy of the vocational training, and the doctor needs to select and measure the body position of the patient according to the examination items, such as a horizontal position, a left lateral position, a right lateral position, a sitting position or a semi-lying position. The doctor himself or herself is fixed, and the direction of looking at the screen is also fixed, so when examining patients in various body positions, the doctor often needs to make a body position on one side to touch the examination part of the patient. So that the scapulohumeral periarthritis and the cervical spondylosis become an occupational disease of the sonographer.
The invention provides an augmented reality processing system for ultrasonic image display, which consists of AR augmented reality glasses, a computing processing device and an infrared tracking mark fixed on an ultrasonic probe. The system can display the ultrasonic image in front of the eyes of a doctor all the time, or display the ultrasonic image at a fixed height in front of the doctor along with the rotation of the body of the doctor, or directly display the ultrasonic image fusion at the affected part of a patient in the ultrasonic guidance type puncture treatment. Thereby reducing the situations of long-term shoulder and cervical vertebra tension of doctors in the process of operating ultrasound and reducing occupational diseases.
The head movement detection matched with the current AR display technology is basically realized based on three technologies, namely an IMU attitude sensor which can sense the acceleration and the angular acceleration on 3 or 6 axes. The second is to use a three-dimensional vision sensor on a helmet, and the third is an image sensor installed in a room space, capturing marker points distributed on augmented reality glasses, so that the angle of the user's head with respect to a fixed position in the room can be calculated. The head motion detection proposed by the present invention does not focus on the position of the head mounted display relative to the actual physical space, but on the position of the head relative to the body. The projection of the image in a specific direction of the user's body is achieved, the image moving with the user. The invention relates to another technology for detecting the position of the ultrasonic probe relative to the head-mounted display of a user and realizing the projection of an image on the relative position of the ultrasonic probe, and because the position of an ultrasonic echo image relative to the ultrasonic probe is fixed, the projection of an ultrasonic image on the corresponding position of an affected part can be finally realized.
Disclosure of Invention
The present invention is directed to an augmented reality processing method and system for displaying an ultrasound image, so as to solve the problems in the background art.
In order to achieve the purpose, the invention provides the following technical scheme:
an augmented reality processing system usable for ultrasound image display, characterized by: the system comprises augmented reality glasses and an ultrasonic probe with a visual marker, wherein the augmented reality glasses comprise a head-mounted display module (2) and a host (3), more than one camera is mounted on the augmented reality glasses, and the host is a computing unit with image processing or interface forwarding capacity; the ultrasonic probe with the visual mark is distributed with a plurality of visual marks at the position of the probe body, and can reflect/emit visible light or infrared light, and the augmented reality glasses and the ultrasonic probe with the visual mark are respectively electrically or wirelessly connected with the ultrasonic host;
preferably, the host (3) is connected with a third-party ultrasonic system host through a wired or wireless interface for data interaction, the head-mounted display module (2) is electrically connected with the host (3) in a wired mode, the host (3) is hung on the neck of a human body, a first set of posture sensors (6) are installed in the host (3), a second set of posture sensors (9) are installed in the head-mounted display module (2), and the posture sensors comprise three-axis angular acceleration sensors or three-axis geomagnetic sensors;
the method comprises the following steps of outputting images to two screens of a left eye and a right eye in the head-mounted display module (2) by adopting a first mode, so that a user can always see suspended virtual screen contents in a specific direction of head rotation around a body, wherein the first mode specifically comprises the following steps:
firstly, presetting a (0,0) coordinate of a virtual screen at a position of a d distance (theta, phi) direction relative to the front of a human body in a coordinate system;
secondly, the angle of the relative geomagnetic field read by the first set of attitude sensors is adoptedRelative geomagnetic field angle read by the second set of attitude sensors
Thirdly, calculating the deviation of two angles relative to the geomagnetic field in the third-party system host to obtain the rotation angle of the head of the user relative to the body
The fourth step, the third party system host computerThe angle is output to the head-mounted display module (2), and the original video content is deviated on a sphere with the radius d in front of the human bodyAfter the angle is output, the head-mounted display module is still positioned in a virtual screen field angle of the head-mounted display module (2);
the method also comprises an augmented reality processing method for displaying the ultrasonic image, and the images are output to the images of the left and right eyes in the head-mounted display module (2) by adopting a second mode, so that a user can see virtual video content at a spatial position fixed relative to the ultrasonic probe with the visual mark, wherein the second mode specifically comprises the following steps:
firstly, a camera or a plurality of cameras on the augmented reality glasses position the ultrasonic probe with the visual mark by adopting a structured light method or a parallax method, and the three-dimensional space position (x, y, z) of the ultrasonic probe relative to the augmented reality glasses and the pointing direction angle (theta ', phi') of the ultrasonic probe are calculated according to the visual mark;
secondly, assuming that an image needs to be presented at a distance L (L is more than or equal to 0) in front of the probe, the (0,0) coordinate of the virtual screen is located at a position (x + L Sin theta ' Cos phi ', y + LSin theta ' Sin phi ', z + LCos theta ') relative to the augmented reality glasses;
thirdly, the third-party system host outputs the video content to the position of (x + L Sin theta ' Cos phi ', y + LSin theta ' Sin phi ', z + LCos theta ') according to the (x, y, z) coordinates and (theta ', phi ') read by the current camera;
preferably, the visual indicia form a non-rotationally symmetric pattern, wherein at least one three-dimensional point is directed;
preferably, the augmented reality glasses are provided with a Wifi communication module or a cellular mobile communication module.
In addition, the ultrasonic diagnosis/detection system also comprises an ultrasonic detection device and an augmented reality processing system which is connected with the equipment through a wired or wireless mode and can be used for displaying ultrasonic images, wherein the augmented reality processing system is used as one of the display devices of the ultrasonic diagnosis/detection system.
Preferably, host computer (3) hang at human neck through neck circle (4), neck circle (4) include mounting panel, first elastic plate and second elastic plate, one side at the mounting panel is fixed to the host computer, one side of keeping away from the host computer at the mounting panel is fixed to first elastic plate and second elastic plate symmetry, first elastic plate and second elastic plate are circular-arcly, and the interval that mounting panel one end was kept away from to first elastic plate and second elastic plate is 5 centimetres, first elastic plate, second elastic plate and mounting panel three enclose into the wearing area.
Preferably, the one end that the mounting panel was kept away from to first elastic plate and second elastic plate is fixed with adjustment mechanism, adjustment mechanism includes adjusting band, elastic cord, turns to device and stop device, the both ends of elastic cord are fixed with the inboard of first elastic plate and second elastic plate respectively, it is fixed with the one end that the mounting panel was kept away from to turn to device and stop device respectively with first elastic plate and second elastic plate, the adjusting band is the flexible band, the port department that mounting panel one end was kept away from to the one end of adjusting band and second elastic plate is fixed, and it is fixed that the other end passes the spacing end that turns to device and stop device.
Preferably, the steering device comprises a supporting block and a steering roller, the supporting block is fixed on the outer side of one end, away from the mounting plate, of the first elastic plate, a device groove is formed in one side, away from the first elastic plate, of the supporting block, two ends of the steering roller are respectively connected with two side walls of the device groove in a rotating mode, one end, away from the second elastic plate, of the adjusting belt penetrates through a gap between the steering roller and the bottom of the device groove and bypasses the limiting end of the steering roller and is fixed with the limiting end of the limiting device.
Preferably, the limiting device comprises an installation block, a baffle, a limiting block, a limiting plate, a limiting spring, a limiting rod and a convex block, the installation block is fixed at one end of the second elastic plate far away from the installation plate, a through groove is arranged on the installation block, two baffles are symmetrically fixed at the inner side of the through groove, one end of the adjusting belt, which bypasses the steering roller, is inserted into a gap between the baffle and the through groove at one side close to the second elastic plate, a plurality of limiting grooves are equidistantly arranged at the outer side of the adjusting belt, the limiting grooves are of a right-angle triangular pyramid structure, one end of the limiting block is of a conical structure matched with the limiting grooves, one end of the limiting block in the conical structure is inserted into the limiting grooves, the other end of the limiting plate is fixed with the limiting spring, the other end of the limiting spring is fixed with one side of the through groove far away from the second elastic plate, one end of the, and the limiting rod is fixed with the limiting plate, one end of the limiting rod, which is far away from the limiting plate, penetrates through the mounting block to be fixed with the lug, and the limiting rod is connected with the mounting block in a sliding manner.
Preferably, a guard is fixed to the inner sides of the first and second elastic plates, the guard includes a guard band and a support, one end of the guard band is fixed to the second elastic plate, and the other end of the guard band is overlapped on the inner side of the first elastic plate through the support.
Preferably, the supporting member comprises a U-shaped frame, two ends of the U-shaped frame are fixed to the inner side of the first elastic plate, and one end of the protective belt, which is far away from the second elastic plate, is inserted into a gap between the U-shaped frame and the first elastic plate.
Compared with the prior art, the invention has the beneficial effects that:
the invention adopts two schemes of differential angular acceleration or differential geomagnetic sensors to realize the tracking of the visual field of the user in the movement process, so that the user can always see fixed display contents at a fixed angle of head swing. Or projecting the ultrasonic image on the front, back, upper part or the relative position of the ultrasonic probe such as the affected part of the patient.
Drawings
FIG. 1 is a schematic view of the overall structure of the present invention
FIG. 2 is a schematic view of a neck ring structure according to the present invention;
FIG. 3 is a schematic structural view of a limiting mechanism of the present invention;
FIG. 4 is an enlarged view of area A of FIG. 2;
FIG. 5 is an enlarged view of area B of FIG. 2;
FIG. 6 is an enlarged view of area C of FIG. 3;
FIG. 7 is a block diagram of the system architecture of the present invention;
FIG. 8 is a schematic view of the attitude angle in the non-rotated condition;
FIG. 9 is a schematic diagram of the use of mode one;
FIG. 10 is a schematic diagram of the second mode;
in the figure: 1-a computational processing means; 2-a display module; 3-a host; 4-neck ring; 5-a processor; 6-a first set of attitude sensors; 7-a battery; 8-a near-eye display module; 9-a second set of attitude sensors; 10-mounting a plate; 11-a first elastic plate; 12-a second resilient plate; 13-a wearing area; 14-an adjustment mechanism; 15-an adjustment band; 16-an elastic band; 17-a steering device; 18-a stop device; 19-a support block; 20-a turning roll; 21-device slot; 22-an installation block; 23-a baffle plate; 24-a stopper; 25-a limiting plate; 26-a limit spring; 27-a limiting rod; 28-a bump; 29-through slots; 30-a limiting groove; 31-a guard; 32-a protective band; 33-a support; 34-U-shaped frame.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.
Referring to fig. 1-10, the present invention provides a technical solution:
an augmented reality processing system usable for ultrasound image display, characterized by: including augmented reality glasses and the ultrasonic probe who takes the visual marking, augmented reality glasses include wear display module assembly (2) and host computer (3), install the more than one camera on the augmented reality glasses, the ultrasonic probe who takes the visual marking distributes at probe fuselage position has a plurality of visual markings, can reflect/launch visible light or infrared light, augmented reality glasses and the ultrasonic probe who takes the visual marking, respectively with supersound host computer electric connection or wireless connection, perhaps the probe itself just is portable host computer thereby augmented reality glasses and ultrasonic probe electric connection or wireless connection.
Augmented reality glasses can split into two structures, wear display module assembly 2 and host computer 3, host computer 3 carries out the data interaction through wired or wireless interface and third party system host computer connection, wear to display module assembly 2 and host computer 3 between wired electric connection, host computer 3 hangs at human neck, install first set of attitude sensor in the host computer 3, wear to install second set attitude sensor 9 in the display module assembly 2, attitude sensor is including triaxial angular acceleration sensor or triaxial geomagnetic sensor, and attitude sensor has 3 angular acceleration sensors of quadrature of each other at least, and geomagnetic sensor contains the geomagnetic field sensor of three quadrature of each other at least.
Preferably, the head-mounted display module 2 includes a second set of attitude sensor 9, a near-to-eye display module and a matching driving circuit, and is provided with a pair of cameras, and the second set of attitude sensor 9 includes a triaxial angular acceleration or a triaxial geomagnetic field sensor.
The method also comprises an augmented reality processing method for ultrasonic image display, and the method adopts a mode I to output images to the left and right eyes of the head-mounted display module 2, so that a user can always see suspended virtual screen content in a specific direction of head rotation around the body, wherein the mode I specifically comprises the following steps:
firstly, presetting a (0,0) coordinate of a virtual screen at a position of a d distance (theta, phi) direction relative to the front of a human body in a coordinate system;
second, using the first set of attitude sensors to readRelative geomagnetic field angleRelative geomagnetic field angle read by the second set of attitude sensors
Thirdly, calculating the deviation of two angles relative to the geomagnetic field in the third-party system host to obtain the rotation angle of the head of the user relative to the body
The fourth step, the third party system host computerThe angle is output to the head-mounted display module 2, and the original video content is deviated on a sphere with the radius d in front of the human bodyAfter the angle is output, the head-mounted display module is still positioned in the angle of field of the virtual screen of the head-mounted display module 2;
the method also comprises an augmented reality processing method for displaying the ultrasonic image, and the images are output to the images of the left and right eyes in the head-mounted display module 2 by adopting a second mode, so that a user can see virtual video content at a space position fixed relative to the ultrasonic probe with the visual mark, wherein the second mode specifically comprises the following steps:
firstly, a camera or a plurality of cameras on the augmented reality glasses position the ultrasonic probe with the visual mark by adopting a structured light method or a parallax method, and the three-dimensional space position (x, y, z) of the ultrasonic probe relative to the augmented reality glasses and the pointing direction angle (theta ', phi') of the ultrasonic probe are calculated according to the visual mark;
secondly, assuming that an image needs to be presented at a distance L (L is more than or equal to 0) in front of the probe, the (0,0) coordinate of the virtual screen is located at a position (x + L Sin theta ' Cos phi ', y + LSin theta ' Sin phi ', z + LCos theta ') relative to the augmented reality glasses;
thirdly, the third-party system host outputs the video content to the position of (x + L Sin theta ' Cos phi ', y + LSin theta ' Sin phi ', z + LCos theta ') according to the (x, y, z) coordinates and (theta ', phi ') read by the current camera;
preferably, the visual indicia form a non-rotationally symmetric pattern, wherein at least one three-dimensional point is directed;
preferably, a special triangular mark is arranged outside the circular mark to generate a special direction of the whole graph.
Preferably, the augmented reality glasses are provided with a Wifi communication module or a cellular mobile communication module.
In addition, the ultrasonic diagnosis/detection system also comprises an ultrasonic detection device and an augmented reality processing system which is connected with the equipment through a wired or wireless mode and can be used for displaying ultrasonic images, wherein the augmented reality processing system is used as one of the display devices of the ultrasonic diagnosis/detection system.
The main machine 3 is hung on the neck of a human body through a neck ring 4, the neck ring 4 comprises a mounting plate 10, a first elastic plate 11 and a second elastic plate 12, the main body 3 is fixed on one side of the mounting plate 10, the first elastic plate 11 and the second elastic plate 12 are symmetrically fixed on one side of the mounting plate 10 far away from the main body 3, the first elastic plate 11 and the second elastic plate 12 are both arc-shaped, and the distance between the ends of the first elastic plate 11 and the second elastic plate 12 far away from the mounting plate 10 is 5 cm, the first elastic plate 11, the second elastic plate 12 and the mounting plate 10 enclose a wearing area 13, and when worn, the user can exert the stress to first elastic plate 11 and second elastic plate 12 respectively, makes first elastic plate 11 and second elastic plate 12 keep away from the interval grow of mounting panel 10 one end, and the wearing to host computer 3 can be realized in user's neck entering wearing zone 13.
The end of the first elastic plate 11 and the end of the second elastic plate 12 far away from the mounting plate 10 are fixed with an adjusting mechanism 14, the size of the wearing area 13 can be adjusted through the adjusting mechanism 14, so that the neck ring 4 is suitable for users with different body shapes, and the first elastic plate 11 and the second elastic plate 12 are prevented from moving around the neck of a human body when the user walks.
Adjustment mechanism 14 includes adjusting band 15, elastic cord 16, turns to device 17 and stop device 18, the both ends of elastic cord 16 are fixed with the inboard of first elastic plate 11 and second elastic plate 12 respectively, it is fixed with the one end that mounting panel 10 was kept away from to turn to device 17 and stop device 18 respectively with first elastic plate 11 and second elastic plate 12, adjusting band 15 is the flexible band, the port department that mounting panel 10 one end was kept away from to the one end of adjusting band 15 and second elastic plate 12 is fixed, and the other end passes and turns to the spacing end of device 17 and stop device 18 and fix, and the length of elastic cord 16 can change according to user's bodily form, and through the length of control adjusting band 15, can realize keeping away from the regulation of the interval of mounting panel 10 one end to first elastic plate 11 and second elastic plate 12 to the realization is to the regulation of dressing district 13 size.
The steering device 17 comprises a supporting block 19 and a steering roller 20, the supporting block 19 is fixed on the outer side of one end, away from the mounting plate 10, of the first elastic plate 11, a device groove 21 is formed in one side, away from the first elastic plate 11, of the supporting block 19, two ends of the steering roller 20 are respectively connected with two side walls of the device groove 21 in a rotating mode, one end, away from the second elastic plate 12, of the adjusting belt 15 penetrates through a gap between the steering roller 20 and the bottom of the device groove 21 and bypasses the limiting end of the steering roller 20 and the limiting device 18 to be fixed, the direction of the adjusting belt 15 is changed by the steering device 17, and the adjusting belt 15 is convenient to fix.
The limiting device 18 comprises an installation block 22, a baffle plate 23, a limiting block 24, a limiting plate 25, a limiting spring 26, a limiting rod 27 and a convex block 28, the installation block 22 is fixed at one end of the second elastic plate 12 far away from the installation plate 10, a through groove 29 is arranged on the installation block 22, two baffle plates 23 are symmetrically fixed at the inner side of the through groove 29, the adjusting belt 15 bypasses one end of a steering roller 20 and is inserted into a gap between the baffle plate 23 and the through groove 29 near one side of the second elastic plate 12, a plurality of limiting grooves 30 are arranged at the outer side of the adjusting belt 15 at equal intervals, the limiting grooves 30 are right-angle triangular pyramid structures, one end of the limiting block 24 is a conical structure matched with the limiting grooves 30, one end of the limiting block 24 which is a conical structure is inserted into the limiting grooves 30, the other end of the limiting block 25 is fixed with the limiting plate 25, the limiting plate 25 is fixed with one end of the limiting spring 26, the other end of, one end of the limiting rod 27 is sleeved on the inner side of the limiting spring 26 and fixed with the limiting plate 25, one end of the limiting rod 27, which is far away from the limiting plate 25, penetrates through the mounting block 22 and is fixed with the bump 28, and the limiting rod 27 is connected with the mounting block 22 in a sliding manner; when the size of the wearing area 13 is reduced, a pulling force is applied to one end of the adjusting belt 15, the limiting block 24 slides out of the current limiting groove 30 and is clamped with the limiting grooves 30 at different positions on the adjusting belt 15, and the wearing area 13 can be reduced; when the size of wearing region 13 is increased, through applying pulling force to lug 28, make stopper 24 break away from spacing groove 30, remove the position of adjusting band 15 again, increase the interval that first elastic plate 11 and second elastic plate 12 kept away from mounting panel 10 one end, after the position of adjusting band 15 was confirmed, cancel the pulling force to lug 28, stopper 24 joint is in spacing groove 30 again, can realize wearing the enlargeing of region 13.
The inner sides of the first elastic plate 11 and the second elastic plate 12 are fixed with a protection piece 31, the protection piece 31 comprises a protection belt 32 and a support piece 33, one end of the protection belt 32 is fixed with the second elastic plate 12, the other end of the protection belt is lapped on the inner side of the first elastic plate 11 through the support piece 33, and the protection belt 32 can isolate the gap between the ends of the first elastic plate 11 and the second elastic plate 12 far away from the mounting plate 10 and prevent the gap from wearing the neck of a human body.
The supporting member 33 includes a U-shaped frame 34, both ends of the U-shaped frame 34 are fixed to the inner side of the first elastic plate 11, one end of the protective belt 32, which is far away from the second elastic plate 12, is inserted into a gap between the U-shaped frame 34 and the first elastic plate 11, and the U-shaped frame 34 supports the protective belt 32, so that the positions of the protective belt 32 are respectively consistent with the first elastic plate 11 and the second elastic plate 12.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (10)
1. An augmented reality processing system usable for ultrasound image display, characterized by: the system comprises augmented reality glasses and an ultrasonic probe with a visual marker, wherein the augmented reality glasses comprise a head-mounted display module (2) and a host (3), more than one camera is mounted on the augmented reality glasses, and the host (3) is a computing unit with image processing or interface forwarding capacity; the ultrasonic probe with the visual marks is distributed with a plurality of visual marks at the position of the probe body, and can reflect/emit visible light or infrared light, and the augmented reality glasses and the ultrasonic probe with the visual marks are respectively and electrically connected or wirelessly connected with the ultrasonic host.
2. An augmented reality processing system usable for ultrasound image display according to claim 1, wherein: host computer (3) carry out the data interaction through wired or wireless interface and third party's ultrasonic system host computer connection, wear display module assembly (2) with wired electric connection between host computer (3), host computer (3) hang at human neck, install first set of attitude sensor (6) in host computer (3), install second set of attitude sensor (9) in wearing display module assembly (2), attitude sensor includes triaxial angular acceleration sensor or triaxial geomagnetic sensor.
3. An augmented reality processing method for displaying an ultrasound image, comprising: the system of claim 2, wherein the mode one is used to output images to the left and right eye screens of the head-mounted display module (2), so that the user can always see the suspended virtual screen content in a specific direction of the head rotating around the body, and the mode one is specifically:
firstly, presetting a (0,0) coordinate of a virtual screen at a position of a d distance (theta, phi) direction relative to the front of a human body in a coordinate system;
secondly, the angle of the relative geomagnetic field read by the first set of attitude sensors is adoptedRelative geomagnetic field angle read by the second set of attitude sensors
Thirdly, calculating the deviation of two angles relative to the geomagnetic field in the third-party system host to obtain the rotation angle of the head of the user relative to the body
The fourth step, the third party system host computerThe angle is output to the head-mounted display module (2), and the original video content is deviated on a sphere with the radius d in front of the human bodyAfter the angle is output, the head-mounted display module is still positioned in the virtual screen field angle of the head-mounted display module (2).
4. An augmented reality processing method for displaying an ultrasound image, comprising: the system of claim 2, wherein a mode two is adopted to output images to the left and right eye screens of the head-mounted display module (2), so that a user can see virtual video content at a fixed spatial position relative to the ultrasound probe with the visual marker, and the mode two is specifically:
firstly, a camera or a plurality of cameras on the augmented reality glasses position the ultrasonic probe with the visual mark by adopting a structured light method or a parallax method, and according to the visual mark, the three-dimensional space position (x, y, z) of the ultrasonic probe relative to the augmented reality glasses and the pointing direction angle (theta ', phi') of the ultrasonic probe are calculated;
secondly, assuming that an image needs to be presented at a distance L (L is more than or equal to 0) in front of the probe, the (0,0) coordinate of the virtual screen is located at a position (x + Lsin theta ' Cos phi ', y + LSin theta ' Sin phi ', z + LCos theta ') relative to the augmented reality glasses;
and thirdly, the third-party system host outputs the video content to the position of (x + Lsin theta ' Cos phi ', y + LSin theta ' Sin phi ', z + LCos theta ') according to the (x, y, z) coordinates and (theta ', phi ') read by the current camera.
5. An augmented reality processing method as claimed in claim 3 or 4, wherein the visual marker forms a non-rotationally symmetric pattern in which at least one three-dimensional point is directed.
6. An augmented reality processing method available for ultrasound image display according to claim 3 or 4, wherein: the augmented reality glasses are provided with a Wifi communication module or a honeycomb mobile communication module.
7. An ultrasonic diagnosis/detection system, characterized in that an ultrasonic detection device and peripheral equipment of the device are connected with the augmented reality processing system for ultrasonic image display of claim 1 in a wired or wireless manner, and the augmented reality processing system is used as one of the display devices of the ultrasonic diagnosis/detection system.
8. An ultrasonic diagnostic/detection system according to claim 7, characterized in that: hang in human neck through neck circle (4) host computer (3), neck circle (4) are including mounting panel (10), first elastic plate (11) and second elastic plate (12), one side at mounting panel (10) is fixed in host computer (3), one side of keeping away from host computer (3) in mounting panel (10) is fixed to first elastic plate (11) and second elastic plate (12) symmetry, first elastic plate (11) and second elastic plate (12) are circular-arc, and the interval that mounting panel (10) one end was kept away from in first elastic plate (11) and second elastic plate (12) is 5 centimetres, first elastic plate (11), second elastic plate (12) and mounting panel (10) three enclose into wearing district (13).
9. An ultrasonic diagnostic/detection system according to claim 8, characterized in that: the one end that mounting panel (10) were kept away from in first elastic plate (11) and second elastic plate (12) is fixed with adjustment mechanism (14), adjustment mechanism (14) include adjusting band (15), elastic cord (16), turn to device (17) and stop device (18), the both ends of elastic cord (16) are fixed with the inboard of first elastic plate (11) and second elastic plate (12) respectively, turn to device (17) and stop device (18) and fix with the one end that mounting panel (10) were kept away from in first elastic plate (11) and second elastic plate (12) respectively, adjusting band (15) are the bandolier, the port department that mounting panel (10) one end was kept away from in one end and second elastic plate (12) of adjusting band (15) is fixed, and the other end passes the spacing end that turns to device (17) and stop device (18) and fixes.
10. An ultrasonic diagnostic/detection system according to claim 9, wherein: the steering device (17) comprises a supporting block (19) and a steering roller (20), the supporting block (19) is fixed on the outer side of one end, away from the mounting plate (10), of the first elastic plate (11), a device groove (21) is formed in one side, away from the first elastic plate (11), of the supporting block (19), two ends of the steering roller (20) are respectively in rotating connection with two side walls of the device groove (21), one end, away from the second elastic plate (12), of the adjusting belt (15) penetrates through a gap between the steering roller (20) and the bottom of the device groove (21) and bypasses the steering roller (20) to be fixed with a limiting end of the limiting device (18); the limiting device (18) comprises an installation block (22), baffles (23), limiting blocks (24), limiting plates (25), limiting springs (26), limiting rods (27) and a convex block (28), the installation block (22) is fixed at one end, far away from the installation plate (10), of the second elastic plate (12), a through groove (29) is formed in the installation block (22), two baffles (23) are symmetrically fixed on the inner side of the through groove (29), one end, bypassing the steering roller (20), of the adjusting belt (15) is inserted into a gap between the baffles (23) and the through groove (29) close to one side of the second elastic plate (12), a plurality of limiting grooves (30) are formed in the outer side of the adjusting belt (15) in an equal distance mode, the limiting grooves (30) are of right-angle triangular pyramid structures, one ends of the limiting blocks (24) are of cone-shaped structures, and one ends of the limiting blocks (24) are inserted into the limiting grooves (30), the other end of the limiting plate is fixed with a limiting plate (25), the limiting plate (25) is fixed with one end of a limiting spring (26), the other end of the limiting spring (26) is fixed with one side, away from the second elastic plate (12), of the through groove (29), one end of the limiting rod (27) is sleeved on the inner side of the limiting spring (26) and is fixed with the limiting plate (25), one end, away from the limiting plate (25), of the limiting rod (27) penetrates through the mounting block (22) to be fixed with the bump (28), and the limiting rod (27) is in sliding connection with the mounting block (22); a protection piece (31) is fixed on the inner sides of the first elastic plate (11) and the second elastic plate (12), the protection piece (31) comprises a protection belt (32) and a supporting piece (33), one end of the protection belt (32) is fixed with the second elastic plate (12), and the other end of the protection belt is overlapped on the inner side of the first elastic plate (11) through the supporting piece (33); the supporting piece (33) comprises a U-shaped frame (34), two ends of the U-shaped frame (34) are fixed to the inner side of the first elastic plate (11), and one end, far away from the second elastic plate (12), of the protective belt (32) is inserted into a gap between the U-shaped frame (34) and the first elastic plate (11).
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910902733.7A CN110609388A (en) | 2019-09-24 | 2019-09-24 | Augmented reality processing method and system for ultrasonic image display |
| CN202010991008.4A CN112180600B (en) | 2019-09-24 | 2020-09-23 | Augmented reality processing method and system for ultrasonic image display |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910902733.7A CN110609388A (en) | 2019-09-24 | 2019-09-24 | Augmented reality processing method and system for ultrasonic image display |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110609388A true CN110609388A (en) | 2019-12-24 |
Family
ID=68892055
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910902733.7A Pending CN110609388A (en) | 2019-09-24 | 2019-09-24 | Augmented reality processing method and system for ultrasonic image display |
| CN202010991008.4A Active CN112180600B (en) | 2019-09-24 | 2020-09-23 | Augmented reality processing method and system for ultrasonic image display |
Family Applications After (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010991008.4A Active CN112180600B (en) | 2019-09-24 | 2020-09-23 | Augmented reality processing method and system for ultrasonic image display |
Country Status (1)
| Country | Link |
|---|---|
| CN (2) | CN110609388A (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113100828A (en) * | 2021-05-11 | 2021-07-13 | 天津大学 | Head hoop type sounding movement ultrasonic image detection helmet |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104856720B (en) * | 2015-05-07 | 2017-08-08 | 东北电力大学 | A kind of robot assisted ultrasonic scanning system based on RGB D sensors |
| CN109069103B (en) * | 2016-04-19 | 2022-01-25 | 皇家飞利浦有限公司 | Ultrasound imaging probe positioning |
| CN108616752B (en) * | 2018-04-25 | 2020-11-06 | 北京赛博恩福科技有限公司 | Head-mounted equipment supporting augmented reality interaction and control method |
-
2019
- 2019-09-24 CN CN201910902733.7A patent/CN110609388A/en active Pending
-
2020
- 2020-09-23 CN CN202010991008.4A patent/CN112180600B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN112180600B (en) | 2022-03-25 |
| CN112180600A (en) | 2021-01-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR101988110B1 (en) | Virtual Reality Education System and Method based on Bio Sensors | |
| CA2934987A1 (en) | Stereoscopic viewing of an image on a body supported computer | |
| CN117043714A (en) | Split construction for wristband system and related apparatus and method | |
| JP2004537227A (en) | Image display method and image display device | |
| KR20130031120A (en) | Apparatus and method for assisting in positioning user's posture | |
| JP6774975B2 (en) | Eye rotation detectors, electronic devices and systems | |
| CN107028613B (en) | Cervical vertebra health state monitor | |
| JP2023116494A (en) | motion capture system | |
| KR20220148921A (en) | Eyewear that uses muscle sensors to determine facial expressions | |
| US10572002B2 (en) | Distributed artificial reality system with contextualized hand tracking | |
| US10739618B2 (en) | Wearable systems and methods for treatment of a neurocognitive condition | |
| KR101670667B1 (en) | Device for improving binocular vision | |
| KR101203921B1 (en) | Information providing apparatus using an eye tracking and local based service | |
| CN203722793U (en) | A wearable apparatus | |
| CN112180600B (en) | Augmented reality processing method and system for ultrasonic image display | |
| CN106842565A (en) | A kind of wearable intelligent vision enhancing equipment of separate type | |
| CN210626783U (en) | Augmented reality processing system and diagnosis/detection system for ultrasonic image display | |
| JP6342440B2 (en) | Eye movement detection device, eye movement detection method, and eyewear | |
| JP2018511082A (en) | Glasses to align the cervical spine | |
| US20130278741A1 (en) | Ear hanging type eye control device | |
| JP2023174820A (en) | Wearable apparatus and display method | |
| KR20150104338A (en) | Head tracking bar for head mount display | |
| CN206193355U (en) | Collapsible glasses based on VRAR technique | |
| JP2699259B2 (en) | Training method and training device for adapting spatial awareness | |
| Hanna et al. | Multisensory control of a straight locomotor trajectory |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20191224 |