CN116236222A - Ultrasonic probe pose positioning system and method of medical remote ultrasonic scanning robot - Google Patents
Ultrasonic probe pose positioning system and method of medical remote ultrasonic scanning robot Download PDFInfo
- Publication number
- CN116236222A CN116236222A CN202310281858.9A CN202310281858A CN116236222A CN 116236222 A CN116236222 A CN 116236222A CN 202310281858 A CN202310281858 A CN 202310281858A CN 116236222 A CN116236222 A CN 116236222A
- Authority
- CN
- China
- Prior art keywords
- coordinate
- ultrasonic probe
- ultrasonic
- pose
- positioning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 239000000523 sample Substances 0.000 title claims abstract description 104
- 238000000034 method Methods 0.000 title claims abstract description 33
- 239000013598 vector Substances 0.000 claims description 26
- 238000012937 correction Methods 0.000 claims description 22
- 238000007781 pre-processing Methods 0.000 claims description 17
- 238000006243 chemical reaction Methods 0.000 claims description 10
- 238000004364 calculation method Methods 0.000 claims description 7
- 238000002604 ultrasonography Methods 0.000 claims description 7
- 238000001514 detection method Methods 0.000 claims description 6
- 239000011159 matrix material Substances 0.000 claims description 6
- 238000013507 mapping Methods 0.000 claims description 4
- 230000007246 mechanism Effects 0.000 claims description 4
- 230000006872 improvement Effects 0.000 claims description 3
- 238000005070 sampling Methods 0.000 claims description 2
- 230000009466 transformation Effects 0.000 claims description 2
- 238000011217 control strategy Methods 0.000 claims 1
- 238000010606 normalization Methods 0.000 claims 1
- 230000000007 visual effect Effects 0.000 abstract description 3
- 230000004927 fusion Effects 0.000 abstract description 2
- 238000003384 imaging method Methods 0.000 description 4
- 210000004072 lung Anatomy 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
- A61B8/42—Details of probe positioning or probe attachment to the patient
- A61B8/4245—Details of probe positioning or probe attachment to the patient involving determining the position of the probe, e.g. with respect to an external reference frame or to the patient
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
- A61B8/08—Detecting organic movements or changes, e.g. tumours, cysts, swellings
- A61B8/0808—Detecting organic movements or changes, e.g. tumours, cysts, swellings for diagnosis of the brain
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
- A61B8/44—Constructional features of the ultrasonic, sonic or infrasonic diagnostic device
- A61B8/4444—Constructional features of the ultrasonic, sonic or infrasonic diagnostic device related to the probe
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
- A61B8/56—Details of data transmission or power supply
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1656—Programme controls characterised by programming, planning systems for manipulators
- B25J9/1664—Programme controls characterised by programming, planning systems for manipulators characterised by motion, path, trajectory planning
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1694—Programme controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
- B25J9/1697—Vision controlled systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
-
- G06T5/90—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10132—Ultrasound image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2210/00—Indexing scheme for image generation or computer graphics
- G06T2210/41—Medical
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Physics & Mathematics (AREA)
- Surgery (AREA)
- Public Health (AREA)
- Radiology & Medical Imaging (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Pathology (AREA)
- Veterinary Medicine (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Robotics (AREA)
- Mechanical Engineering (AREA)
- Computer Networks & Wireless Communication (AREA)
- Neurology (AREA)
- Ultra Sonic Daignosis Equipment (AREA)
Abstract
The invention discloses an ultrasonic probe posture positioning system and method of a medical remote ultrasonic scanning robot. According to the invention, the ultrasonic scanning target area is positioned by acquiring the image containing the target point through a target positioning method based on fusion of visual information and depth information, the probe posture is corrected through a multi-scale compensation method based on time and space, and finally, the probe posture is further optimized through the binding force feedback sensor, so that the ultrasonic probe posture positioning of the medical remote ultrasonic scanning robot is realized, the real-time, accurate and convenient posture positioning can be realized on the premise of using a low-cost sensor, the positioning precision is greatly improved, and the autonomy of the medical ultrasonic scanning robot is expanded.
Description
Technical Field
The invention belongs to the technical field of robots, relates to an ultrasonic probe pose positioning method, and in particular relates to an ultrasonic probe pose positioning method and system of a medical remote ultrasonic scanning robot.
Background
In medical diagnosis and detection, compared with other imaging modes such as CT, MRI, X-ray and the like, the ultrasonic imaging method has the advantages of low cost, no damage, high instantaneity, simplicity, convenience and the like, and along with development, the ultrasonic scanning robot is developed, so that the problem of medical resource shortage is greatly relieved, the working pressure of doctors is relieved, and meanwhile, the risk of indirect illness of the doctors is effectively avoided. However, the accuracy of the pose positioning of the ultrasonic probe directly influences the imaging effect, and the pose positioning method of the ultrasonic probe commonly used at present comprises visual positioning, point cloud positioning, electromagnetic positioning and the like. The prior art has some disadvantages.
CN112336374B discloses a method for realizing accurate positioning of an ultrasonic probe by using a binocular camera, and under the assistance of coding marks, the gesture of the ultrasonic probe is accurately positioned by using the binocular camera, and the method can only obtain the relative gesture of the probe, and needs coding assistance, so that the limitation is large. CN113940699a discloses a self-positioning method of an ultrasonic probe, which utilizes the reflective particle lines distributed inside the designed ultrasonic coupling gasket to realize the self-positioning of the ultrasonic probe, and has higher cost and poorer suitability for medical ultrasonic scanning. Therefore, there is an urgent need for a simple, fast, accurate and real-time method for positioning the pose of an ultrasonic probe to serve an ultrasonic scanning robot.
Disclosure of Invention
The invention provides an ultrasonic probe pose positioning system and method of a medical remote ultrasonic scanning robot, which aims to solve the problem that an ultrasonic imaging effect is poor due to a large positioning error of the ultrasonic probe pose before scanning of the medical ultrasonic scanning robot.
The invention aims at realizing the following technical scheme:
an ultrasonic probe gesture positioning system of a medical remote ultrasonic scanning robot comprises an ultrasonic probe, a depth camera, an image preprocessing module, a scanning target positioning module, a coordinate conversion module, a gesture positioning module, a gesture correction module, a force feedback correction module, a mechanical arm and a matched clamp, wherein:
the ultrasonic probe is used for collecting ultrasonic image information;
the depth camera is used for collecting an image of an area containing a scanning target point, and simultaneously obtaining depth information of each pixel point on the image;
the image preprocessing module is used for performing relevant preprocessing operations such as quality detection, size unification, contrast improvement and the like on images acquired by the depth camera;
the scanning target positioning module is used for automatically identifying and positioning a scanning target point according to manual setting, outputting two-dimensional coordinates of the scanning target point and calculating a first coordinate of a landing point by combining depth information;
the coordinate conversion module is used for converting the three-dimensional coordinate of the landing point under the depth camera coordinate system into the mechanical arm base coordinate system to obtain a second coordinate which is used as input for driving the mechanical arm to move;
the gesture positioning module is used for obtaining three-dimensional gesture information of the ultrasonic scanning probe at a second coordinate point, namely three included angles of the probe and a coordinate axis of the mechanical arm base;
the pose correction module is used for correcting the three-dimensional pose information of the ultrasonic probe based on multi-scale compensation, so that positioning errors are reduced;
the force feedback correction module is used for further finely optimizing the pose of the ultrasonic probe through a compensation strategy by combining the force feedback information of the mechanical arm end actuating mechanism, so that the ultrasonic image with higher quality can be obtained;
the matched clamp is used for fixing the depth camera and the ultrasonic probe at the tail end of the mechanical arm.
An ultrasonic probe pose positioning method for a medical remote ultrasonic scanning robot by using the system comprises the following steps:
step one: acquiring an image containing a region to be scanned of a patient by using a depth camera arranged on a fixed position of the mechanical arm, and calibrating a color channel and a depth channel of the depth camera;
step two: inputting the image acquired by the depth camera into an image preprocessing module for changing the image size, improving the contrast, detecting the quality and the like;
step three: inputting the image processed by the image preprocessing module into a scanning target positioning module, and solving the two-dimensional coordinates P of the landing coordinate point 0 (x,y)；
Step four: mapping the landing coordinate point to a three-dimensional coordinate under a camera coordinate system by combining the depth data value d of the landing coordinate point, and calling the coordinate as a first coordinate P 1 ；
Step five: determining a second coordinate P of the first coordinate under the coordinate system of the mechanical arm base through a coordinate conversion module 2 ；
Step six: inputting the second coordinates to a gesture positioning module to obtain a three-dimensional gesture angle of the ultrasonic probe, so that the three-dimensional gesture P of the ultrasonic probe is finally output by combining the second coordinates prode ；
Step seven: inputting the three-dimensional pose of the ultrasonic probe of the ultrasonic scanning robot into a pose correction module to obtain a compensated three-dimensional pose P';
step eight: after the probe contacts human skin, the three-dimensional pose of the ultrasonic probe is corrected again through the force feedback correction module, and the finally obtained three-dimensional pose P of the ultrasonic probe is output.
Compared with the prior art, the invention has the following advantages:
according to the invention, an ultrasonic scanning target area is positioned by acquiring an image containing a target point through a target positioning method based on fusion of visual information and depth information, a plane normal vector is calculated near a landing coordinate point to serve as the posture of an ultrasonic probe, then the posture of the probe is corrected through a multi-scale compensation method based on time and space, and finally, the posture of the probe is further optimized through a binding force feedback sensor, so that the ultrasonic probe posture positioning of the medical remote ultrasonic scanning robot is realized, and the real-time, accurate and convenient posture positioning can be realized on the premise of using a low-cost sensor, so that the positioning precision is greatly improved and the autonomy of the medical ultrasonic scanning robot is expanded. The ultrasonic automatic scanning robot provides a good foundation for realizing high-quality ultrasonic scanning detection on the premise of ensuring the safety of patients and systems.
Drawings
Fig. 1 is a flowchart of an ultrasonic probe pose positioning method of a medical remote ultrasonic scanning robot in an embodiment of the present invention:
FIG. 2 is an illustration of an ultrasound probe pose positioning system installation of a medical remote ultrasound scanning robot in an embodiment of the present invention;
FIG. 3 is a schematic view of an ultrasonic probe pose positioning system of a medical remote ultrasonic scanning robot in an embodiment of the invention;
FIG. 4 is a schematic illustration of an ultrasonic probe plane normal vector solution for a medical remote ultrasound scanning robot in an embodiment of the invention;
FIG. 5 is a schematic diagram of a multi-scale compensation method according to an embodiment of the present invention.
Detailed Description
The following description of the present invention is provided with reference to the accompanying drawings, but is not limited to the following description, and any modifications or equivalent substitutions of the present invention should be included in the scope of the present invention without departing from the spirit and scope of the present invention.
The invention provides an ultrasonic probe posture positioning system of a medical remote ultrasonic scanning robot, as shown in fig. 3, the system comprises an ultrasonic probe, a depth camera, an image preprocessing module, a scanning target positioning module, a coordinate conversion module, a posture positioning module, a posture correction module, a force feedback correction module, a mechanical arm and a matched clamp, wherein:
the ultrasonic probe is used for collecting ultrasonic image information;
the depth camera is used for collecting an image of an area containing a scanning target point, and simultaneously obtaining depth information of each pixel point on the image;
the image preprocessing module is used for performing relevant preprocessing operations such as quality detection, size unification, contrast improvement and the like on images acquired by the depth camera;
the scanning target positioning module is used for automatically identifying and positioning a scanning target point according to manual setting, outputting two-dimensional coordinates of the scanning target point and calculating a first coordinate of a landing point by combining depth information;
the coordinate conversion module is used for converting the three-dimensional landing point coordinate under the depth camera coordinate system into the mechanical arm base coordinate system to obtain a second coordinate which is used as input for driving the mechanical arm to move;
the gesture positioning module is used for obtaining three-dimensional gesture information of the ultrasonic scanning probe at a second coordinate point, namely three included angles of the probe and a coordinate axis of the mechanical arm base;
the pose correction module is used for correcting the three-dimensional pose information of the ultrasonic probe based on multi-scale compensation, so that positioning errors are reduced;
the force feedback correction module is used for further finely optimizing the pose of the ultrasonic probe through a compensation strategy by combining the force feedback information of the mechanical arm end actuating mechanism, so that the ultrasonic image with higher quality can be obtained;
the matched clamp is used for fixing the depth camera and the ultrasonic probe at the tail end of the mechanical arm.
The invention also provides an ultrasonic probe pose positioning method for the medical remote ultrasonic scanning robot by using the system, which comprises the following steps:
step one: and acquiring an image containing the region to be scanned of the patient by using a depth camera arranged on a fixed position of the mechanical arm, and calibrating a color channel and a depth channel of the depth camera.
Step two: and inputting the image acquired by the depth camera into an image preprocessing module for changing the image size, improving the contrast, detecting the quality and the like.
Step three: inputting the image processed by the image preprocessing module into a scanning target positioning module, adopting a target detection algorithm based on Yolov4 to position coordinates of landing points of a scanning area in real time, and solving two-dimensional coordinates P of the landing coordinate points 0 (x,y)。
Step four: mapping the landing coordinate point to a three-dimensional coordinate under a camera coordinate system by combining the depth data value d of the landing coordinate point, and calling the coordinate as a first coordinate P 1 Find the first coordinate P 1 The calculation formula of (2) is shown as formula (1):
where f represents the focal length of the infrared camera of the depth camera.
Step five: determining a second coordinate P of the first coordinate under the coordinate system of the mechanical arm base through a coordinate conversion module 2 . Rotation matrix from camera coordinate system to ultrasonic probe coordinate system
And a rotation transformation matrix from the ultrasonic probe coordinate system to the mechanical arm base coordinate system +.>
Find the second coordinate P 2 Computational formulas such asThe formula is shown as follows:
wherein: rotation matrix
The rotation matrix is determined by the position of the camera mounted on the mechanical arm>
Is determined by the size of the mechanical arm.
Step six: inputting the second coordinates to a gesture positioning module to obtain a three-dimensional gesture angle of the ultrasonic probe, so that the three-dimensional gesture P of the ultrasonic probe is finally output by combining the second coordinates prode . The three-dimensional pose of the ultrasonic probe includes the landing coordinates P of the probe 2 And attitude angle. The attitude angle is determined by the normal vector of the human chest skin surface near the landing site, which is determined by the plane made up of three non-collinear points within the vicinity of the point attachment. The normal vector and the coordinate axis included angle of the mechanical arm base form the attitude angle of the ultrasonic probe.
Step seven: and inputting the three-dimensional pose of the ultrasonic probe of the ultrasonic scanning robot into a pose correction module to obtain a compensated three-dimensional pose P'. The posture correction module is used for eliminating positioning errors caused by small-range movement of the body of the patient, so that the positioning accuracy is improved as much as possible. The method is based on a three-dimensional probe posture positioning algorithm of multi-scale compensation, and the three-dimensional probe posture is compensated from two dimensions of space and time. The spatial compensation is to calculate normal vectors of four pixels by using four pixels with a distance delta near the second coordinate point, and move the start points of the normal vectors to the landing point P 2 And summing and normalizing to obtain the compensated normal vector. In time, the normal vector after spatial compensation of continuous five sampling time deltat is summed and normalized to obtain the final normal vector
The calculation formula is shown as formula (3):
where a represents the normal vector of the vector,
the normal vector representing the target landing site, the normal vector of the remaining symbols being the normal vector involved in compensation.
Step eight: after the probe contacts the skin of the human body, the three-dimensional pose of the ultrasonic probe is corrected again through the force feedback correction module, and the finally obtained three-dimensional pose P of the ultrasonic probe is output, so that high-quality ultrasonic images can be obtained under the condition of ensuring the safety of the human body and a robot system. The force feedback correction module corrects the three-dimensional posture of the ultrasonic probe by using a pressure expected value sensed by an end actuating mechanism of the mechanical arm, and further optimizes the three-dimensional posture of the ultrasonic probe by using a compensation strategy of the formula (4):
wherein k is p Is a proportionality coefficient, k d As differential coefficient, pressure F end Feedback is from the pressure sensor at the end of the robot arm. And finally, calculating the component of the compensation quantity L in the coordinate system of the mechanical arm base to obtain the final three-dimensional pose.
Examples:
as shown in fig. 1, the present embodiment performs scan target positioning of the automatic lung ultrasound scan robot according to the following steps:
step one, acquiring an image of an area to be scanned of a patient by using a depth camera arranged on a fixed position of a mechanical arm, wherein in the embodiment, an ultrasonic couplant needs to be smeared on the area to be scanned of the patient in advance, and meanwhile, a color channel and a depth channel of the depth camera are calibrated, so that the color channel and the depth channel are in the same coordinate system.
And step two, inputting the acquired image into an image preprocessing module. In this embodiment, the image size is converted into 512×512 after passing through the image preprocessing module, and the blurred image is removed and the contrast of the image after being retained is improved.
Step three, inputting the processed image into a scanning target positioning module, wherein the target positioning module can calculate the two-dimensional coordinates P of the landing coordinate point 0 (x, y). In this embodiment, the method for determining the coordinates of the landing coordinate point is to identify and locate the target by using a target locating method based on a deep convolutional neural network YoLov5 according to a preset scanning target.
Fourth, combining the depth data value d of the landing coordinate point, mapping the landing coordinate point to a three-dimensional coordinate under a camera coordinate system, and calling the coordinate as a first coordinate P 1 . The calculation formula for obtaining the first coordinates is shown in formula (1).
Step five, determining a second coordinate P of the first coordinate under the coordinate system of the mechanical arm base through a coordinate conversion module 2 The calculation formula is shown as formula (2).
Step six, inputting the second coordinates to a gesture positioning module to obtain a three-dimensional gesture angle of the ultrasonic probe, so that the three-dimensional gesture P of the ultrasonic probe is finally output by combining the second coordinates prode . In this embodiment, a schematic diagram of normal vector solution is shown in fig. 4.
And step seven, inputting the three-dimensional pose of the ultrasonic probe of the ultrasonic scanning robot into a pose correction module to obtain a compensated three-dimensional pose P'. In this embodiment, the multi-scale compensated three-dimensional probe pose positioning algorithm is shown in fig. 5.
And step eight, correcting the three-dimensional pose of the ultrasonic probe through a force feedback correction module after the probe contacts human skin, further optimizing the three-dimensional pose of the ultrasonic probe by using a compensation strategy of a formula (4), and finally calculating the component of the compensation quantity L in a coordinate system of a mechanical arm base to obtain the final three-dimensional pose P of the ultrasonic probe, so that a high-quality ultrasonic image can be obtained under the condition of ensuring the safety of a human body and a robot system.
Taking a medical remote ultrasound scanning robot as an example for scanning the lungs of a patient, five feature points of the chest of the patient are usually scanned and ultrasound images are obtained. The three-dimensional pose positioning error of the ultrasonic probe when the pose positioning method of the ultrasonic probe of the embodiment is adopted is shown in table 1, wherein the distance error is the euclidean distance between the probe landing point and the actual target point, and the angle error is the absolute value of the difference value between the probe pose angle and the actual pose angle. The average distance error is about 6.38mm, the average angle error is about 0.15rad, the method accords with the error range of lung ultrasonic scanning, and higher-precision positioning can be provided for the acquisition of subsequent ultrasonic images.
TABLE 1
Claims (8)
1. An ultrasonic probe posture positioning system of a medical remote ultrasonic scanning robot is characterized by comprising an ultrasonic probe, a depth camera, an image preprocessing module, a scanning target positioning module, a coordinate conversion module, a posture positioning module, a posture correction module, a force feedback correction module, a mechanical arm and a matched clamp, wherein:
the ultrasonic probe is used for collecting ultrasonic image information;
the depth camera is used for collecting an image of an area containing a scanning target point, and simultaneously obtaining depth information of each pixel point on the image;
the image preprocessing module is used for performing quality detection, size unification and contrast improvement related preprocessing operations on images acquired by the depth camera;
the scanning target positioning module is used for automatically identifying and positioning a scanning target point according to manual setting, outputting two-dimensional coordinates of the scanning target point and calculating a first coordinate of a landing point by combining depth information;
the coordinate conversion module is used for converting the three-dimensional coordinate of the landing point under the depth camera coordinate system into the mechanical arm base coordinate system to obtain a second coordinate which is used as input for driving the mechanical arm to move;
the gesture positioning module is used for obtaining three-dimensional gesture information of the ultrasonic scanning probe at the second coordinate point;
the pose correction module is used for correcting the three-dimensional pose information of the ultrasonic probe based on multi-scale compensation, so that positioning errors are reduced;
the force feedback correction module is used for further finely optimizing the pose of the ultrasonic probe by combining the force feedback information of the mechanical arm end actuating mechanism through a designed control strategy, so that a higher-quality ultrasonic image can be obtained;
the matched clamp is used for fixing the depth camera and the ultrasonic probe at the tail end of the mechanical arm.
2. The ultrasonic probe pose positioning system of a medical remote ultrasonic scanning robot according to claim 1, characterized in that the compensation strategy is as follows:
wherein L is the compensation amount, k p Is a proportionality coefficient, k d As differential coefficient, pressure F end Feedback is from the pressure sensor at the end of the robot arm.
3. A method for positioning the pose of an ultrasonic probe of a medical remote ultrasonic scanning robot by using the system of any one of claims 1-2, characterized in that the method comprises the following steps:
step one: acquiring an image containing a region to be scanned of a patient by using a depth camera arranged on a fixed position of the mechanical arm, and calibrating a color channel and a depth channel of the depth camera;
step two: inputting the image acquired by the depth camera into an image preprocessing module for changing the image size, improving the contrast and detecting the quality;
step three: will map the figureThe image processed by the image preprocessing module is input into the scanning target positioning module to obtain the two-dimensional coordinates P of the landing coordinate point 0 (x,y)；
Step four: mapping the landing coordinate point to a three-dimensional coordinate under a camera coordinate system by combining the depth data value d of the landing coordinate point, and calling the coordinate as a first coordinate P 1 ；
Step five: determining a second coordinate P of the first coordinate under the coordinate system of the mechanical arm base through a coordinate conversion module 2 ；
Step six: inputting the second coordinates to a gesture positioning module to obtain a three-dimensional gesture angle of the ultrasonic probe, so that the three-dimensional gesture P of the ultrasonic probe is finally output by combining the second coordinates prode ；
Step seven: inputting the three-dimensional pose of the ultrasonic probe of the ultrasonic scanning robot into a pose correction module to obtain a compensated three-dimensional pose P';
step eight: after the probe contacts human skin, the three-dimensional pose of the ultrasonic probe is corrected again through the force feedback correction module, and the finally obtained three-dimensional pose P of the ultrasonic probe is output.
4. The method for positioning the pose of an ultrasonic probe of a medical remote ultrasonic scanning robot according to claim 3, wherein in said fourth step, the first coordinate P 1 The calculation formula of (2) is as follows:
z 1 ＝d
where f represents the focal length of the infrared camera of the depth camera.
5. A medical remote ultrasound scanning robot according to claim 3The ultrasonic probe pose positioning method is characterized in that in the fifth step, a second coordinate P 2 The calculation formula of (2) is as follows:
wherein the method comprises the steps of
A rotation matrix from a camera coordinate system to an ultrasonic probe coordinate system; />
The rotation transformation matrix from the ultrasonic probe coordinate system to the mechanical arm base coordinate system is adopted.
6. The method for positioning the ultrasonic probe pose of the medical remote ultrasonic scanning robot according to claim 3, wherein in the sixth step, the three-dimensional pose of the ultrasonic probe comprises a landing coordinate P of the probe 2 And attitude angle, wherein: the attitude angle is determined by the normal vector of the human chest skin surface near the landing point, the normal vector is determined by the plane formed by the non-collinear three points in the vicinity of the point, and the included angle between the normal vector and the coordinate axis of the mechanical arm base forms the attitude angle of the ultrasonic probe.
7. The ultrasonic probe pose positioning method of a medical remote ultrasonic scanning robot according to claim 3, wherein in the seventh step, a three-dimensional probe pose is compensated from two dimensions of space and time by adopting a three-dimensional probe pose positioning algorithm based on multi-scale compensation, wherein:
the spatial compensation is to calculate normal vectors of four pixels by using four pixels with a distance delta near the second coordinate point, and move the start points of the normal vectors to the landing point P 2 Summing and normalizing to obtain a compensated normal vector;
spatially compensating for successive five sampling times DeltatThe normal vector after summation and normalization are carried out to obtain the final normal vector
8. The ultrasonic probe pose positioning method of a medical remote ultrasonic scanning robot according to claim 7, characterized in that the method comprises the following steps of
The calculation formula of (2) is as follows:
where a represents the normal vector of the vector,
representing the normal vector of the target landing site. />
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310281858.9A CN116236222A (en) | 2023-03-22 | 2023-03-22 | Ultrasonic probe pose positioning system and method of medical remote ultrasonic scanning robot |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310281858.9A CN116236222A (en) | 2023-03-22 | 2023-03-22 | Ultrasonic probe pose positioning system and method of medical remote ultrasonic scanning robot |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116236222A true CN116236222A (en) | 2023-06-09 |
Family
ID=86635898
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310281858.9A Pending CN116236222A (en) | 2023-03-22 | 2023-03-22 | Ultrasonic probe pose positioning system and method of medical remote ultrasonic scanning robot |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116236222A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116725730A (en) * | 2023-08-11 | 2023-09-12 | 北京市农林科学院智能装备技术研究中心 | Pig vaccine injection method, system and storage medium based on visual guidance |
| CN116972973A (en) * | 2023-07-17 | 2023-10-31 | 中国科学院上海光学精密机械研究所 | Full-automatic space spectrum measuring device and method for luminous object |
| CN117147699A (en) * | 2023-10-31 | 2023-12-01 | 江苏蓝格卫生护理用品有限公司 | Medical non-woven fabric detection method and system |
-
2023
- 2023-03-22 CN CN202310281858.9A patent/CN116236222A/en active Pending
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116972973A (en) * | 2023-07-17 | 2023-10-31 | 中国科学院上海光学精密机械研究所 | Full-automatic space spectrum measuring device and method for luminous object |
| CN116725730A (en) * | 2023-08-11 | 2023-09-12 | 北京市农林科学院智能装备技术研究中心 | Pig vaccine injection method, system and storage medium based on visual guidance |
| CN116725730B (en) * | 2023-08-11 | 2023-12-05 | 北京市农林科学院智能装备技术研究中心 | Pig vaccine injection method, system and storage medium based on visual guidance |
| CN117147699A (en) * | 2023-10-31 | 2023-12-01 | 江苏蓝格卫生护理用品有限公司 | Medical non-woven fabric detection method and system |
| CN117147699B (en) * | 2023-10-31 | 2024-01-02 | 江苏蓝格卫生护理用品有限公司 | Medical non-woven fabric detection method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116236222A (en) | Ultrasonic probe pose positioning system and method of medical remote ultrasonic scanning robot | |
| CN112006779B (en) | Precision detection method of surgical navigation system | |
| US8147503B2 (en) | Methods of locating and tracking robotic instruments in robotic surgical systems | |
| CN113137920B (en) | Underwater measurement equipment and underwater measurement method | |
| EP1942662B1 (en) | Automatic improvement of tracking data for intraoperative C-arm images in image guided surgery | |
| CN113876426B (en) | Intraoperative positioning and tracking system and method combined with shadowless lamp | |
| CN108245788B (en) | Binocular distance measuring device and method and accelerator radiotherapy system comprising same | |
| CN111524194B (en) | Positioning method and terminal for mutually fusing laser radar and binocular vision | |
| CN112168357B (en) | System and method for constructing spatial positioning model of C-arm machine | |
| EP3799790B1 (en) | Method for tracking location of two-dimensional non-destructive inspection scanner on target object using scanned structural features | |
| CN111759463A (en) | Method for improving positioning precision of surgical mechanical arm | |
| CN113724337B (en) | Camera dynamic external parameter calibration method and device without depending on tripod head angle | |
| CN113211431A (en) | Pose estimation method based on two-dimensional code correction robot system | |
| CN116883471B (en) | Line structured light contact-point-free cloud registration method for chest and abdomen percutaneous puncture | |
| CN111145267B (en) | 360-degree panoramic view multi-camera calibration method based on IMU assistance | |
| CN114463482A (en) | Calibration model and method of optical tracking three-dimensional scanner and surgical navigation system thereof | |
| CN116158851B (en) | Scanning target positioning system and method of medical remote ultrasonic automatic scanning robot | |
| CN113733078B (en) | Method for interpreting fine control quantity of mechanical arm and computer-readable storage medium | |
| CN115902925A (en) | Towed body posture automatic identification method | |
| CN115836875A (en) | Correction method and system | |
| CN113379710B (en) | Underwater target sonar accurate measurement system and method | |
| CN111553891B (en) | Handheld object existence detection method | |
| CN114748169A (en) | Autonomous endoscope moving method of laparoscopic surgery robot based on image experience | |
| CN114549648B (en) | RPC model positioning system based on high-resolution remote sensing satellite stereoscopic image | |
| CN116543057B (en) | Underwater multi-camera and IMU integrated calibration method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |