CN115869009B - Remote ultrasonic diagnosis system and interactive control method - Google Patents
Remote ultrasonic diagnosis system and interactive control method Download PDFInfo
- Publication number
- CN115869009B CN115869009B CN202310139469.2A CN202310139469A CN115869009B CN 115869009 B CN115869009 B CN 115869009B CN 202310139469 A CN202310139469 A CN 202310139469A CN 115869009 B CN115869009 B CN 115869009B
- Authority
- CN
- China
- Prior art keywords
- ultrasonic
- diagnosis
- ultrasonic probe
- image
- operator
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003745 diagnosis Methods 0.000 title claims abstract description 158
- 238000000034 method Methods 0.000 title claims abstract description 18
- 230000002452 interceptive effect Effects 0.000 title claims abstract description 11
- 239000000523 sample Substances 0.000 claims abstract description 154
- 238000012544 monitoring process Methods 0.000 claims abstract description 72
- 230000009471 action Effects 0.000 claims abstract description 20
- 238000004891 communication Methods 0.000 claims abstract description 17
- 238000012545 processing Methods 0.000 claims description 19
- 238000004171 remote diagnosis Methods 0.000 abstract description 3
- 230000009286 beneficial effect Effects 0.000 abstract description 2
- 230000001276 controlling effect Effects 0.000 description 23
- 230000008569 process Effects 0.000 description 5
- 238000006073 displacement reaction Methods 0.000 description 4
- 238000002604 ultrasonography Methods 0.000 description 4
- 230000000875 corresponding effect Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000013519 translation Methods 0.000 description 2
- 230000001419 dependent effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000005674 electromagnetic induction Effects 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 231100000957 no side effect Toxicity 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Abstract
The invention relates to the technical field of ultrasonic diagnosis, in particular to a remote ultrasonic diagnosis system and an interactive control method. The technical scheme is as follows: the remote execution end comprises a remote communication module, an execution mechanical arm, a monitoring camera and ultrasonic scanning equipment, wherein the execution mechanical arm is connected with the monitoring camera and the ultrasonic probe of the ultrasonic scanning equipment in a driving way. The beneficial effects are that: the touch screen and the manipulator are controlled to control the action of the ultrasonic probe, the monitoring camera is arranged at the tail end of the execution mechanical arm to monitor the diagnosis part in real time, and the image and the ultrasonic image of the diagnosis part are displayed on the diagnosis display screen, so that an operator can accurately judge the remote diagnosis part at the operation end and accurately control the action of the ultrasonic probe, and the accuracy of remote ultrasonic diagnosis is improved.
Description
Technical Field
The invention relates to the technical field of ultrasonic diagnosis, in particular to a remote ultrasonic diagnosis system and an interactive control method.
Background
Ultrasonic scanning is used as a detection and diagnosis means which is simple and portable, has low cost and no side effect, and is increasingly widely applied in the medical industry. However, the diagnosis of the ultrasonic scanning is highly dependent on the experience of doctors, and it is important to grasp the position and direction of the probe in order to obtain a high-resolution ultrasonic image. The ultrasonic expert resources with abundant experience are relatively lack and are unevenly distributed, and accurate diagnosis on the ultrasonic difficult cases is difficult to carry out in primary hospitals in most small cities. With the development of network communication technology, the problems of insufficient resources and uneven distribution of ultrasonic expert can be effectively solved through remote consultation at present. At present, in the field of remote ultrasonic diagnosis, a conventional remote control mechanical arm is adopted to control an ultrasonic probe, however, in the ultrasonic scanning process, an ultrasonic expert is required to control the ultrasonic probe to move, meanwhile, the angle, the pressing force and the like of the ultrasonic probe are required to be adjusted according to the actual feedback of a diagnosis part, and the control requirement of the ultrasonic expert on the ultrasonic probe cannot be met by the conventional remote diagnosis system. In addition, in the process of remote ultrasonic scanning and diagnosis, an ultrasonic expert pays attention to the condition of a diagnosis part, and also looks at an ultrasonic image obtained by ultrasonic probe scanning, and the ultrasonic image is directly transmitted and displayed, so that the ultrasonic expert is difficult to well correspond to the ultrasonic image and the diagnosis part when performing remote ultrasonic diagnosis, and the efficiency and the accuracy of the remote ultrasonic diagnosis are affected. Meanwhile, in the ultrasonic scanning diagnosis process, when a fixed camera is used for shooting a diagnosis part, the probe or a manipulator for controlling the movement of the probe easily shields the diagnosis part, so that a real-time image of the diagnosis part cannot be clearly acquired at an operation end, and when the camera is arranged on the probe, the movement of the probe causes the movement transformation of the image of the diagnosis part, so that the specific judgment of an ultrasonic expert at the operation end on the diagnosis part is easily affected.
Disclosure of Invention
The invention aims to provide a remote ultrasonic diagnosis system and an interactive control method, which are used for solving at least one of the problems proposed by the background technology and improving the accuracy of remote ultrasonic diagnosis.
In order to achieve the above purpose, the present invention adopts the following technical scheme: the remote ultrasonic diagnosis system comprises a control end and a remote execution end, wherein the control end comprises a control platform and an upper computer, a diagnosis display screen, a control touch screen and a movable manipulator are arranged on the control platform, a three-axis gyroscope and a track sensor are arranged in the manipulator, the remote execution end comprises a remote communication module, an execution mechanical arm, a monitoring camera and ultrasonic scanning equipment, and the tail end of the execution mechanical arm is in driving connection with an ultrasonic probe of the monitoring camera and the ultrasonic scanning equipment; the control end and the remote execution end are in communication connection through the upper computer and the remote communication module.
Specifically, be provided with eye discernment camera on controlling the platform, eye discernment camera is used for discernment to detect the position of operator's eyes before controlling the platform.
Specifically, the manipulator comprises a base and a handle, the track sensor is arranged on the base, the handle is rotationally connected with the base, the triaxial gyroscope is arranged in the handle, and the handle is provided with a probe operation button.
Specifically, the tail end of the handle is provided with an elastic feedback component, a first pressure sensor is arranged between the handle and the elastic feedback component, and a second pressure sensor is arranged between the tail end of the executing mechanical arm and the ultrasonic probe.
Specifically, be provided with first laser rangefinder sensor on the handle, carry out the arm end and be provided with second laser rangefinder sensor.
Specifically, the monitoring cameras are two and are respectively arranged at two sides of the ultrasonic probe.
The interactive control method for remote ultrasonic diagnosis adopts the remote ultrasonic diagnosis system to carry out remote ultrasonic diagnosis, specifically, an operator sets parameters of an execution mechanical arm and ultrasonic scanning equipment by using an upper computer, the execution mechanical arm is controlled by using an operation touch screen or an operator before an operation platform is operated, the execution mechanical arm drives and moves an ultrasonic probe of the ultrasonic scanning equipment according to control signals of the operation touch screen and the operator, a monitoring camera at a remote execution end carries out image acquisition on a diagnosis part and displays the image on a diagnosis display screen of the operation platform, the ultrasonic probe of the ultrasonic scanning equipment carries out ultrasonic scanning on the diagnosis part, and the ultrasonic probe is simultaneously displayed on the diagnosis display screen of the operation platform after the ultrasonic image is obtained by processing of an ultrasonic host; when the control touch screen is used for controlling the execution mechanical arm, the action of the execution mechanical arm is controlled by using different touch gestures on the control touch screen; when the manipulator is used for controlling the execution mechanical arm, the manipulator moves on the control platform, and the motion information of the manipulator is obtained by obtaining the sensing information of the track sensor and the three-axis gyroscope of the manipulator, so that the execution mechanical arm is controlled to drive the ultrasonic probe to perform the motion with the same motion of the manipulator.
Further, when the monitoring camera collects images of the diagnosis part and displays the images on the diagnosis display screen, the control touch screen or the manipulator controls the execution mechanical arm to drive the monitoring camera to aim at the diagnosis part, then the monitoring program of the diagnosis part is pneumatically executed, and the images shot by the monitoring camera are processed according to the movement or rotation of the ultrasonic probe driven by the execution mechanical arm in the X-axis, Y-axis and Z-axis directions, so that the images of the diagnosis part displayed on the diagnosis display screen are kept still; when the ultrasonic probe moves in the X-axis direction and the Y-axis direction, translating an image shot by the monitoring camera according to the moving direction and the moving amount; when the ultrasonic probe moves in the Z-axis direction, scaling the image shot by the monitoring camera according to the moving direction and the moving amount; when the ultrasonic probe rotates in the X-axis and Y-axis directions, stretching and deforming an image shot by the monitoring camera according to the rotation direction and the rotation angle; when the ultrasonic probe rotates in the Z-axis direction, the image shot by the monitoring camera is rotated according to the rotation direction and the rotation angle.
Further, an ultrasonic probe of the ultrasonic scanning equipment performs ultrasonic scanning on a diagnosis part, and when an ultrasonic image is obtained through processing of an ultrasonic host and is simultaneously displayed on a diagnosis display screen of the control platform, an eye recognition camera is arranged on the control platform to recognize and position the position of eyes of an operator, so that the position relation between the eyes of the operator and the diagnosis display screen is determined; meanwhile, determining the position relation between the ultrasonic probe and the diagnosis part according to the gesture of the ultrasonic probe driven by the execution mechanical arm and the image of the diagnosis part shot by the monitoring camera, so as to determine the position relation between the ultrasonic image scanned by the ultrasonic probe and the diagnosis part; and finally, determining the position relation between the eyes of the operator and the diagnostic part image displayed on the diagnostic display screen according to the position relation between the eyes of the operator and the diagnostic display screen, carrying out projection deformation processing on the ultrasonic image by combining the position relation between the ultrasonic image and the diagnostic part, and displaying the ultrasonic image subjected to the projection deformation processing and the image of the diagnostic part on the diagnostic display screen after overlapping.
Further, when the operator is used for controlling the action of the ultrasonic probe, an elastic feedback component is arranged at the tail end of the operator to apply an elastic force to the operator so as to simulate the elastic force of the ultrasonic probe on the diagnosis part; the elastic coefficient of the diagnosis part is detected by arranging pressure sensors at the connection parts of the tail end of the execution mechanical arm and the ultrasonic probe, so that the elastic coefficient of the elastic feedback part is adjusted; and a pressure sensor is also arranged on the elastic feedback component of the manipulator and used for detecting the pressure applied by an operator on the manipulator, so that the control execution mechanical arm controls the ultrasonic probe to be applied on the diagnosis site with the same pressure.
The invention has the beneficial effects that: the control touch screen and the manipulator are arranged to control the control execution mechanical arm so as to control the action of the ultrasonic probe, the monitoring camera is arranged at the tail end of the execution mechanical arm to monitor the diagnosis part in real time, and the image and the ultrasonic image of the diagnosis part are displayed on the diagnosis display screen, so that an operator can accurately judge the remote diagnosis part at the operation end and accurately control the action of the ultrasonic probe, and the accuracy of remote ultrasonic diagnosis is improved. The manipulator can truly simulate the situation that the ultrasonic probe moves on the diagnosis part, and simulate the elasticity of the diagnosis part to the ultrasonic probe by utilizing the elastic feedback part, so that the accuracy of remote ultrasonic diagnosis can be further improved; when the diagnostic part is subjected to image acquisition, a monitoring camera which is used for executing the movement of the tail end of the mechanical arm and moves together with the ultrasonic probe is used for shooting, and the image shot by the monitoring camera is subjected to deformation processing according to the action of the ultrasonic probe, so that the diagnostic part image displayed on the diagnostic display screen is kept still; meanwhile, an eye recognition camera is arranged on the control platform to recognize and position the position of eyes of an operator, an ultrasonic image obtained by scanning by an ultrasonic probe is projected onto a diagnosis display screen according to the position of the glasses of the operator and is overlapped with the image of a diagnosis part, so that virtual display of the ultrasonic image is realized, and the accuracy of remote ultrasonic diagnosis is further improved.
Drawings
FIG. 1 is a schematic diagram of the connection of a remote ultrasound diagnostic system in an embodiment;
fig. 2 is a schematic diagram of the principle of the ultrasonic probe and the monitoring camera shooting the diagnosis part in the embodiment;
fig. 3 is a schematic diagram of the principle of displaying the ultrasonic image and the diagnostic part image on the diagnostic display screen after the projection deformation processing in the embodiment.
Description of the embodiments
1, referring to fig. 1-3, a remote ultrasonic diagnostic system comprises a control end and a remote execution end, wherein the control end comprises a control platform and an upper computer, a diagnostic display screen, a control touch screen and a movable manipulator are arranged on the control platform, a triaxial gyroscope and a track sensor are arranged in the manipulator, the remote execution end comprises a remote communication module, an execution mechanical arm, a monitoring camera and ultrasonic scanning equipment, and the tail end of the execution mechanical arm is in driving connection with an ultrasonic probe of the monitoring camera and the ultrasonic scanning equipment; the control end and the remote execution end are in communication connection through the upper computer and the remote communication module.
In this embodiment, the upper computer of the control end is used for performing remote communication with the remote communication module of the remote execution end, so that an operator can set parameters of the execution mechanical arm and the ultrasonic scanning device of the remote execution end by the upper computer, and meanwhile, the upper computer performs remote communication with the remote communication module to realize communication connection among the diagnosis display screen of the control platform, the control touch screen and the operator, and the execution mechanical arm and the ultrasonic scanning device; the upper computer may use a computer with remote control software.
The control platform of the operation end is used for setting a diagnosis display screen, controlling a touch screen and a movable operator, the diagnosis display screen is used for displaying an image of a diagnosis part sent by the remote execution end and an ultrasonic image acquired and processed by the ultrasonic scanning equipment, the control touch screen and the operator are used for controlling the execution mechanical arm, so that the execution mechanical arm drives an ultrasonic probe of the ultrasonic scanning equipment to perform corresponding actions, and an operator can select to use the control touch screen or use the operator to control the ultrasonic probe according to control requirements. When the control touch screen is selected for control, an operator can control the action of the ultrasonic probe by different sliding gestures on the control touch screen, and the control touch screen can control the ultrasonic probe to move rapidly, so that the ultrasonic probe can reach the position to be diagnosed rapidly and the angle is adjusted; when the manipulator is selected to be used for control, an operator can detect the moving track of the manipulator on the horizontal plane by placing the manipulator on the control platform, so as to control the movement of the ultrasonic probe on the horizontal plane, in addition, a triaxial gyroscope is arranged in the manipulator and can detect the rotating action of the manipulator, so that the manipulator is controlled to drive the ultrasonic probe to rotate according to the rotating action of the manipulator, and particularly, the manipulator with six degrees of freedom can move and rotate in the X-axis, Y-axis and Z-axis directions, wherein the X-axis and the Y-axis are positioned on the horizontal plane, the Z-axis is perpendicular to the X-axis and the Y-axis, the moving track of the manipulator detected by the track sensor of the manipulator on the horizontal plane corresponds to the movement of the manipulator driving the ultrasonic probe on the X-axis and the Y-axis directions, and the tilting rotation of the manipulator detected by the triaxial gyroscope corresponds to the manipulator driving the ultrasonic probe to the rotation of the manipulator driving the X-axis, the Y-axis and the Z-axis directions; the operator can control the micro-motion of the ultrasonic probe on the diagnosis part. In addition, control different sliding gestures on the touch screen, can realize controlling and carry out the arm and drive the ultrasonic probe at the removal and the rotation of X axle, Y axle and Z axle direction, this embodiment provides a feasible sliding gesture scheme, specifically does: (1) controlling the ultrasonic probe to move in the X-axis and Y-axis directions by single-finger sliding on the control touch screen; (2) controlling the ultrasonic probe to move in the Z-axis direction by double-finger scaling on the control touch screen; (3) controlling the rotation of the ultrasonic probe along the X-axis and Y-axis directions by double-finger sliding on the control touch screen; (4) and controlling the rotation of the ultrasonic probe along the Z-axis direction by double-finger rotation on the control touch screen.
In addition, the monitoring camera of the remote execution end is arranged at the tail end of the execution mechanical arm together with the ultrasonic probe and moves together with the ultrasonic probe, so that the monitoring camera can be prevented from being blocked by the ultrasonic probe or the execution mechanical arm when shooting a diagnosis part by the monitoring camera. In addition, after the monitoring camera shoots the diagnosis part to obtain an image of the diagnosis part, the image is deformed according to the action of driving the ultrasonic probe by the mechanical arm, so that the image of the diagnosis part displayed on the diagnosis display screen of the control platform is kept still. Specifically, an operator firstly controls the execution mechanical arm to drive the ultrasonic probe and the monitoring camera to aim at the diagnosis part by using the control touch screen or the operator, so that the monitoring camera shoots an image of the complete diagnosis part, then starts a monitoring program of the diagnosis part, and processes the image shot by the monitoring camera according to the movement or rotation of the ultrasonic probe in the X-axis, Y-axis and Z-axis directions under the driving of the execution mechanical arm, so that the image of the diagnosis part displayed on the diagnosis display screen is kept still. The X-axis, Y-axis and Z-axis in this embodiment refer to a coordinate axis system of a mechanical arm itself for driving an ultrasonic probe to move, where planes of the X-axis and the Y-axis are parallel to a horizontal plane, the axes are perpendicular to the horizontal plane, and the mechanical arm for driving the ultrasonic probe and the monitoring camera to move is a six-axis mechanical arm with six degrees of freedom, which can drive the ultrasonic probe and the monitoring camera to move in the X-axis, Y-axis and Z-axis directions and rotate in the X-axis, Y-axis and Z-axis directions, and can directly acquire the motion of the mechanical arm for driving the ultrasonic probe to move to determine the direction and the movement amount of the movement or rotation of the ultrasonic probe in the X-axis, Y-axis and Z-axis directions, so as to implement real-time processing of the image of the diagnostic part captured by the monitoring camera. When the ultrasonic probe moves in the X-axis direction and the Y-axis direction, carrying out translation processing on the image shot by the monitoring camera according to the moving direction and the moving amount, specifically, if the ultrasonic probe moves in the positive direction of the X-axis direction, carrying out translation processing on the image shot by the monitoring camera by a corresponding distance in the negative direction of the X-axis direction; when the ultrasonic probe moves in the Z-axis direction, scaling the image shot by the monitoring camera according to the moving direction and the moving amount, specifically, if the ultrasonic probe moves in the positive direction of the Z-axis direction, namely moves upwards, the image shot by the monitoring camera is amplified, and the image shot by the monitoring camera is reduced in the negative direction; when the ultrasonic probe rotates in the X-axis and Y-axis directions, stretching and deforming an image shot by the monitoring camera according to the rotation direction and the rotation angle; when the ultrasonic probe rotates in the Z-axis direction, the image shot by the monitoring camera is rotated according to the rotation direction and the rotation angle.
The two monitoring cameras are preferably arranged on two sides of the ultrasonic probe respectively, the images of the two diagnosis positions shot by the two monitoring cameras are combined in an image splicing mode, and the same picture area is combined, so that the images of the two diagnosis positions are combined into one image and displayed on a display screen of an operation end. Meanwhile, when a monitoring program for the diagnosis part is started and the ultrasonic probe moves or rotates in the X-axis, Y-axis and Z-axis directions, and the images shot by the monitoring cameras are processed, the images shot by the two monitoring cameras after the images of the two diagnosis parts are fused are processed. The monitoring cameras are arranged on two sides of the ultrasonic probe, and pictures shot by the two monitoring cameras are combined into one image, so that the shooting range of the monitoring cameras can be enlarged, and the effect of real-time monitoring is improved.
In a further embodiment, an eye recognition camera is arranged on the control platform, and the eye recognition camera is used for recognizing and detecting the position of eyes of an operator in front of the control platform. The eye recognition camera is used for recognizing and positioning the position of eyes of an operator, so that the ultrasonic image acquired by the ultrasonic scanning equipment is projected and deformed and then is displayed on the diagnosis display screen after being overlapped with the image of the diagnosis part, and the operator can accurately judge the position relationship between the ultrasonic image and the diagnosis part. The specific projection deformation processing mode is as follows: firstly, identifying and positioning eyes of an operator so as to determine the position relationship between the eyes of the operator and a diagnosis display screen; meanwhile, determining the position relation between the ultrasonic probe and the diagnosis part according to the gesture of the ultrasonic probe driven by the execution mechanical arm and the image of the diagnosis part shot by the monitoring camera, so as to determine the position relation between the ultrasonic image scanned by the ultrasonic probe and the diagnosis part; and finally, determining the position relation between the eyes of the operator and the diagnostic part image displayed on the diagnostic display screen according to the position relation between the eyes of the operator and the diagnostic display screen, carrying out projection deformation processing on the ultrasonic image by combining the position relation between the ultrasonic image and the diagnostic part, and displaying the ultrasonic image subjected to the projection deformation processing and the image of the diagnostic part on the diagnostic display screen after overlapping. Specifically, when the eyes of the operator are identified and positioned, the diagnosis display screen is horizontally placed, so that a scene that a patient lies on a bed for ultrasonic diagnosis is simulated, the front of the display screen of the operator, which is generally positioned at the operation end, is aligned with the diagnosis part on the display screen, when the eyes of the operator are identified and positioned, the eyes of the operator are photographed by using the eye identification cameras, the positions of the eyes of the operator are identified and positioned, a plurality of eye identification cameras can be arranged around the display screen of the operation end, more specifically, cameras can be arranged on the side surface and the upper side of the operator, and the positional relationship of the eyes of the operator relative to the display screen can be determined.
In addition, because the position of the ultrasonic probe changes to change the position relation of the ultrasonic image relative to the diagnosis part, when an operator remotely controls the ultrasonic probe to move, the ultrasonic image needs to be processed in real time according to the movement condition of the ultrasonic probe, and in order to improve the real-time performance of ultrasonic image processing, the control signals of a touch screen or an operator on the ultrasonic probe can be directly collected and controlled to determine the action of the ultrasonic probe, so that the position relation of the ultrasonic probe and the diagnosis part can be monitored in real time; so that the ultrasound image transmitted from the remote can be processed quickly and displayed on the display screen in real time.
Specifically, the manipulator comprises a base and a handle, the track sensor is arranged on the base, the handle is rotationally connected with the base, the triaxial gyroscope is arranged in the handle, and the handle is provided with a probe operation button. The handle of the manipulator is used for simulating the ultrasonic probe, the probe operation buttons on the handle correspond to the operation buttons on the actual ultrasonic probe, the ultrasonic probe can be subjected to corresponding ultrasonic scanning control by utilizing the probe operation buttons on the handle through the remote communication module, the handle and the base are in 360-degree rotating connection, the base can enable the manipulator to slide on the control platform, the handle can rotate by 360 degrees on the base, and therefore the ultrasonic probe is controlled to rotate and incline in different directions, and ultrasonic scanning of different angles is carried out on a diagnosis part by adjusting the ultrasonic probe. Wherein, 360 degrees rotation connection can be carried out through spherical connecting piece between handle and the base.
In a further embodiment, the handle end is further provided with an elastic feedback component, a first pressure sensor is arranged between the handle and the elastic feedback component, and a second pressure sensor is arranged between the execution mechanical arm end and the ultrasonic probe. The elastic feedback component arranged at the tail end of the handle can provide elastic feedback for the handle and is used for simulating the elastic force generated when the ultrasonic probe moves on the diagnosis part, specifically, the second pressure sensor arranged between the tail end of the mechanical arm and the ultrasonic probe is used for detecting the elastic force applied to the diagnosis part by the ultrasonic probe, and the elastic feedback component is combined with the displacement of the mechanical arm for driving the ultrasonic probe to press up and down on the diagnosis part, so that the elastic coefficient of the diagnosis part can be calculated, and then the elastic feedback component adjusts the elastic coefficient of the handle according to the elastic coefficient of the diagnosis part, so that the elastic feedback applied to the ultrasonic probe when moving on the diagnosis part is simulated. Meanwhile, the executing mechanical arm can drive the ultrasonic probe to act on the diagnosis part with the same pressure according to the pressure value detected by the first pressure sensor between the handle and the elastic feedback component.
Specifically, the elastic feedback component can adopt an electromagnetic push rod structure, and comprises a permanent magnet and an electromagnetic coil, wherein the permanent magnet is movably inserted into the electromagnetic coil. The electromagnetic push rod structure adopts an electromagnetic induction principle, a permanent magnet is movably inserted into an electromagnetic coil, a magnetic field opposite to the permanent magnet is generated after the electromagnetic coil is electrified and acts on the permanent magnet, the elastic coefficient of the electromagnetic coil to the permanent magnet is in direct proportion to the voltage values at two ends of the electromagnetic coil and in inverse proportion to the depth of the permanent magnet inserted into the electromagnetic coil, the electromagnetic coil can be fixed on a base, the permanent magnet is fixed at the tail end of a handle, a first pressure sensor can be arranged between the tail end of the handle and the permanent magnet, the acting force of the electromagnetic coil to the permanent magnet can be determined by utilizing the pressure value detected by the first pressure sensor, so that the depth of the permanent magnet inserted into the electromagnetic coil is determined, the voltage values at two ends of the electromagnetic coil are adjusted, and the elastic feedback component can provide the handle with the elastic coefficient equal to the elastic coefficient of a diagnosis part.
Further, a first laser ranging sensor is arranged on the handle, and a second laser ranging sensor is arranged at the tail end of the execution mechanical arm. Through set up first laser rangefinder sensor on the handle, can measure the handle and control the distance of platform towards, combine the handle and control inclination between the platform (this angle can be confirmed according to the triaxial gyroscope in the handle), can confirm the handle and control the distance change between the platform, can detect the displacement of handle in the Z axle direction to this control carries out the removal of arm drive ultrasonic probe in the Z axle direction. In addition, the second laser ranging sensor arranged at the tail end of the mechanical arm or on the ultrasonic probe is executed, the distance between the ultrasonic probe and the diagnosis part can be measured, the distance between the ultrasonic probe and the diagnosis part can be kept consistent with the distance between the manipulator and the control platform by matching with the first laser ranging sensor arranged on the manipulator, the image of the diagnosis part is displayed by arranging the display screen on the control platform, and an operator can truly simulate the action of the ultrasonic probe on the diagnosis part by using the manipulator on the control platform, so that the authenticity of a remote ultrasonic diagnosis process is further improved.
In the discussion of the remote ultrasonic diagnostic system above, the interactive control method using the remote ultrasonic diagnostic system has been discussed, and is summarized here as follows: the method comprises the steps that an operator sets parameters of an execution mechanical arm and ultrasonic scanning equipment by using an upper computer, controls the execution mechanical arm by using an operation touch screen or an operator before an operation platform, drives and moves an ultrasonic probe of the ultrasonic scanning equipment according to control signals of the operation touch screen and the operator, a monitoring camera at a remote execution end acquires an image of a diagnosis part and displays the image on a diagnosis display screen of the operation platform, the ultrasonic probe of the ultrasonic scanning equipment performs ultrasonic scanning on the diagnosis part, and the ultrasonic probe is processed by an ultrasonic host to obtain an ultrasonic image and simultaneously displays the ultrasonic image on the diagnosis display screen of the operation platform; when the control touch screen is used for controlling the execution mechanical arm, the action of the execution mechanical arm is controlled by using different touch gestures on the control touch screen; when the manipulator is used for controlling the execution mechanical arm, the manipulator moves on the control platform, and the motion information of the manipulator is obtained by obtaining the sensing information of the track sensor and the three-axis gyroscope of the manipulator, so that the execution mechanical arm is controlled to drive the ultrasonic probe to perform the motion with the same motion of the manipulator.
When the monitoring camera collects images of the diagnosis part and displays the images on the diagnosis display screen, the control touch screen or the manipulator controls the execution mechanical arm to drive the monitoring camera to aim at the diagnosis part, then the monitoring program of the diagnosis part is pneumatically executed, and the images shot by the monitoring camera are processed according to the movement or rotation of the ultrasonic probe driven by the execution mechanical arm in the X-axis, Y-axis and Z-axis directions, so that the images of the diagnosis part displayed on the diagnosis display screen are kept stationary; when the ultrasonic probe moves in the X-axis direction and the Y-axis direction, translating an image shot by the monitoring camera according to the moving direction and the moving amount; when the ultrasonic probe moves in the Z-axis direction, scaling the image shot by the monitoring camera according to the moving direction and the moving amount; when the ultrasonic probe rotates in the X-axis and Y-axis directions, stretching and deforming an image shot by the monitoring camera according to the rotation direction and the rotation angle; when the ultrasonic probe rotates in the Z-axis direction, the image shot by the monitoring camera is rotated according to the rotation direction and the rotation angle. The specific real-time monitoring method for the diagnosis part can refer to the related discussion of the remote ultrasonic diagnosis system, and is not repeated herein.
The ultrasonic probe of the ultrasonic scanning equipment carries out ultrasonic scanning on a diagnosis part, an ultrasonic image is obtained through processing by an ultrasonic host, and when the ultrasonic image is displayed on a diagnosis display screen of the control platform, the position of eyes of an operator is identified and positioned through arranging an eye identification camera on the control platform, so that the position relation between the eyes of the operator and the diagnosis display screen is determined; meanwhile, determining the position relation between the ultrasonic probe and the diagnosis part according to the gesture of the ultrasonic probe driven by the execution mechanical arm and the image of the diagnosis part shot by the monitoring camera, so as to determine the position relation between the ultrasonic image scanned by the ultrasonic probe and the diagnosis part; and finally, determining the position relation between the eyes of the operator and the diagnostic part image displayed on the diagnostic display screen according to the position relation between the eyes of the operator and the diagnostic display screen, carrying out projection deformation processing on the ultrasonic image by combining the position relation between the ultrasonic image and the diagnostic part, and displaying the ultrasonic image subjected to the projection deformation processing and the image of the diagnostic part on the diagnostic display screen after overlapping. The specific manner in which the projection-deformed ultrasound image and the diagnostic portion image are superimposed may refer to the above discussion about the remote ultrasound diagnostic system, and will not be described herein.
When the operation of the ultrasonic probe is controlled by using the control touch screen, the following touch gestures are adopted to control the operation of the ultrasonic probe: (1) controlling the ultrasonic probe to move in the X-axis and Y-axis directions by single-finger sliding on the control touch screen; (2) controlling the ultrasonic probe to move in the Z-axis direction by double-finger scaling on the control touch screen; (3) controlling the rotation of the ultrasonic probe along the X-axis and Y-axis directions by double-finger sliding on the control touch screen; (4) and controlling the rotation of the ultrasonic probe along the Z-axis direction by double-finger rotation on the control touch screen.
When the operator is used for controlling the action of the ultrasonic probe, an elastic feedback component is arranged at the tail end of the operator to apply an elastic force to the operator so as to simulate the elastic force of the ultrasonic probe on a diagnosis part; the elastic coefficient of the diagnosis part is detected by arranging pressure sensors at the connection parts of the tail end of the execution mechanical arm and the ultrasonic probe, so that the elastic coefficient of the elastic feedback part is adjusted; and a pressure sensor is also arranged on the elastic feedback component of the manipulator and used for detecting the pressure applied by an operator on the manipulator, so that the control execution mechanical arm controls the ultrasonic probe to be applied on the diagnosis site with the same pressure.
Further, when the manipulator is used for controlling the motion of the ultrasonic probe, the distance between the manipulator and the operation platform is detected by the laser ranging sensor arranged on the manipulator, so that the displacement of the manipulator in the Z-axis direction is judged, and the displacement of the ultrasonic probe in the Z-axis direction is controlled by the execution mechanical arm.
In combination with the image of the diagnostic part displayed on the diagnostic display screen, the operation control touch screen is used for controlling the action of the ultrasonic probe to enable the ultrasonic probe to be aligned with the diagnostic part to be subjected to ultrasonic scanning, and then the operator is also aligned with the image of the diagnostic part displayed on the operation control platform and is used for controlling the action of the ultrasonic probe, so that the operator is used for moving the diagnostic part image displayed on the operation control platform to simulate the action of ultrasonic probe moving at the diagnostic part to carry out ultrasonic scanning.
Of course, the above embodiments are only preferred embodiments of the present invention, and the scope of the present invention is not limited thereto, so that all equivalent modifications made in the principles of the present invention are included in the scope of the present invention.
Claims (4)
1. An interactive control method for remote ultrasonic diagnosis is characterized in that: the remote ultrasonic diagnosis system is adopted for remote ultrasonic diagnosis, wherein the remote ultrasonic diagnosis system comprises a control end and a remote execution end, the control end comprises a control platform and an upper computer, a diagnosis display screen, a control touch screen and a movable operator are arranged on the control platform, a triaxial gyroscope and a track sensor are arranged in the operator, the remote execution end comprises a remote communication module, an execution mechanical arm, a monitoring camera and ultrasonic scanning equipment, and the tail end of the execution mechanical arm is in driving connection with an ultrasonic probe of the monitoring camera and the ultrasonic scanning equipment; the control end and the remote execution end are in communication connection with each other through an upper computer and a remote communication module; the control platform is provided with an eye recognition camera which is used for recognizing and detecting the position of eyes of an operator in front of the control platform;
specifically, an operator sets parameters of an execution mechanical arm and ultrasonic scanning equipment by using an upper computer, controls the execution mechanical arm by using an operation touch screen or an operator before an operation platform, drives and moves an ultrasonic probe of the ultrasonic scanning equipment according to control signals of the operation touch screen and the operator, and a monitoring camera at a remote execution end acquires an image of a diagnosis part and displays the image on a diagnosis display screen of the operation platform; when the control touch screen is used for controlling the execution mechanical arm, the action of the execution mechanical arm is controlled by using different touch gestures on the control touch screen; when the manipulator is used for controlling the execution mechanical arm, the manipulator moves on the control platform, and the motion information of the manipulator is obtained by obtaining the sensing information of the track sensor and the three-axis gyroscope of the manipulator, so that the execution mechanical arm is controlled to drive the ultrasonic probe to perform the motion with the same motion of the manipulator;
when the monitoring camera collects images of the diagnosis part and displays the images on the diagnosis display screen, the control touch screen or the manipulator controls the execution mechanical arm to drive the monitoring camera to aim at the diagnosis part, then a monitoring program of the diagnosis part is started, and the images shot by the monitoring camera are processed according to the movement or rotation of the X-axis, Y-axis and Z-axis directions of the ultrasonic probe driven by the execution mechanical arm, so that the images of the diagnosis part displayed on the diagnosis display screen are kept still; when the ultrasonic probe moves in the X-axis direction and the Y-axis direction, translating an image shot by the monitoring camera according to the moving direction and the moving amount; when the ultrasonic probe moves in the Z-axis direction, scaling the image shot by the monitoring camera according to the moving direction and the moving amount; when the ultrasonic probe rotates in the X-axis and Y-axis directions, stretching and deforming an image shot by the monitoring camera according to the rotation direction and the rotation angle; when the ultrasonic probe rotates in the Z-axis direction, the image shot by the monitoring camera is rotated according to the rotation direction and the rotation angle.
2. The interactive control method for remote ultrasonic diagnosis according to claim 1, wherein: an ultrasonic probe of the ultrasonic scanning equipment carries out ultrasonic scanning on a diagnosis part, and when an ultrasonic image is obtained by processing of an ultrasonic host and is simultaneously displayed on a diagnosis display screen of a control platform, an eye recognition camera is arranged on the control platform to recognize and position the position of eyes of an operator, so that the position relationship between the eyes of the operator and the diagnosis display screen is determined; meanwhile, determining the position relation between the ultrasonic probe and the diagnosis part according to the gesture of the ultrasonic probe driven by the execution mechanical arm and the image of the diagnosis part shot by the monitoring camera, so as to determine the position relation between the ultrasonic image scanned by the ultrasonic probe and the diagnosis part; and finally, determining the position relation between the eyes of the operator and the diagnostic part image displayed on the diagnostic display screen according to the position relation between the eyes of the operator and the diagnostic display screen, carrying out projection deformation processing on the ultrasonic image by combining the position relation between the ultrasonic image and the diagnostic part, and displaying the ultrasonic image subjected to the projection deformation processing and the image of the diagnostic part on the diagnostic display screen after overlapping.
3. The interactive control method for remote ultrasonic diagnosis according to claim 1, wherein: when the operator is used for controlling the action of the ultrasonic probe, an elastic feedback component is arranged on the operator to apply an elastic force to the operator so as to simulate the elastic force of the ultrasonic probe on a diagnosis part; the elastic coefficient of the diagnosis part is detected by arranging pressure sensors at the connection parts of the tail end of the execution mechanical arm and the ultrasonic probe, so that the elastic coefficient of the elastic feedback part is adjusted; and a pressure sensor is also arranged on the elastic feedback component of the manipulator and used for detecting the pressure applied by an operator on the manipulator, so that the control execution mechanical arm controls the ultrasonic probe to be applied on the diagnosis site with the same pressure.
4. The interactive control method for remote ultrasonic diagnosis according to claim 1, wherein: the two monitoring cameras are respectively arranged at two sides of the ultrasonic probe, images of two diagnosis parts shot by the two monitoring cameras are combined in an image splicing mode, and the same picture areas are combined, so that the images of the two diagnosis parts are combined into one image and displayed on a display screen of an operation end; meanwhile, when a monitoring program for the diagnosis part is started and the ultrasonic probe moves or rotates in the X-axis, Y-axis and Z-axis directions, and the images shot by the monitoring cameras are processed, the images shot by the two monitoring cameras after the images of the two diagnosis parts are fused are processed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310139469.2A CN115869009B (en) | 2023-02-21 | 2023-02-21 | Remote ultrasonic diagnosis system and interactive control method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310139469.2A CN115869009B (en) | 2023-02-21 | 2023-02-21 | Remote ultrasonic diagnosis system and interactive control method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115869009A CN115869009A (en) | 2023-03-31 |
| CN115869009B true CN115869009B (en) | 2023-08-29 |
Family
ID=85761376
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310139469.2A Active CN115869009B (en) | 2023-02-21 | 2023-02-21 | Remote ultrasonic diagnosis system and interactive control method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115869009B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2011200533A (en) * | 2010-03-26 | 2011-10-13 | Fujifilm Corp | Ultrasonic diagnostic apparatus |
| CN102349838A (en) * | 2011-09-22 | 2012-02-15 | 东南大学 | Long-distance ultrasonic diagnosis system |
| CN206365899U (en) * | 2016-07-29 | 2017-08-01 | 深圳威瑞生物科技有限公司 | A kind of adjuvant Ultrasonographic Diagnosis system |
| CN206365898U (en) * | 2016-07-29 | 2017-08-01 | 深圳威瑞生物科技有限公司 | A kind of long-range ultrasonic wave added inspection system |
| JP2017159027A (en) * | 2016-03-07 | 2017-09-14 | 東芝メディカルシステムズ株式会社 | Ultrasonograph and ultrasonic diagnosis support device |
| CN108078585A (en) * | 2016-11-21 | 2018-05-29 | 沈阳普飞克森科技有限公司 | A kind of long-distance ultrasonic diagnosis system |
| CN108577886A (en) * | 2018-04-12 | 2018-09-28 | 叶舟 | Long-distance ultrasonic diagnosis system and application method |
| CN110477956A (en) * | 2019-09-27 | 2019-11-22 | 哈尔滨工业大学 | A kind of intelligent checking method of the robotic diagnostic system based on ultrasound image guidance |
-
2023
- 2023-02-21 CN CN202310139469.2A patent/CN115869009B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2011200533A (en) * | 2010-03-26 | 2011-10-13 | Fujifilm Corp | Ultrasonic diagnostic apparatus |
| CN102349838A (en) * | 2011-09-22 | 2012-02-15 | 东南大学 | Long-distance ultrasonic diagnosis system |
| JP2017159027A (en) * | 2016-03-07 | 2017-09-14 | 東芝メディカルシステムズ株式会社 | Ultrasonograph and ultrasonic diagnosis support device |
| CN206365899U (en) * | 2016-07-29 | 2017-08-01 | 深圳威瑞生物科技有限公司 | A kind of adjuvant Ultrasonographic Diagnosis system |
| CN206365898U (en) * | 2016-07-29 | 2017-08-01 | 深圳威瑞生物科技有限公司 | A kind of long-range ultrasonic wave added inspection system |
| CN108078585A (en) * | 2016-11-21 | 2018-05-29 | 沈阳普飞克森科技有限公司 | A kind of long-distance ultrasonic diagnosis system |
| CN108577886A (en) * | 2018-04-12 | 2018-09-28 | 叶舟 | Long-distance ultrasonic diagnosis system and application method |
| CN110477956A (en) * | 2019-09-27 | 2019-11-22 | 哈尔滨工业大学 | A kind of intelligent checking method of the robotic diagnostic system based on ultrasound image guidance |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115869009A (en) | 2023-03-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11801035B2 (en) | Systems and methods for remote graphical feedback of ultrasound scanning technique | |
| US11911214B2 (en) | System and methods for at home ultrasound imaging | |
| CN111443619B (en) | Virtual-real fused human-computer cooperation simulation method and system | |
| KR101494344B1 (en) | method and system for motion control in humanoid robot | |
| KR20180080630A (en) | Robot and electronic device for performing hand-eye calibration | |
| US20090234502A1 (en) | Apparatus for determining pickup pose of robot arm with camera | |
| WO2023061113A1 (en) | Portable remote ultrasound scanning system and compliance control method for safe ultrasound scanning | |
| KR20160138233A (en) | Surgical system with haptic feedback based upon quantitative three-dimensional imaging | |
| JP2010210585A (en) | Model display method in three-dimensional visual sensor, and three-dimensional visual sensor | |
| JPH1051711A (en) | Three-dimension virtual object display device and its method | |
| CN111166387B (en) | Method and device for ultrasonic imaging of thyroid | |
| US20230042756A1 (en) | Autonomous mobile grabbing method for mechanical arm based on visual-haptic fusion under complex illumination condition | |
| CN107945607B (en) | Ultrasonic demonstration system and device | |
| CN115998331B (en) | Probe control system and control method for remote ultrasonic diagnosis | |
| CN112109069A (en) | Robot teaching device and robot system | |
| CN115869009B (en) | Remote ultrasonic diagnosis system and interactive control method | |
| CN102768541B (en) | The control method of operating robot and system | |
| JP2019077026A (en) | Control device, robot system, and control device operating method and program | |
| CN115886870B (en) | Ultrasonic image virtual display method for remote ultrasonic diagnosis | |
| WO2023000085A1 (en) | System and apparatus for remote interaction with an object | |
| JP2020142323A (en) | Robot control device, robot control method and robot control program | |
| JP2003004666A (en) | Radiographic/fluoroscopic imaging apparatus | |
| WO2023283957A1 (en) | Specimen collection system and specimen collection method | |
| CN108319372A (en) | A kind of virtual unit and method | |
| KR101230950B1 (en) | An Haptic Apparatus and method of controlling thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CB03 | Change of inventor or designer information |
Inventor after: Xu Yihan Inventor after: Su Shudian Inventor after: Chen Yinghuai Inventor after: Zhou Guang Inventor after: Liu Hangen Inventor after: Chen Yue Inventor before: Xu Yihan Inventor before: Su Shudian Inventor before: Chen Yinghuai Inventor before: Zhou Guang Inventor before: Liu Hangen Inventor before: Chen Yue |
|
| CB03 | Change of inventor or designer information |