CN111857342A - Eye movement tracking system and method based on medical endoscope - Google Patents
Eye movement tracking system and method based on medical endoscope Download PDFInfo
- Publication number
- CN111857342A CN111857342A CN202010699498.0A CN202010699498A CN111857342A CN 111857342 A CN111857342 A CN 111857342A CN 202010699498 A CN202010699498 A CN 202010699498A CN 111857342 A CN111857342 A CN 111857342A
- Authority
- CN
- China
- Prior art keywords
- endoscope
- module
- eye
- eye movement
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/013—Eye tracking input arrangements
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00002—Operational features of endoscopes
- A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
- A61B1/00006—Operational features of endoscopes characterised by electronic signal processing of control signals
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00002—Operational features of endoscopes
- A61B1/00011—Operational features of endoscopes characterised by signal transmission
- A61B1/00016—Operational features of endoscopes characterised by signal transmission using wireless means
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/16—Sound input; Sound output
- G06F3/167—Audio in a user interface, e.g. using voice commands for navigating, audio feedback
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Surgery (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Medical Informatics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Optics & Photonics (AREA)
- Pathology (AREA)
- Radiology & Medical Imaging (AREA)
- Veterinary Medicine (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Biophysics (AREA)
- General Physics & Mathematics (AREA)
- Animal Behavior & Ethology (AREA)
- Human Computer Interaction (AREA)
- Public Health (AREA)
- Signal Processing (AREA)
- Multimedia (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Computer Networks & Wireless Communication (AREA)
- Endoscopes (AREA)
Abstract
The invention discloses an eye movement tracking system based on a medical endoscope, which comprises: the endoscope is in communication connection with the central processing module, the eye movement data acquisition module is used for acquiring eye movement data in real time and sending the eye movement data to the central processing module, the central processing module receives and processes the data and controls a control instruction of the endoscope and the control display module to display, and the wireless communication module is used for transmitting the data and the instruction with the endoscope. The invention enhances the convenience of operation, thereby improving the operation efficiency and reducing the operation risk.
Description
Technical Field
The invention relates to the field of medical image processing, in particular to an eye movement tracking system and method based on a medical endoscope.
Background
With the development of economic society, the work pace is accelerated, the environment is polluted, gastrointestinal diseases become common diseases affecting human health, such as gastric cancer, intestinal cancer and the like, and are mostly common diseases and frequently encountered diseases, which have great threat to human health, and capsule endoscope systems are widely applied as effective means for judging gastrointestinal diseases due to the advantages of high safety, high reliability and the like. Many of the gastrointestinal disorders are accompanied by bleeding. By combining the computer technology and the image recognition technology, the bleeding images in the image sequence are accurately detected, and the capsule endoscope system can assist doctors in diagnosis in gastrointestinal tract examination, so that the diagnosis efficiency is improved, and the workload of the doctors is reduced.
The current medical endoscope monitoring system technology mainly focuses on the design of the handheld device, and does not consider that the vision switching and the manual operation are both incompetent when a user operates the endoscope and the handheld device simultaneously in the operation process. Because the sight of the user needs to pay attention to the wound condition of the patient, observe the operation condition of the endoscope and frequently turn the head to view the video display picture on the monitoring equipment to observe the video condition collected by the endoscope, the continuous switching of the sight causes a certain time delay in the operation process, and meanwhile, the switching of the sight also makes the user difficult to effectively combine the wound condition of the patient and the video condition collected by the endoscope for observation, so the current endoscope system affects the operation quality of the operation, prolongs the operation time and increases the risk of the operation.
Therefore, there is a need to develop an eye tracking system and method based on medical endoscope.
Disclosure of Invention
In order to solve at least one technical problem, the invention provides an eye tracking system and method based on a medical endoscope.
In order to solve the above-mentioned technical problem, a first aspect of the present invention discloses an eye tracking system based on a medical endoscope, comprising: the endoscope is in communication connection with the central processing module, the eye movement data acquisition module is used for acquiring eye movement data in real time and sending the eye movement data to the central processing module, the central processing module receives and processes the data and controls a control instruction of the endoscope and the control display module to display, and the wireless communication module is used for transmitting the data and the instruction with the endoscope.
In the scheme, the display module is an interactive touch screen display module, the display module is used for displaying a detection picture of the endoscope and displaying eye movement data processed by the central processing module as an operation pointer, and the operation pointer indicates the fixation point of eyeballs of the current operator.
In this scheme, eye movement tracking system still includes voice control module, voice control module with central processing module electric connection, voice control module is used for carrying out the speech control endoscope.
In this scheme, eye movement data acquisition module is independent wearable eye movement data acquisition device.
In this embodiment, the eye tracking system further includes a data storage module for storing image data transmitted back from the endoscope and instruction data of the central processing module.
In a second aspect, the present invention provides a medical endoscope-based eye tracking method, comprising the steps of:
obtaining eye movement data and performing fixation point filtering processing to obtain an effective fixation point;
establishing a screen coordinate system in an endoscope image display area in a display module, and determining the position of an effective fixation point in the endoscope image display area by taking the upper left corner of the endoscope image display area as a coordinate origin;
The position information of the effective gazing point is sent to the endoscope as an instruction to control the movement of the endoscope.
In this scheme, the specific process of obtaining the effective gaze point by the gaze point filtering process is as follows:
setting a filtering time threshold, and calculating the number N of data samples of the eye movement data sampling module within the length of the filtering time threshold, wherein N is a positive integer;
and taking the average value of the horizontal coordinates and the vertical coordinates of the N sampling data as the coordinate value of the effective fixation point.
In this scheme, the timestamp of the effective fixation point is the timestamp of the nth sampling point.
In the scheme, if the effective point of regard is an interactive menu area in the display module, the corresponding menu item is sent to the endoscope in the form of an instruction.
In this scheme, the interactive menu area includes: eye control magnification, eye control reduction and eye control original image display.
The system comprises an eye movement data acquisition module, a central processing module, a display module, a wireless communication module and an endoscope, and the endoscope is controlled by the central processing module through acquiring eye movement data, so that the convenience of operation is enhanced, the operation efficiency can be improved, and the operation risk can be reduced.
Drawings
Fig. 1 shows a block diagram of a first medical endoscope-based eye tracking system provided by an embodiment of the present invention.
Fig. 2 illustrates a second medical endoscope-based eye tracking system provided by an embodiment of the present invention.
Fig. 3 shows a flow chart of a medical endoscope-based eye tracking method.
Detailed description of the invention
In order that the above objects, features and advantages of the present invention can be more clearly understood, a more particular description of the invention will be rendered by reference to the appended drawings. It should be noted that the embodiments and features of the embodiments of the present application may be combined with each other without conflict.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, however, the present invention may be practiced in other ways than those specifically described herein, and therefore the scope of the present invention is not limited by the specific embodiments disclosed below.
The eye movement tracking method based on the medical endoscope is mainly suitable for medical endoscope systems, such as a hard tube endoscope, a fiber endoscope and an electronic endoscope, the endoscope is controlled through the central processing module by acquiring the eye movement data, the convenience of operation is enhanced, the operation efficiency can be improved, and the operation risk can be reduced. Of course, the present invention is not limited to the type of endoscope used in the room, and any technical solution adopting the present invention will fall into the protection scope of the present invention.
Fig. 1 shows a block diagram of a medical endoscope-based eye tracking system of the present invention.
As shown in fig. 1, a first aspect of the present invention discloses an eye tracking system based on a medical endoscope, comprising: the endoscope is in communication connection with the central processing module, the eye movement data acquisition module is used for acquiring eye movement data in real time and sending the eye movement data to the central processing module, the central processing module receives and processes the data and controls a control instruction of the endoscope and the control display module to display, and the wireless communication module is used for transmitting the data and the instruction with the endoscope.
In a specific embodiment, the eye movement data acquisition module may be an existing eye tracker, which may be used to acquire eye movement data of a user when detecting and tracking the movement of the user's eye, and the eye tracker may employ remote non-invasive tracking, and the basic principle is to illuminate the eye with a light source and cause a highly visible reflection, then capture and display a reflected spectacle image with a camera, and the reflected image identifies the reflection of the light source in the cornea and pupil, and further calculate the vector formed by the included angle of the corneal pupil reflection, and further calculate the gazing direction. Central processing module can adopt DSP processing chip or STM processing chip, wireless communication module can adopt wifi wireless communication module or bluetooth wireless communication module, the endoscope can adopt hard tube formula endoscope, fibre endoscope, electronic endoscope, fibre endoscope: the endoscope consists of an endoscope body and a cold light source, wherein two optical fiber bundles are arranged in the endoscope body, one optical fiber bundle is called as a light beam, the other optical fiber bundle is called as an image bundle, one end of the optical fiber bundle is aligned to an ocular, the other end of the optical fiber bundle is aligned to the surface of an observed object through an objective lens, and a doctor can visually see the surface condition of the visceral organs through the ocular, so that the condition of the visceral organs can be diagnosed conveniently and accurately.
The electronic endoscope is characterized in that: the electronic endoscope is not used for image transmission any more, but is replaced by a photosensitive integrated circuit camera system, and not only has good image quality, strong brightness and large image, but also can detect finer pathological changes, and the electronic endoscope has thinner outer diameter, clearer and more visual image and convenient operation. A capsule type endoscope may be preferably employed in the present invention.
As shown in fig. 2, in this scheme, the display module is an interactive touch screen display module, and the display module is configured to display a detection picture of an endoscope and display eye movement data processed by the central processing module as an operation pointer, where the operation pointer indicates a gaze point of an eyeball of a current operator.
In this scheme, eye movement tracking system still includes voice control module, voice control module with central processing module electric connection, voice control module is used for carrying out the speech control endoscope.
In a specific embodiment, the voice interaction module comprises a sound collection module and an audio processing module; the voice data output end of the voice acquisition module is electrically connected with the audio input end of the audio processing module, and the audio output end of the audio processing module is electrically connected with the voice data input end of the central processing module; the sound acquisition module is used for carrying out audio acquisition on environmental sound and transmitting the acquired original sound audio data to the audio processing module in real time through the sound data output end; the audio processing module is used for acquiring original sound audio data from the sound acquisition module in real time, performing noise reduction and filtering processing on the original sound audio data, transmitting the processed sound audio data to the central processing module in real time through the audio output end, and converting a voice instruction into a control instruction by the central processing module to send the control instruction to the endoscope by an operator in a specific system by sending the voice instruction.
In this scheme, eye movement data acquisition module is independent wearable eye movement data acquisition device.
In a specific embodiment, the independent wearable eye movement data acquisition device can be a headset-based acquisition device or an acquisition device given to glasses, wherein the independent wearable eye movement data acquisition device supplies power to a separate power supply, such as a lithium battery, and is provided with a wireless data transmission module.
In this embodiment, the eye tracking system further includes a data storage module for storing image data transmitted back from the endoscope and instruction data of the central processing module. The data storage module can be arranged to carry out historical query or review on image data returned by the endoscope and instruction data of the central processing module.
Fig. 3 shows a flowchart of an eye tracking method based on a medical endoscope.
The invention provides a medical endoscope-based eye movement tracking method, which is applied to the medical endoscope-based eye movement tracking system and comprises the following steps: the system comprises an eye movement data acquisition module, a central processing module, a display module, a wireless communication module and an endoscope, wherein the eye movement data acquisition module, the display module and the wireless communication module are electrically connected with the central processing module, the endoscope is in communication connection with the central processing module, the eye movement data acquisition module is used for acquiring eye movement data in real time and sending the eye movement data to the central processing module, the central processing module receives and processes the data and controls a control instruction of the endoscope to be sent out and the display module to be controlled to display, and the wireless communication module is used for transmitting the data and the instruction with the endoscope, and the method comprises the following steps:
S302, eye movement data are obtained, and a fixation point is filtered to obtain an effective fixation point;
s304, establishing a screen coordinate system in an endoscope image display area in a display module, and determining the position of an effective fixation point in the endoscope image display area by taking the upper left corner of the endoscope image display area as a coordinate origin;
the position information of the effective gazing point is sent to the endoscope as an instruction to control the movement of the endoscope.
In one particular embodiment, the gaze direction may be calculated by an eye tracker that may be used to acquire eye movement data of a user while detecting and tracking the movement of the user's eye, which may employ remote non-invasive tracking, the basic principle being that a light source is used to illuminate the eye and cause a highly visible reflection, then a camera is used to capture an image of the glasses displaying the reflection, and the reflected image identifies the reflection of the light source in the cornea and pupil, and then the vector formed by the angle of the corneal pupil reflection is calculated, and further the gaze direction is calculated. It should be noted that, in the present invention, calibration of data points is performed before each time of obtaining eye movement data, for example, different calibration points are set at the upper, lower, left, right, and each corner of a display module of the system, an operator watches the calibration points on the screen, then the eye tracker measures and calculates the geometric features of the eyes for accurately calculating the falling point of the sight line of the operator on the screen, and the calibration quality is checked after the calibration is completed. By the alignment, the deviation can be reduced, thereby improving the accuracy of eye tracking.
In a specific embodiment, the eye movement data acquisition module may be an existing eye tracker, which may be used to acquire eye movement data of a user when detecting and tracking the movement of the user's eye, and the eye tracker may employ remote non-invasive tracking, and the basic principle is to illuminate the eye with a light source and cause a highly visible reflection, then capture and display a reflected spectacle image with a camera, and the reflected image identifies the reflection of the light source in the cornea and pupil, and further calculate the vector formed by the included angle of the corneal pupil reflection, and further calculate the gazing direction. Central processing module can adopt DSP processing chip or STM processing chip, wireless communication module can adopt wifi wireless communication module or bluetooth wireless communication module, the endoscope can adopt hard tube formula endoscope, fibre endoscope, electronic endoscope, fibre endoscope: the endoscope consists of an endoscope body and a cold light source, wherein two optical fiber bundles are arranged in the endoscope body, one optical fiber bundle is called as a light beam, the other optical fiber bundle is called as an image bundle, one end of the optical fiber bundle is aligned to an ocular, the other end of the optical fiber bundle is aligned to the surface of an observed object through an objective lens, and a doctor can visually see the surface condition of the visceral organs through the ocular, so that the condition of the visceral organs can be diagnosed conveniently and accurately.
The electronic endoscope is characterized in that: the electronic endoscope is not used for image transmission any more, but is replaced by a photosensitive integrated circuit camera system, and not only has good image quality, strong brightness and large image, but also can detect finer pathological changes, and the electronic endoscope has thinner outer diameter, clearer and more visual image and convenient operation. A capsule type endoscope may be preferably employed in the present invention.
In the scheme, the display module is an interactive touch screen display module, the display module is used for displaying a detection picture of the endoscope and displaying eye movement data processed by the central processing module as an operation pointer, and the operation pointer indicates the fixation point of eyeballs of the current operator.
In this scheme, eye movement tracking system still includes voice control module, voice control module with central processing module electric connection, voice control module is used for carrying out the speech control endoscope.
In a specific embodiment, the voice interaction module comprises a sound collection module and an audio processing module; the voice data output end of the voice acquisition module is electrically connected with the audio input end of the audio processing module, and the audio output end of the audio processing module is electrically connected with the voice data input end of the central processing module; the sound acquisition module is used for carrying out audio acquisition on environmental sound and transmitting the acquired original sound audio data to the audio processing module in real time through the sound data output end; the audio processing module is used for acquiring original sound audio data from the sound acquisition module in real time, performing noise reduction and filtering processing on the original sound audio data, transmitting the processed sound audio data to the central processing module in real time through the audio output end, and converting a voice instruction into a control instruction by the central processing module to send the control instruction to the endoscope by an operator in a specific system by sending the voice instruction.
In this scheme, eye movement data acquisition module is independent wearable eye movement data acquisition device.
In a specific embodiment, the independent wearable eye movement data acquisition device can be a headset-based acquisition device or an acquisition device given to glasses, wherein the independent wearable eye movement data acquisition device supplies power to a separate power supply, such as a lithium battery, and is provided with a wireless data transmission module.
In this embodiment, the eye tracking system further includes a data storage module for storing image data transmitted back from the endoscope and instruction data of the central processing module. The data storage module can be arranged to carry out historical query or review on image data returned by the endoscope and instruction data of the central processing module.
In a specific embodiment, the eye movement data acquisition devices have sampling frequencies, for example, the working frequency is 60Hz, a sampling point is generated every 16.7 seconds, and continuous sampling points are observed within a period of time, so that the annotation data is acquired continuously at high frequency. To avoid the interaction feedback delay of the tracking, it is therefore necessary to reduce the update frequency of the initial point. In the present invention, a gaze point filtering algorithm is used for the past.
In this scheme, the specific process of obtaining the effective gaze point by the gaze point filtering process is as follows:
setting a filtering time threshold, and calculating the number N of data samples of the eye movement data sampling module within the length of the filtering time threshold, wherein N is a positive integer;
and taking the average value of the horizontal coordinates and the vertical coordinates of the N sampling data as the coordinate value of the effective fixation point.
In a specific embodiment, 200ms may be set as a time threshold for filtering, and if a fixation is less than 200ms, its fixation point is filtered out, and if it is more than 200ms, a valid fixation point is extracted from every 200ms fixation point. The invention takes the average value of all the fixation point coordinates in 200 milliseconds fixation as the coordinates of the effective fixation point, and takes the timestamp of the last fixation point in the fixation points as the timestamp of the effective fixation point. For example, an operating frequency of 60Hz and a gaze duration of 60Hz
200ms, the number threshold of the fixation points is 12, and the coordinate calculation formula of the effective fixation points is as follows:
wherein the content of the first and second substances,
abscissa and ordinate, X, representing the effective point of gaze1、X2、XNAbscissa, Y, representing point of gaze1、Y2、YNThe ordinate of the gaze point.
In this scheme, the timestamp of the effective fixation point is the timestamp of the nth sampling point.
In the scheme, if the effective point of regard is an interactive menu area in the display module, the corresponding menu item is sent to the endoscope in the form of an instruction.
In this scheme, the interactive menu area includes: eye control magnification, eye control reduction and eye control original image display. In a specific embodiment, the interactive menu area includes, but is not limited to, the above menu items, which can be set according to actual requirements.
The third aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium includes a medical endoscope-based eye tracking method program, and when the medical endoscope-based eye tracking method program is executed by a processor, the method for implementing the medical endoscope-based eye tracking method comprises the following steps:
obtaining eye movement data and performing fixation point filtering processing to obtain an effective fixation point;
establishing a screen coordinate system in an endoscope image display area in a display module, and determining the position of an effective fixation point in the endoscope image display area by taking the upper left corner of the endoscope image display area as a coordinate origin;
the position information of the effective gazing point is sent to the endoscope as an instruction to control the movement of the endoscope.
In one particular embodiment, the gaze direction may be calculated by an eye tracker that may be used to acquire eye movement data of a user while detecting and tracking the movement of the user's eye, which may employ remote non-invasive tracking, the basic principle being that a light source is used to illuminate the eye and cause a highly visible reflection, then a camera is used to capture an image of the glasses displaying the reflection, and the reflected image identifies the reflection of the light source in the cornea and pupil, and then the vector formed by the angle of the corneal pupil reflection is calculated, and further the gaze direction is calculated. It should be noted that, in the present invention, calibration of data points is performed before each time of obtaining eye movement data, for example, different calibration points are set at the upper, lower, left, right, and each corner of a display module of the system, an operator watches the calibration points on the screen, then the eye tracker measures and calculates the geometric features of the eyes for accurately calculating the falling point of the sight line of the operator on the screen, and the calibration quality is checked after the calibration is completed. By the alignment, the deviation can be reduced, thereby improving the accuracy of eye tracking.
In a specific embodiment, the eye movement data acquisition devices have sampling frequencies, for example, the working frequency is 60Hz, a sampling point is generated every 16.7 seconds, and continuous sampling points are observed within a period of time, so that the annotation data is acquired continuously at high frequency. To avoid the interaction feedback delay of the tracking, it is therefore necessary to reduce the update frequency of the initial point. In the present invention, a gaze point filtering algorithm is used for the past.
In this scheme, the specific process of obtaining the effective gaze point by the gaze point filtering process is as follows:
setting a filtering time threshold, and calculating the number N of data samples of the eye movement data sampling module within the length of the filtering time threshold, wherein N is a positive integer;
and taking the average value of the horizontal coordinates and the vertical coordinates of the N sampling data as the coordinate value of the effective fixation point.
In a specific embodiment, 200ms may be set as a time threshold for filtering, and if a fixation is less than 200ms, its fixation point is filtered out, and if it is more than 200ms, a valid fixation point is extracted from every 200ms fixation point. The invention takes the average value of all the fixation point coordinates in 200 milliseconds fixation as the coordinates of the effective fixation point, and takes the timestamp of the last fixation point in the fixation points as the timestamp of the effective fixation point. For example, an operating frequency of 60Hz and a gaze duration of 60Hz
200ms, the number threshold of the fixation points is 12, and the coordinate calculation formula of the effective fixation points is as follows:
wherein the content of the first and second substances,
abscissa and ordinate, X, representing the effective point of gaze1、X2、XNAbscissa, Y, representing point of gaze1、Y2、YNThe ordinate of the gaze point.
In this scheme, the timestamp of the effective fixation point is the timestamp of the nth sampling point.
In the scheme, if the effective point of regard is an interactive menu area in the display module, the corresponding menu item is sent to the endoscope in the form of an instruction.
In order to better explain the technical solution of the present invention, the following will describe in detail the specific steps of the indoor teaching action analysis and correction method through several embodiments.
The system comprises an eye movement data acquisition module, a central processing module, a display module, a wireless communication module and an endoscope, and the endoscope is controlled by the central processing module through acquiring eye movement data, so that the convenience of operation is enhanced, the operation efficiency can be improved, and the operation risk can be reduced.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described device embodiments are merely illustrative, for example, the division of the unit is only a logical functional division, and there may be other division ways in actual implementation, such as: multiple units or components may be combined, or may be integrated into another system, or some features may be omitted, or not implemented. In addition, the coupling, direct coupling or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between the devices or units may be electrical, mechanical or other forms.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units; can be located in one place or distributed on a plurality of network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, all the functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately regarded as one unit, or two or more units may be integrated into one unit; the integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.
Those of ordinary skill in the art will understand that: all or part of the steps for realizing the method embodiments can be completed by hardware related to program instructions, the program can be stored in a computer readable storage medium, and the program executes the steps comprising the method embodiments when executed; and the aforementioned storage medium includes: a mobile storage device, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
Alternatively, the integrated unit of the present invention may be stored in a computer-readable storage medium if it is implemented in the form of a software functional module and sold or used as a separate product. Based on such understanding, the technical solutions of the embodiments of the present invention may be essentially implemented or a part contributing to the prior art may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the methods described in the embodiments of the present invention. And the aforementioned storage medium includes: a removable storage device, a ROM, a RAM, a magnetic or optical disk, or various other media that can store program code.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the appended claims.
Claims (10)
1. An eye tracking system based on a medical endoscope, comprising: the endoscope is in communication connection with the central processing module, the eye movement data acquisition module is used for acquiring eye movement data in real time and sending the eye movement data to the central processing module, the central processing module receives and processes the data and controls a control instruction of the endoscope and the control display module to display, and the wireless communication module is used for transmitting the data and the instruction with the endoscope.
2. The medical endoscope-based eye tracking system according to claim 1, wherein the display module is an interactive touch screen display module, and the display module is configured to display a detection picture of the endoscope and display eye movement data processed by the central processing module as an operation pointer, and the operation pointer indicates a fixation point of an eyeball of a current operator.
3. The medical endoscope-based eye tracking system of claim 1, further comprising a voice control module, wherein the voice control module is electrically connected to the central processing module, and the voice control module is configured to control the endoscope with voice.
4. The medical endoscope-based eye movement tracking system according to claim 1, wherein said eye movement data acquisition module is a self-contained wearable eye movement data acquisition device.
5. The medical endoscope-based eye tracking system according to claim 1, further comprising a data storage module for storing image data transmitted back from the endoscope and command data from the central processing module.
6. An eye tracking method based on a medical endoscope, the method comprising the steps of:
obtaining eye movement data and performing fixation point filtering processing to obtain an effective fixation point;
establishing a screen coordinate system in an endoscope image display area in a display module, and determining the position of an effective fixation point in the endoscope image display area by taking the upper left corner of the endoscope image display area as a coordinate origin;
the position information of the effective gazing point is sent to the endoscope as an instruction to control the movement of the endoscope.
7. The eye movement tracking method based on the medical endoscope, as claimed in claim 6, wherein the specific process of the gaze point filtering process to obtain the effective gaze point is as follows:
Setting a filtering time threshold, and calculating the number N of data samples of the eye movement data sampling module within the length of the filtering time threshold, wherein N is a positive integer;
and taking the average value of the horizontal coordinates and the vertical coordinates of the N sampling data as the coordinate value of the effective fixation point.
8. The medical endoscope-based eye-tracking method according to claim 7, wherein the timestamp of the effective fixation point is the timestamp of the nth sampling point.
9. The medical endoscope-based eye tracking method according to claim 6, wherein if the effective fixation point is an interactive menu area in the display module, the corresponding menu item is sent to the endoscope in the form of an instruction.
10. The medical endoscope-based eye tracking method according to claim 6, wherein the interactive menu area comprises: eye control magnification, eye control reduction and eye control original image display.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010699498.0A CN111857342A (en) | 2020-07-20 | 2020-07-20 | Eye movement tracking system and method based on medical endoscope |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010699498.0A CN111857342A (en) | 2020-07-20 | 2020-07-20 | Eye movement tracking system and method based on medical endoscope |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111857342A true CN111857342A (en) | 2020-10-30 |
Family
ID=73001794
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010699498.0A Pending CN111857342A (en) | 2020-07-20 | 2020-07-20 | Eye movement tracking system and method based on medical endoscope |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111857342A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114035673A (en) * | 2020-11-18 | 2022-02-11 | 武汉联影医疗科技有限公司 | Ultrasonic eye movement control system and method |
-
2020
- 2020-07-20 CN CN202010699498.0A patent/CN111857342A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114035673A (en) * | 2020-11-18 | 2022-02-11 | 武汉联影医疗科技有限公司 | Ultrasonic eye movement control system and method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11547520B2 (en) | Systems, methods, and computer-readable storage media for controlling aspects of a robotic surgical device and viewer adaptive stereoscopic display | |
| US20210157403A1 (en) | Operating room and surgical site awareness | |
| RU2740259C2 (en) | Ultrasonic imaging sensor positioning | |
| US20230029639A1 (en) | Medical device system for remote monitoring and inspection | |
| CN109645955B (en) | Multifunctional visual function detection device and method based on VR and eye movement tracking | |
| US10600183B2 (en) | Detection of blood vessels | |
| CN106456148A (en) | Medical devices, systems, and methods using eye gaze tracking | |
| US9119570B2 (en) | Eyeball movement monitoring method and device | |
| US9888842B2 (en) | Medical diagnostic gaze tracker | |
| US10706965B2 (en) | Sensor information acquiring device, sensor information acquiring method, recording medium in which sensor information acquiring program is recorded, and medical instrument | |
| KR101580559B1 (en) | Medical image and information real time interaction transfer and remote assist system | |
| US20220071484A1 (en) | Virtual reality-based portable nystagmography device and diagnostic test method using same | |
| Oyekoya et al. | Eye tracking as a new interface for image retrieval | |
| CN111857342A (en) | Eye movement tracking system and method based on medical endoscope | |
| CN114931353A (en) | Convenient and fast contrast sensitivity detection system | |
| WO2019213836A1 (en) | Magnetic resonance based multi-parameter testee monitoring method and monitoring system | |
| WO2024038134A1 (en) | Methods and devices in performing a vision testing procedure on a person | |
| JP2003319907A (en) | Eye ball movement analysis system and eye ball imaging apparatus | |
| US10854005B2 (en) | Visualization of ultrasound images in physical space | |
| KR20180133632A (en) | System of providing augmented reality image for endoscope image diagnosis | |
| CN113633257A (en) | Vestibular function examination method, system, device and medium based on virtual reality | |
| CN114740966A (en) | Multi-modal image display control method and system and computer equipment | |
| CN212853514U (en) | Augmented reality observation device for ultrasound device | |
| JP7345866B2 (en) | Information processing system, information processing method and program | |
| US20230210363A1 (en) | Infrared tele-video-oculography for remote evaluation of eye movements |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |